Autonomous Weapons: Is an Arms Race Really a Threat?

There has certainly been much written about the controversy over autonomous weapons systems, but in my preparation for a Chatham House conference on autonomous weapons, I found one argument made by advocates of a ban on such weapons, however,  that merits some close examination.  These advocates make the point that there will be a robotics arms race that will result in development and deployment of autonomous weapons even if these weapons are not able to comply with international law.  For example, here is a Human Rights Watch statement: “But the temptation will grow to acquire fully autonomous weapons, also known as ‘lethal autonomous robotics’ or ‘killer robots.’ If one nation acquires these weapons, others may feel they have to follow suit to avoid falling behind in a robotic arms race.”

In essence, Human Rights Watch is arguing that even nations, like the United States, that are taking a very cautious approach to autonomous weapons will feel compelled to deploy these weapons for fear of losing a military advantage.  The result will be the deployment of these weapons despite the fact that the technology does not really ensure compliance with international law.

This is a powerful argument except for one fatal flaw: a robotic weapon that cannot meet international norms is unlikely to have a military advantage on the battlefield.

Under well established principles of international law, every targeting decision in war requires a careful set of judgments that are now done by human beings:  Is this target a legitimate military target?  Will there be harm to civilians from the strike?  Is the value of the military target nonetheless proportional to this harm?  As much as progress has been made in robotics, it is unlikely that any autonomous robot in even the near future would have the capacity to determine military targets from civilians with any accuracy or make the critical judgment about the proportionality of military value to civilian harm.

Would deployment of even these inadequate autonomous weapons provide an advantage on the battlefield?  Even if these weapons will have difficulty distinguishing a civilian target from a legitimate military target, would they provide a military advantage over weapons still controlled by humans?  I doubt it.

Effectiveness on the battlefield actually requires a higher degree of judgment than that required to meet international legal requirements.  It is not enough to hit a legitimate target.  Effectiveness on the battlefield requires that a weapon hit the most important targets and in the right sequence.  A computer that even has difficulty making judgments about what is a legitimate target will not do well making the more challenging tactical and operational decisions required on the battlefield.

In addition, an autonomous weapon that can’t easily distinguish civilians from military targets can all too easily be fooled by an enemy.  Sun Tzu famously wrote that all warfare is based on deception and deception techniques (such as decoys) were used quite effectively by Serbia against the NATO air campaign in Kosovo.


Perhaps the best evidence that there will be no robotic arms race is the fact that no major military power is rushing to develop or deploy these weapons.  For example, while there is certainly a great deal of research activity on autonomous systems, there is no current DoD program of record for any autonomous weapon. DoD is showing great caution in the development of autonomous weapons not merely out of concern for international law.  While that is obviously a significant concern, there is also great skepticism that purely autonomous weapons will provide a military advantage even in the battle spaces twenty or more years in the future.

In short, an autonomous weapon that cannot satisfy the laws of war is unlikely to be an effective weapon on the battlefield.  Concerns about  robotic arms race are misplaced.

this post was previously published on the Lawfare Blog.

Should We Ban Autonomous Weapons?

It sounds like something right out of a blockbuster science fiction movie: killer robots that make decisions on who to kill without any human involvement. Not surprisingly, several human rights groups have argued that now is the time for a ban on the development and deployment of these weapons. While there are very real ethical and legal concerns with these potential weapon systems, such a ban is both unnecessary and likely counterproductive.

There are very serious legal concerns with the use of any autonomous weapon. Under well established principles of international law, every targeting decision in war requires a careful set of judgments that are currently made by human beings: Is this target a legitimate military target? Will there be harm to civilians from the strike? Is the value of the military target nonetheless proportional to this harm?

Great progress has been made in robotics, but it is unlikely that any autonomous robot now or in the near future would have the capacity to distinguish military targets from civilians with any accuracy or make the legally critical judgment about the proportionality of military value to civilian harm.

This is true even on battlefields where there are fewer risks of civilian casualties – such the as use of robots to attack underwater submarines, or in strictly machine on machine fights such as missile defence or defence against unmanned drones. We are even further away from machines that can tell the difference between military and civilian targets in much more difficult environments, such as against an un-uniformed enemy in an urban setting.

For these reasons, the official U.S. Department of Defense policy is that autonomous weapon systems can only be used to apply non-lethal, non-kinetic force (such as forms of electronic attack) unless, among other requirements, senior DoD leadership is convinced after rigorous testing that the system will comply with international law.

Even setting aside the legal issues, the limitations of current technology also make autonomous weapons ineffective as a weapon. If a human can do a better job hitting the right target, militaries won’t want to deploy autonomous systems. Indeed, while the US military is certainly doing research on autonomous systems, there are no current plans to acquire or use autonomous lethal weapons.

Given that the technology has not developed sufficiently to field machines that satisfy either international legal requirements or military operational needs, we already effectively have a moratorium in place on the deployment of these systems. But some would say that if deployment of these systems would be unlawful today, why not move forward in imposing a on a ban on the further development and deployment of autonomous systems in the future?

The reason is that such a ban on development would either be ineffective, or would stifle peaceful uses of robotics and artificial intelligence.  The vast majority of research done today in developing autonomous systems is being done by industry and academia with an focus on peaceful, not military uses.  Some prominent examples include efforts to develop self-driving cars, search and rescue robots, and even surgical robots.  The technology needed for these peaceful uses, however, would be directly applicable to lethal military uses.  All require greatly improved sensing technology and advances in artificial intelligence – exactly what would be necessary and useful in military applications. If a moratorium were imposed, we would either have an ineffective ban on the development of military technology or an unfortunate ban on technologies that could greatly improve our lives.

In addition, a ban could some day prevent the use of technologies that actually reduce civilian casualties. Useful military deployment of autonomous systems will require a greater degree of capability than that required to comply with international law. To win a battle, it is not enough that a machine can hit a lawful target.  Instead, military success requires careful judgments about which targets to hit and in what order. And a machine that doesn’t do a good job of distinguishing civilians from soldiers can be too easily fooled by an enemy.  As such, until autonomous weapon systems reach a capacity well beyond that needed to comply with international law, military considerations  will mean that weapons will stay under human control.

This has implications for civilian casualties.  If robotics technologies reach the point at which they become militarily useful, they will also likely be more capable than humans of distinguishing between civilian and military targets. If that is the case, deployment of autonomous systems could have the effect of reducing civilian casualties, and increasing compliance with international law. As the last ten years have shown, even well-disciplined soldiers can make serious errors – particularly in the heat of battle.  The result of this human error has been the death and maiming of civilians.  While we can’t predict the future of robotics, do we really want to ban weapon systems that potentially could be less likely to cause harm to civilians?

Autonomous weapons must be treated with great caution, and the international community needs to raise the alarm when these systems are used before the technology ensures compliance with international law. There needs to be a discussion about the norms that must be followed before any deployment. A ban, however, is not the right answer.

Originally posted on the Reuters (UK) Great Debate blog.