Systematic killing has long been associated with some of the darkest episodes in human history. Increasingly, however, it is framed as a desirable outcome in war, particularly in the context of military AI and lethal autonomy. Autonomous weapons systems, defenders argue, will surpass humans not only militarily but also morally, enabling a more precise and dispassionate mode of violence, free of the emotion and uncertainty that too often weaken compliance with the rules and standards of war. We contest this framing. Drawing on the history of systematic killing, we argue that lethal autonomous weapons systems reproduce, and in some cases intensify, the moral challenges of the past. Autonomous violence incentivizes a moral devaluation of those targeted and erodes the moral agency of those who kill. Both outcomes imperil essential restraints on the use of military force.

This article explains why regulating autonomy in weapons systems, entailing the codification of a legally binding obligation to retain meaningful human control over the use of force, is such a challenging task within the framework of the United Nations Convention on Certain Conventional Weapons. It is difficult because it requires new diplomatic language, and because the military value of weapon autonomy is hard to forego in the current arms control winter. The article argues that regulation is nevertheless imperative, because the strategic as well as ethical risks outweigh the military benefits of unshackled weapon autonomy. To this end, it offers some thoughts on how the implementation of regulation can be expedited.

The Great War ushered in a new era of long-distance combat. For the first time, weapons with a very long range were massively deployed, in previously unheard-of places: under the sea and in the air. Stealth fighting also included espionage and propaganda, now orchestrated on a global scale. In reaction to the carnage in the trenches, a degree of moral rehabilitation came to be conferred on the weapons initially associated with a “cowards’ war”. This in turn encouraged experimentation with the new, unmanned technology that would lead to the first prototypes of guided munitions and drones.