Jacob Moore

During the writing of this paper, the San Francisco city council voted 8-3 to allow its police department to use lethal robots.¹ The question of whether we have moral reason to regulate the development and use of robots designed for combat, lethal autonomous weapon systems (hereafter LAWS), has become more prescient. I will argue in this paper that we do. The argument for this is as follows:

P1: Developing and using LAWS allows for small groups² of people to hold disproportionate capacity for directed violence.

P2: Capacity for directed violence is sufficient for political power.

C1: Developing and using LAWS allows for small groups of people to hold disproportionate political power.

P3: We have moral reason to regulate mechanisms that allow for small groups of people to hold disproportionate political power.

C2: We have moral reason to regulate the development and use of LAWS.

To motivate this argument, I will first explain some helpful terminology, including a specific definition of LAWS. Then I will justify each premise of my argument in turn. Finally, I will anticipate some objections to my argument, and provide responses to them.

There is no commonly agreed-upon definition of a LAWS, as experts disagree about what the threshold for autonomy should be. The argument I am making can remain almost entirely agnostic about where this threshold lies. As such, I will define a weapon system as lethal and autonomous if, and only if, it can be activated, then determine its own targets  and execute a “kill decision,” both without further human interaction. A kill decision is simply a process that determines whether a given target should receive lethal force and then action matching this determination. In humans, this process is cognitive; in LAWS, it is algorithmic. The criteria for target selection and the “kill decision” can either be given by a human or otherwise generated by the LAWS.³ I will now introduce two terms relevant to my argument: directed violence and gross violence. Directed violence is the precision with which the targets of a weapon system can be targeted. Gross violence can be defined as  simply the raw amount of harm that can be done by a weapon system. Gross violence  takes no account of the ability of a weapon system to achieve a secondary aim beyond harm. For example, a suicide bomber may have more capacity for gross violence than someone with an assault rifle, but the person with the assault rifle would have a larger capacity for directed violence, due to the imprecise nature of the suicide bomber’s explosive.

Developing and using LAWS allows for small groups of people to hold a disproportionate capacity for directed violence. This capacity is disproportionate in that, without the use of LAWS, an equivalent capacity for directed violence would require more human interaction and thus more people. A helpful way of roughly quantifying directed violence is to evaluate the amount of “kill decisions” a weapon system can execute in a given time period. For conventional weapons⁴ to have a large capacity for directed violence, a large number of humans are needed to select targets and execute “kill decisions.” Since LAWS are defined by their ability to perform such acts without human interaction, less, or possibly even no humans are needed.⁵

Imagine three scenarios. In the first, a crazed tyrant possesses a nuclear arsenal, which she alone controls from a remote device that can aim and fire warheads. In the second, that same crazed tyrant possesses a vast arsenal of non-nuclear, conventional weapons, from tanks to guns to combat aircraft. In a third scenario, she possesses a network of LAWS, to which she can issue directives with varying levels of specificity.⁶ Suppose this dictator wants to maintain an iron grip over her nation in all these scenarios. In the first scenario, the dictator has an immense capacity for gross violence per “kill decision,” but very little capacity for directed violence, as nuclear weapons lack the ability to precisely select targets. In the second, the dictator alone possesses very little capacity for directed violence. This is because non-nuclear conventional weapons have a much lower capacity for gross violence per “kill decision,” and thus require more “kill decisions” to have a large capacity for gross violence.⁷ The dictator herself can only perform so many “kill decisions.” In order to increase her capacity for directed violence, she needs a large group of people to help her. In the third scenario, she possesses an immense capacity for directed violence all by herself. This is because the network LAWS can execute a large amount of “kill decisions” and precisely select the targets of those decisions, all without further human interaction. I have provided a table below summarizing the differences between the described weapon system types. LAWS provide large amounts of target selectivity and gross violence, without the need for a large number of people, and directed violence is the product of gross violence and target selectivity. Thus, the development and use of LAWS allow a small group of people to have a disproportionate⁸ capacity for directed violence.

	Low Capacity for Gross Violence (with Few People)	High Capacity for Gross Violence (with Few People)
Low Target Selectivity	Blindfolded dictator with a revolver	Nuclear weapons
High Target Selectivity	Conventional, non-nuclear weapon systems	LAWS

If LAWS allow small groups of people to hold a disproportionate capacity for directed violence, proving that directed violence is sufficient for political power will imply that LAWS allow small groups of people to maintain disproportionate political power. The following section will do as much.

Political power is the ability to determine the actions of other individuals within an organization. This organization can be as small as a group of coworkers (hence the term office politics), or as large as an international governing body like the UN. Political power can be achieved through both incentivization and coercion, the carrot or the stick. Directed violence, as a consequence of its target selectivity, can be used in pursuit of a goal. If the crazed dictator were to say to the population of her nation “Stand on your heads, and I will execute anyone who does not!” the population would likely do so, but had she no ability to do so, she would likely only be laughed at. However, If she did possess the means to execute anyone who did not stand on their head, which would be directed violence, the people would do so.⁹ Thus, we can see that directed violence is sufficient for political power.

Logically, it follows from the statements “LAWS allow small groups of people to hold a disproportionate capacity for directed violence” and “directed violence is sufficient for political power” that LAWS allows small groups of people to hold disproportionate political power. This conclusion can also be motivated by the previous three scenarios. When the dictator has a nuclear weapon device, every use of it will come at a great collateral cost to the nation she is attempting to control. As such, her ability to leverage her power toward specific ends will be limited. If Citizen K is not doing her daily headstands, the cost of annihilating her and everything else in a mile radius will be prohibitively high, and thus the dictator has no leverage over K.¹⁰ In the scenario with the vast arsenal of conventional weapons, the dictator can roll her tank up to K’s front lawn, point the turret at her door, and demand that K do her headstands, but she will be unable to enforce this rule on many more people than K and perhaps her neighbors unless the dictator gets the help of many more people. With access to a vast network of LAWS, however, the dictator can simply order them to go to every house in the country and force every citizen at gunpoint to do their headstands, even K. This demonstrates that the scenario in which the dictator has the largest capacity for directed violence is the scenario in which she has the most political power.

I have so far shown that LAWS allows small groups of people to hold disproportionate political power. This statement makes no explicit moral claim. To show that we have moral reason to regulate the use and development of LAWS, I will show that we have moral reason to regulate any mechanism that allows a small group of people to hold disproportionate political power.

Imagine two potential regimes, one in which every citizen has approximately the same amount of political power,¹¹ and one in which the majority of the political power is in the hands of a small group of people. For the least advantaged person, the first more egalitarian regime is the most beneficial choice. For this reason, it would be in the best interest of contractors behind a veil of ignorance.¹² Therefore, a Rawlsian analysis would find a more egalitarian regime morally superior. If a state is morally superior to its alternatives, we have moral reason to bring about that state. Likewise, we have moral reason to prevent actions that can bring about morally inferior alternatives.¹³ Regulating the development and use of mechanisms that can bring about a morally inferior state constitutes such prevention. Thus, we have moral reason to prevent the development and use of mechanisms that allow a small group of people to hold disproportionate political power, in this case, LAWS.

The remainder of this paper is concerned with anticipating and responding to potential objections to my argument. Those objections are as follows:

Ambiguous Definition Objection: The definition of LAWS provided in this paper is largely stipulative, but a general definition of LAWS is highly debated. This ambiguity concerning what can and cannot be considered a LAWS can be presented as an objection in two ways.

The first objection states that, while we may have moral reason to regulate the use and development of LAWS, the ambiguous definition of LAWS makes prohibits us from doing so. This objection shows not that we should not attempt to regulate LAWS, but, merely that doing so may prove difficult. Additionally, we regulate many things that have ambiguous definitions,¹⁴ such as pornography and sexual harassment ¹⁵ by simply picking a good enough definition.

The second form of this objection suggests our inability to find a clear definition of a LAWS reflects that there is no morally relevant difference between LAWS and conventional weapon systems. Firstly, as demonstrated above, LAWS give small groups of people a disproportionate capacity for directed violence, greatly beyond what conventional weapons can give. This is a morally relevant difference. However, even if there is no morally relevant difference between LAWS and conventional weapons, there is still reason to regulate them. If what I have called LAWS are simply a new kind of conventional weapon, then they should be regulated similarly to the most analogous existing class of conventional weapons. Since the defining characteristic of what I call LAWS is their ability to make a “kill decision” on their own, they are most similar to anti-personnel landmines. Anti-personnel landmines are currently regulated by the United Nations (United Nations 1997), so what I call LAWS should be as well. If LAWS are different only in form, and not in kind, from landmines, then they should be banned like landmines. If they are different in kind, then my argument applies.

Welfare Objection: When describing how violence is sufficient for political power, I mention that political power can be garnered with incentivization as well as coercion. An objection can be raised stating that since such nonviolent welfare acts are sufficient for political power and come at less collective cost than violence, such actions will be selected over the violent use of LAWS. This objection fails for a couple of reasons. Firstly, if this logic held true, no state would have ever engaged in violence to maintain power, which is demonstrably untrue. Additionally, the precise nature of LAWS presents this objection with further difficulty. Conventional weapons need a large number of people in order to hold a large enough capacity for directed violence to maintain political power. As such, many states engage in targeted welfare to motivate people to perform “state-sponsored” directed violence.¹⁶ If this large group of people is made obsolete by LAWS, these welfare programs would become pointless and disappear. Also, welfare is not a fool proof way to ensure loyalty. Machines do as they are designed, and nothing else. For this reason, using LAWS is a more reliable option for a state than using humans incentivized welfare. Finally, even if welfare methods can compete against violent control via LAWS, that does not mean we should not regulate LAWS. Not all human decision-making is rational, so actors within the state may still attempt to use LAWS even if it is not the best method.

Platonic Objection: All things being equal, it is morally superior to have a regime in which political power is distributed equally. However, it can be objected that all things would never be equal. Plato theorized that the justest regime would be one in which a small class of intellectually pure “guardians” would rule justly out of a love for knowledge (“philosophy”)(Plato, ca 375 BCE). While this theory is very optimistic, sympathetic thinkers could argue that a smaller ruling group could have less infighting and run the state in a more efficient manner, yielding better well-being for the entirety of the state. This objection fails because it presupposes that political power is not an intrinsic good,¹⁷ but this is dubious at best. People often want the agency, even if it is not in their best interest, and political power is a form of agency that allows one to decide what rules they live under, to a degree.¹⁸ Plato suggests a “noble lie” is the solution to all this. He believed that the population could be deceived into not pursuing political agency. However, this seems questionable, both practically and morally.

Anarchist Objection: This objection takes the opposite stance of the Platonic Objection. It argues that if we accept that we have moral reason to prevent the development and use of mechanisms that allow a small group of people to hold disproportionate political power, then we have moral reason to prevent the existence of any state since the state is a small group (relative to the whole population) that holds the majority of political power. While this view is not incompatible with saying we should regulate LAWS,¹⁹ some may present the argument as a reductio ad absurdum²⁰ for my argument. Firstly, it is not immediately apparent that this conclusion is absurd. Indeed, the anarchists may be correct, and the existence of a state may be morally wrong. However, this objection also fails to consider that, just because we have moral reason to do something doesn’t mean that we do not have greater moral reason not to do it. There may be a moral reason not to have a state, in fact, there probably is, but there is much greater reason to have one.²¹ Such an overriding reason does not exist to develop and use LAWS.

Crazier Dictator Objection: This objection states that nuclear weapons can rival LAWS in their ability to maintain political power for a small group. If our crazed dictator were to be replaced by an even crazier dictator, that dictator would discover that K had not been doing her daily headstands and then glass K’s entire town. Going forward, no one would dare skip their headstands, for fear of the horrific consequences. While it is plausible that this could happen, it does not follow that we should regulate LAWS. If nuclear weapons are analogous to LAWS, then we should regulate LAWS because we regulate nuclear weapons.

Conclusion

The international community has moral reason to regulate the use and development of LAWS. This moral reason is derivative from a moral interest in preventing the consolidation of political power with small groups of people. LAWS enable smaller groups of people to gain large amounts of political power, to a degree that conventional weaponry cannot. Insofar as we have moral reason to pursue a more egalitarian system of governance, we have moral reason to regulate LAWS.²²

Notes

^[1] The robots authorized do not fit this paper’s definition of being lethal and autonomous. However, the council’s actions represent a general trend toward using lethal robots in more contexts.

^[2] This argument can apply to groups as small as one person.

^[3] Some argue that a LAWS must also generate its own targeting and/or “kill decision” criteria. My argument will apply to any weapon systems that fit my definition of LAWS. If that definition is correct, then my argument simply also applies to certain types of conventional weapon systems.

^[4] In this paper, “conventional” refers to weapon systems that do not fit my definition of LAWS. Weapons that are conventional in the sense that they are not nuclear weapons will be referred to as “non-nuclear.”

^[5] The Terminator-esque scenario where a malfunctioning or maliciously programmed LAWS goes on an unrestrained killing spree is beyond the scope of this paper, but may also provide moral reason to regulate the use and development of LAWS.

^[6] This is to say directives could be as broad as “suppress rebel activity in this region” and as specific as “find and execute this individual.”

^[7] Some non-nuclear weapons of mass destruction do possess a large amount of gross violence per “kill decision.” However, these lack target selectivity in the same way that nuclear weapons do, and can be treated like nuclear weapons for this argument.

^[8] The capacity for directed violence is disproportionate to that which can be achieved with conventional weapon systems.

^[9] Some may not, but they would be in the very principled minority, and would also be dead shortly after. This level of dissent does not imply that the dictator lacks political power.

^[10] A possible response to this is that while the dictator may not have an actual capacity for directed violence, they are still able to exert control through perceived power. This may be true, but I still maintain that it is better for the dictator to only have perceived power, as opposed to actual power to back it up.

^[11] This is of course an idealized democracy that is unlikely to exist. However, it is still useful to show that a more equal distribution of political distribution is ethically preferable, all things being equal.

^[12] This is an application of Rawlsian contractarian thinking, which applies well in political contexts because it mirrors the social contract between the governed and the state. However, this claim can be motivated by both utilitarianism and Harsanyian contractarianism if political power is a good that is subject to the law of diminishing marginal return.

^[13] “If a state is morally superior to its alternatives, we have moral reason to bring about that state” is the contrapositive of “If a state is morally inferior, we have moral reason not to bring about that state.”

^[14] Justice Potter Stewart is famous for having declared about obscenity “I know it when I see it.”

^[15] This is not to imply that sexual harassment is in some way relative, only that laying out any definition of it will likely not perfectly describe every potential instance of it.

^[16] Examples include government pensions for cops and free college for military personnel.

^[17] Intrinsic goods are good in and of themselves, independent of their consequences.

^[18] Examples of this include pushback to the regulation of cigarettes and the prohibition of alcohol.

^[19] It may in fact be incompatible, as the question of who is doing the regulating arises. However, anarchists have replies to this, and such discussion is beyond the scope of this paper.

^[20] Reduction to the absurd, an argumentative strategy of showing how an argument’s line of reasoning leads to absurd conclusions, and thus must be unsound.

^[21] Reasons include the prevention of rape, murder, and theft, as well as collective infrastructure.

^[22] Special thanks to Alyssa Haines, Justin Lee, and Raymond Zhen for their help in the writing of this paper.

References

Plato. (ca 375 BCE). The Republic. (B. Jowett, Trans.) Project Gutenberg. Retrieved from https://www.gutenberg.org/cache/epub/1497/pg1497-images.html.

United Nations. (1997). Convention on the Prohibition of the Use, Stockpiling, Production and Transfer of Anti-Personnel Mines and on their Destruction, 18 September 1997. Retrieved from https://www.un.org/en/genocideprevention/documents/atrocity-crimes/Doc.44_convention%20antipersonnel%20mines.pdf