14 - The Reverse Spider-Man Principle

Summary

An entity – a landlord, a manufacturer, a phone company, a credit card company, an internet platform, a self-driving-car manufacturer – is making money off its customers’ activities. Some of those customers are using the entity’s services in ways that are criminal, tortious, or otherwise reprehensible. Should the entity be held responsible, legally or morally, for its role (however unintentional) in facilitating its customers’ activities? This question has famously been at the center of the debates about platform content moderation,¹ but it can come up in other contexts as well.²

14.1 Introduction

An entity – a landlord, a manufacturer, a phone company, a credit card company, an internet platform, a self-driving-car manufacturer – is making money off its customers’ activities. Some of those customers are using the entity’s services in ways that are criminal, tortious, or otherwise reprehensible. Should the entity be held responsible, legally or morally, for its role (however unintentional) in facilitating its customers’ activities? This question has famously been at the center of the debates about platform content moderation,¹ but it can come up in other contexts as well.²

It is a broad question, and there might be no general answer. (Perhaps it is two broad questions – one about legal responsibility and one about moral responsibility – but I think the two are connected enough to be worth discussing together.) In this chapter, though, I’d like to focus on one downside of answering it “yes”: what I call the Reverse Spider-Man Principle – with great responsibility comes great power.³ Whenever we are contemplating holding entities responsible for their customers’ behavior, we should think about whether we want to empower such entities to surveil, investigate, and police their customers, both as to that particular behavior and as to other behavior.⁴ And that is especially so when the behavior consists of speech, and the exercise of power can thus affect public debate.

Of course, some of the entities with whom we have relationships do have power over us. Employers are a classic example: In part precisely because they are responsible for our actions (through principles such as respondeat superior or negligent hiring/supervision liability), they have great power to control what we do, both on the job and in some measure off the job.⁵ Doctors have the power to decide what prescription drugs we can buy, and psychiatrists have the responsibility (and the power) to report when their patients make credible threats against third parties.⁶ And of course we are all subject to the power of police officers, who have the professional though not the legal responsibility to prevent and investigate crime.

On the other hand, we generally do not expect to be in such subordinate relationships to phone companies, or to manufacturers selling us products. We generally do not expect them to monitor how we use their products or services (except in rare situations where our use of a service interferes with the operation of the service itself), or to monitor our politics to see if we are the sorts of people who might use the products or services badly. At most, we expect some establishments to perform some narrow checks at the time of a sale, often defined specifically and clearly by statute, for instance by laws that require bars not to serve people who are drunk or that require gun dealers to perform background checks on buyers.⁷

Many of us value the fact that, in service-oriented economies, companies try hard to do what it takes to keep customers (consider the mentality that “the customer is always right”), rather than expecting customers to comply with the companies’ demands. But if we insist on more “responsibility” from such providers, we will effectively push them to exercise more power over us, and thus fundamentally change the nature of their relationships with us. If companies are required to police the use or users of their products and services (what some call “third-party policing”⁸) then people’s relationship with them may become more and more like people’s relationship with the police.

To be sure, none of this is a dispositive argument against demanding such responsibility. Perhaps sometimes such responsibility is called for. My point, though, is that this responsibility also carries costs. We should take those costs into account when we engage in “balancing,” “proportionality tests,” Learned Hand cost–benefit analysis, or something similar – whether as a matter of adjudication, policymaking, or even just moral judgment – in deciding whether to demand such responsibility.

14.2 The Virtues of Irresponsibility

Let me begin by offering three examples of where some courts have balked at imposing legal liability, precisely because they did not want to require or encourage businesses to exercise power over their customers.

14.2.1 Telephone and Telegraph Companies

The first came in the early 1900s, when some government officials demanded that telephone and telegraph companies block access to their services by people suspected of running illegal gambling operations. Prosecutors could have gone after the bookies, of course, and they did. But they also argued that the companies should have done the same – and indeed sometimes prosecuted the companies for allowing their services to be used for such criminal purposes.

No, held some courts (though not all⁹); to quote one:

A railroad company has a right to refuse to carry a passenger who is disorderly, or whose conduct imperils the lives of his fellow passengers or the officers or the property of the company. It would have no right to refuse to carry a person who tendered or paid his fare simply because those in charge of the train believed that his purpose in going to a certain point was to commit an offense. A railroad company would have no right to refuse to carry persons because its officers were aware of the fact that they were going to visit the house of [the bookmaker], and thus make it possible for him and his associates to conduct a gambling house.

Common carriers are not the censors of public or private morals. They cannot regulate the public and private conduct of those who ask service at their hands.¹⁰

If the telegraph or telephone company (or the railroad) were held responsible for the actions of its customers, the court reasoned, then it would acquire power – as “censor[] of public or private morals” – that it ought not possess.

And indeed, Cloudflare, a provider of internet services that prevents denial-of-service attacks, drew an analogy to a phone company in saying that it would generally not reject customers based on their views (though it might stop service to them if their services were actively being used to organize criminal attacks¹¹):

Our conclusion … is that voluntarily terminating access to services that protect against cyberattack is not the correct approach…. Just as the telephone company does not terminate your line if you say awful, racist, bigoted things, we have concluded in consultation with politicians, policy makers, and experts that turning off security services because we think what you publish is despicable is the wrong policy. To be clear, just because we did it in a limited set of cases before does not mean we were right when we did. Or that we will ever do it again.¹²

14.2.2 EMail Systems

Telegraph and telephone companies were common carriers, denied such power (and therefore, those courts said, responsibility) by law. But consider a second example, Lunney v. Prodigy Services Co., a 1999 case in which the New York high court held that email systems were immune from liability for allegedly defamatory material sent by their users.¹³

Email systems aren’t common carriers, but the court nonetheless reasoned that they should not be held responsible for failing to block messages, even if they had the legal authority to block them: An email system’s “role in transmitting e-mail is akin to that of a telephone company,” the court held, “which one neither wants nor expects to superintend the content of its subscribers’ conversations.”¹⁴ Even though email systems aren’t forbidden from being the censors of their users’ communications, the court concluded that the law should not pressure them into becoming such censors.

14.2.3 Landlords

Courts have likewise balked at imposing obligations on residential landlords that would encourage the landlords to surveil and police their tenants. Consider Castaneda v. Olsher, where a mobile-home-park tenant injured in a gang-related shootout involving another tenant sued the landlord, claiming it “had breached a duty not to rent to known gang members.”¹⁵ No, said the California Supreme Court:

[W]e are not persuaded that imposing a duty on landlords to withhold rental units from those they believe to be gang members is a fair or workable solution to [the] problem [of gang violence], or one consistent with our state’s public policy as a whole….

If landlords regularly face liability for injuries gang members cause on the premises, they will tend to deny rental to anyone who might be a gang member or, even more broadly, to any family one of whose members might be in a gang.¹⁶

This would in turn tend to lead to “arbitrary discrimination on the basis of race, ethnicity, family composition, dress and appearance, or reputation,”¹⁷ which may itself be illegal (so the duty would put the landlord in a damned-if-you-do-damned-if-you-do-not position).

But even apart from such likely reactions by landlords possibly being illegal, making landlords liable would jeopardize people’s housing options and undermine their freedom even if they aren’t gang members, further subjecting them to the power of their landlords: “[F]amilies whose ethnicity, teenage children, or mode of dress or personal appearance could, to some, suggest a gang association would face an additional obstacle to finding housing.”¹⁸ Likewise, even if landlords respond only by legally and evenhandedly checking all tenants’ criminal histories, “refusing to rent to anyone with arrests or convictions for any crime that could have involved a gang” would “unfairly deprive many Californians of housing.”¹⁹ This “likely social cost” helped turn the court against recognizing such a responsibility on the part of landlords.²⁰

Other courts have taken similar views. In Francis v. Kings Park Manor, Inc., for instance, the Second Circuit sitting en banc refused to hold a landlord liable for its tenants’ racial harassment of fellow tenants, partly because of concern that such responsibility would pressure landlords to exercise undue power over tenants:

[U]nder the alternative proposed by Francis, … prospective and current renters would confront more restrictive leases rife with in terrorem clauses, intensified tenant screening procedures, and intrusions into their dealings with neighbors, all of which could result in greater hostility and danger, even culminating in (or beginning with) unwarranted evictions.

Our holding should also be of special interest to those concerned with the evolution of surveillance by state actors or by those purporting to act at their direction. See Note 44, ante (warning against broad liability schemes that would encourage landlords to act as law enforcement).²¹

The New York intermediate appellate court took a similar view in Gill v. New York City Housing Authority, rejecting liability for tenant-on-tenant crime that the plaintiff claimed might have been avoided had the landlord dealt better with a tenant’s mental illness:

The practical consequences of an affirmance in this case would be devastating. The Housing Authority would be forced to conduct legally offensive and completely unwarranted “follow-up” of all those tenants within its projects known to have a psychiatric condition possibly … injurious to another tenant…. [E]viction, which is described in the Housing Authority Management Manual as a “last resort,” would become almost commonplace.²²

A New Jersey intermediate appellate court took the same view in Estate of Campagna v. Pleasant Point Properties, LLC, rejecting a claim that landlords should be responsible for doing background checks on tenants.²³ Likewise, in the related context of university liability for students’ consumption of alcohol, the Massachusetts high court concluded:

As many courts have noted, requiring colleges and universities to police all on-campus use of alcohol would be inappropriate and unrealistic. Although “[t]here was a time when college administrators and faculties assumed a role in loco parentis” and “[s]tudents were committed to their charge because the students were considered minors,” “[c]ollege administrators no longer control the broad arena of general morals.” College-aged students, while sometimes underage for the purposes of the purchase and consumption of alcohol, otherwise are adults expected to manage their own social activities…. [T]he additional intrusion into the private lives of students that would be necessary to control alcohol use on campus would be both impractical for universities and intolerable to students.²⁴

To be sure, the pattern here is not uniform. Sometimes landlords are held responsible (by statutes, ordinances, or tort law rules), for monitoring their tenants for potentially illegal behavior, such as the distribution of drugs; for failing to evict tenants who are violating the law,²⁵ or even tenants who are being victimized by criminals, and are thus calling 911 too often;²⁶ for failing to warn co-tenants of tenants’ past criminal records;²⁷ or even for renting to tenants who have criminal records.²⁸ But the result of those decisions has indeed been what the courts quoted above warned about: greater surveillance of tenants by landlords, and greater landlord power being exercised over tenants.²⁹

14.2.4 The Limits of Complicity

One way of understanding these cases is that they put limits on concepts of complicity. The law does sometimes hold people liable for enabling or otherwise facilitating others’ wrongful conduct, even in the absence of a specific wrongful purpose to aid such conduct;³⁰ consider tort law principles such as negligent hiring and negligent entrustment. But there are often good public-policy reasons to limit this.

Sometimes those reasons stem from our sense of professional roles. We do not fault a doctor for curing a career criminal, even if as a result the criminal goes on to commit more crimes. It’s not a doctor’s job to decide whether someone merits healing, or to bear responsibility for the consequences of successfully healing bad people.

Likewise, the legal system expects defense lawyers to do their best to get clients acquitted, and does not hold the lawyers responsible for the clients’ future crimes. (Indeed, historically the legal system allowed courts to order unwilling lawyers to represent indigent defendants.³¹) When there is public pressure on lawyers to refuse to represent certain clients, the legal establishment often speaks out against such pressure.³²

And sometimes those reasons stem from our sense of who should and who should not be “censors of public or private morals.” The police may enforce gambling laws, or arrest gang members for gang-related crimes. The courts may enforce libel law. But various private entities, such as phone companies, email services, and landlords, should not be pressured into doing so.³³

14.3 Practical Limits on Private Companies’ Power, in the Absence of Responsibility

Of course, many such companies (setting aside the common carriers or similarly regulated monopolies) already have great power over whom to deal with and what to allow on their property, even when they aren’t held responsible – by law or by public attitudes – for what happens on their property. In theory, for instance, Prodigy’s owners could have decided that they wanted to kick off users who were using Prodigy email for purposes that they found objectionable: libel, racist speech, Communist advocacy, or whatever else. Likewise, some companies may decide not to deal with people who they view as belonging to hate groups or anti-American organizations, just because their shareholders or managers think that’s the right thing to do, entirely apart from any social or legal norms of responsibility.

But in practice, in the absence of responsibility (whether imposed by law or social norms), many companies will eschew such power, for several related reasons – even setting aside the presumably minor loss of business from the particular customers who are ejected:

1. 1. Policing customers takes time, effort, and money.

2. 2. Policing customers risks error and bad publicity associated with such error, which could alienate many more customers than the few who are actually denied service.

3. 3. Policing customers risks allegations of discriminatory policing, which may itself be illegal and at least is especially likely to yield bad publicity.

4. 4. Policing some customers will often lead to public demands for broader policing: “You kicked group X, which we sort of like, off your platform; why aren’t you also kicking off group Y, which we loathe and which we view as similar to X?”³⁴

5. 5. Conversely, a policy of “we do not police our customers” – buttressed by social norms that do not require (or even affirmatively condemn) such policing – offers the company a simple response to all such demands.

6. 6. Policing customers creates tension even with customers who aren’t violating the company’s rules – people often do not like even the prospect that some business is judging what they say, how they dress, or whom they associate with.

7. 7. Policing customers gives an edge to competitors who publicly refuse to engage in such policing and who sell their services as “our only job is to serve you, not to judge you or eject you.”

Imposing legal responsibility on such companies can thus pressure them to exercise power even when they otherwise would not have. And that is so in some measure even if responsibility is accepted just as a broad moral norm, created and enforced by public pressure (likely stemming from influential sectors of society, such as the media or activists or professional organizations), and not a legal norm.³⁵ That moral norm would increase the countervailing costs of non-policing. It would decrease the costs of policing: For instance, the norm and the corresponding pressure would likely act on all major competitors, so the normal competitive pressures encouraging a “the customer is always right” attitude would be sharply reduced. And at some point, the norm might become the standard against which the reasonableness of behavior is measured as a legal matter.

Likewise, when people fault a company for errors or perceived discrimination, the company can use the norm as cover, for instance arguing that “regrettably, errors will happen, especially when one has to do policing at scale.” “After all, you have told us you want us to police, have not you?”

Accepting such norms of responsibility could also change the culture and organization of the companies. It would habituate the companies to exercising such power. It would create internal bureaucracies staffed with people whose jobs rely on exercising the power – and who might be looking for more reasons to exercise that power.

And by making policing part of the companies’ official mission, the acceptance of responsibility norms would subtly encourage employees to make sure that the policing is done effectively and comprehensively, and not just at the minimum that laws or existing social norms command. Modest initial policing missions, based on claims of responsibility for a narrow range of misuse, can thus creep into much more comprehensive use of such powers.³⁶

Indeed, it appears that something like this happened with social-media platforms. Title 47 U.S.C. § 230 freed online companies of legal responsibility for the content of users’ speech, and many such companies therefore did not exercise their legal power to restrict what users posted, or did so only lightly.³⁷ But the mid-2010s saw a combination of social and congressional pressure that held platforms responsible for supposed misinformation and other bad speech on their platforms, which caused the leading platforms to exercise such power more and more.³⁸ Platforms have now begun making decisions about which political candidates and officials to deplatform and which important political stories to block (including in the heat of an election campaign).³⁹ One might approve or disapprove of such power exercised by large business corporations over public discourse;⁴⁰ but my point here is simply that calls for great responsibility have indeed increased the exercise of such power.

14.4 The Internet of Things, Constant Customer/Seller Interaction for Tangible Products, and the Future of Responsibility

So far, there has been something of a constraint on calls for business “responsibility” for the actions of their customers: Such calls have generally involved ongoing business–customer relationships, for instance when Facebook can monitor what its users are posting (or at least respond to other users’ complaints).

Occasionally, some have called on businesses to simply not deal with certain people at the outset – consider Castaneda v. Olsher, where the plaintiffs argued that the defendants just should not have rented the mobile homes to likely gang members. But such exclusionary calls have been rare.

I expect, for instance, that few people would think of arguing that car dealers should refuse to sell cars to suspected gang members who might use the cars for drive-by shootings or for crime getaways.⁴¹ Presumably, most people would agree that even gang members are entitled to buy and use cars in the many lawful ways that cars can be used, and that car dealers should not see their job as judging the likely law-abidingness of their customers.⁴² If the legislature wants to impose such responsibilities, for instance by banning the sale of guns to felons or of spray paint to minors, then presumably the legislature should create such narrow and clearly defined rules, which would rely on objective criteria that do not require seller judgment about which customers merely seem likely to be dangerous.

But now more and more products involve constant interaction between the customer and the seller.⁴³ Say, for instance, that I’m driving a partly self-driving Tesla that is in constant contact with the company. Recall how Airbnb refused to rent to people who it suspected were going to a “Unite the Right” rally.⁴⁴ If that is seen as proper – and indeed is seen as mandated by corporate social responsibility principles – then one can imagine similar pressure on Tesla to stop Teslas from driving to the rally (or at least to stop such trips by Teslas owned by those people suspected of planning to participate in the rally).⁴⁵

To be sure, this might arouse some hostility, because it’s my car, not Tesla’s. But Airbnb was likewise refusing to arrange bookings for other people’s properties, not its own. Airbnb’s rationale was that it had a responsibility to stop its service from being used to promote a racist, violent event.⁴⁶ Why would not Tesla then have a similar responsibility to stop its intellectual property and its central computers (assuming they are in constant communication with my car) from being used the same way?

True, the connection between the Tesla and its user’s driving to the rally is somewhat indirect – but not more so than Airbnb’s. Indeed, Tesla’s connection is a bit more direct: Its product and the accompanying services would get the driver the last mile to the rally itself, rather than just providing a place to stay the night before. Indeed, there’s just one eminently foreseeable step (a short walk from the parking space) between the use of the Tesla and the driver’s attendance at the rally. And conversely, if we think Tesla should not be viewed as responsible for its cars being used to get to rallies that express certain views, what should that tell us about whether Facebook should be responsible for use of its service to convey those views?

Now, Tesla’s sales contract might be seen as implicitly assuring that its software will always try to get me to my destination. But that is just a matter of the contract. If companies are seen as responsible for the misuse of their services, why wouldn’t they have an obligation to draft contracts that let them fulfill that responsibility?

Of course, maybe some line might be drawn here: Perhaps, for instance, we might have a special rule for services that are ancillary to the sale of goods (Tesla, yes; Airbnb, no), under which the transfer of the goods carries with it the legal or moral obligation for the seller to keep providing the services even when one thinks the goods are likely to be used in illegal or immoral ways. (Though what if I lease my Tesla rather than buying it outright, or rent it for the day just as I might rent an Airbnb apartment for the night?) Or at least we might say there’s nothing irresponsible about a product seller refusing to police customers’ continuing use of the services that make those products work.

But that would just be a special case of the broader approach that I’m suggesting here: For at least some kinds of commercial relationships, a business should not be held responsible for what its customers do – because we do not want it exercising power over its customers’ actions. We might then ask whether we should apply the same principle to other commercial relationships.

14.5 Big Data and the Future of Responsibility

There has historically also been another constraint on such calls for business “responsibility”: It’s often very hard for a business to determine what a customer’s plans are. Even if there is social pressure to get businesses to boycott people who associate with supposed “hate groups”⁴⁷ – or even if the owners of a business (say, Airbnb) just want to engage in such a boycott – how is a business to know what groups a person associates with, at least unless the person is famous, or unless someone expressly complains about the person to the business?⁴⁸

But these days we can get a lot more data about people, just by searching the Internet and some other databases (some of which may cost money, but all of which are well within the means of most big businesses). To be sure, this might yield too much data about each prospective customer for a typical business to process at scale. But AI technology will likely reduce the cost of such processing by enabling computers to quickly and cheaply sift through all that data, and to produce some fairly reliable estimate: Joe Schmoe is 93 percent likely to be closely associated with one of the groups that a business is being pressured to boycott. At that point, the rhetoric of responsibility may suggest that what now can be done (identifying supposedly evil potential clients) should be done.

Consider one area in which technological change has sharply increased the scope of employer responsibility – and constrained the freedom of many prospective employees. American tort law has long held employers responsible for negligent hiring, negligent supervision, or negligent retention when they unreasonably hire employees who are incompetent at their jobs in a way that injures third parties,⁴⁹ or who have a tendency to commit crimes that are facilitated by the job.⁵⁰ But until at least the late 1960s, this had not required employers to do nationwide background checks, because such checks were seen as too expensive, and thus any such requirement “would place an unfair burden on the business community.”⁵¹ Even someone who had been convicted of a crime could thus often start over and get a job, at least in a different locale, without being dogged by his criminal record.

Now, though, as nationwide employee background checks have gotten cheaper, they have in effect become mandatory for many employers: “Lower costs and easier access provide [an] incentive to perform [background] checks, potentially leaving employers who choose not to conduct such checks in a difficult position when trying to prove they were not negligent in hiring.”⁵² As a result, people with criminal records now often find it especially hard to get jobs.

Perhaps that’s good, given the need to protect customers from criminal attack. Or perhaps it’s bad, given the social value of giving people a way to get back to productive, law-abiding life. Or perhaps it’s a mix of both. But my key point here is that, while the employer’s responsibility for screening his employees has formally remained the same – the test is reasonable care – technological change has required employers to exercise that responsibility in a way that limits the job opportunities of prospective employees much more than it did before.

Similarly, commercial property owners have long been held responsible for taking reasonable – which is to say, cost-effective – measures to protect their business visitors from criminal attack. Thus, as video surveillance cameras became cheap enough to be cost-effective, courts began to hold that defendants may be negligent for failing to install surveillance cameras,⁵³ even though such surveillance would not have been required when cameras were much more expensive.

We can expect to see something similar as technological change renders cost-effective other forms of investigation and surveillance – not just of employees or of outside intruders, but of customers. If it is a company’s responsibility to make sure that bad people do not use the company’s products or services for bad purposes, then as technology allows companies to investigate their clients’ affiliations and beliefs more cost-effectively, companies will feel pressure to engage in such investigation.

14.6 Conclusion

“Responsibility” is often viewed as an unalloyed good. Who, after all, wants to be known as “irresponsible”?⁵⁴ Sometimes we should indeed hold people and organizations legally or morally responsible for providing tools that others misuse. People and organizations are also of course entitled to choose to accept such responsibility, even if they are not pressured to do so.⁵⁵ And sometimes even if they do not feel responsible for doing something, they might still choose to do it, whether because they think it’s good for their users and thus good for business, or because they think it’s good for society. In particular, I’m not trying to take a position here on what sort of moderation social-media platforms should engage in.⁵⁶

My point here is simply that such responsibility has an important cost, and refusal to take responsibility has a corresponding benefit. Those who are held responsible for what we do will need to assert their power over us, surveilling, second-guessing, and blocking our decisions. A phone company or an email provider or a landlord that’s responsible for what we do with its property will need to control whether we are allowed to use its property, and control what we do with that property; likewise for a social-media platform or a driverless-car manufacturer. If we want freedom from such control, we should try to keep those companies from being held responsible for their users’ behavior.

There is value in businesses being encouraged to “stay in their lane,” with their lane being defined as providing a particular product or service. They should be free to say that they “are not the censors of public or private morals,” and that they should not “regulate the public and private conduct of those who ask service at their hands.”⁵⁷ Even if, unlike with telephone and telegraph cases, they have the legal right to reject some customers, they should be free to refrain from exercising that right. Sometimes the responsibility for stopping misuse of the product should be placed solely on the users and on law enforcement – not on businesses that are enlisted as largely legally unsupervised private police forces, doing what the police are unable to do or (as with speech restrictions) are constitutionally forbidden from doing.