In the two decades since it was decided, Zeran v America Online has been extensively analyzed, criticized, assessed and re-assessed by commentators, yet one of the Fourth Circuit’s central claims in the decision—that the “spectre” of tort liability on the internet would have an “obvious chilling effect”—has notably escaped more systematic study and evaluation, at least empirically. Despite the importance of these “chilling effect” claims to the court’s decision, this lack of empirical study is not altogether surprising. There has been strikingly little such systematic study of such chilling effect claims in various areas of law over the years. Part of the problem is that chilling effects are often subtle, difficult to measure, and require interdisciplinary research and methods going beyond traditional legal analysis. Thus, Leslie Kendrick found in 2013, after reviewing existing literature, that empirical support for such chilling effect claims was “flimsy” and thus requiring more far study.

Today, this systematic empirical study has finally begun to take shape. Several recent studies have documented “chilling effects” in different contexts, including my own work on surveillance related chilling effects, which received extensive media coverage last year, as well as a more recent study, examining the comparative dimensions of regulatory chilling effects online, which I wrote about recently in Slate. With these, and other recent empirical work, it is now possible to critically assess the chilling effect claims in Zeran with more insight and understanding than any time previously.

Drawing on this research, including new findings from my own recently published chilling effects research paper, I argue that the Fourth Circuit was right to raise chilling effect concerns in this context but likely wrong about how they would arise.

Zeran was the first case wherein §230 of the Communications Decency Act was raised as a defense, and has also turned out to be the most important and influential (e.g., it has been cited at least 1,400 times). The facts essentially involved a case of online harassment whereby an unidentified person posted on America Online (AOL)’s message board false and defamatory messages about the plaintiff Kenneth Zeran, who sued AOL for failing to remove the postings promptly on notice.

In dismissing Zeran’s lawsuit, the Fourth Circuit made two chilling effect claims. First, that the possibility or “spectre” of tort liability more generally, would have an “obvious chilling effect” as it would lead online service providers (OSPs) to restrict speech on their services as policing “millions” of postings for problems would be “impossible.” Second, liability on notice would similarly have a chilling effect on internet speech due to over-enforcement — because OSPs would be liable only for publishing and not removal, they would have an incentive to remove content or messages on notice, whether defamatory or not. The court did not cite social science or empirical research to support either assertion. And while there were some previous studies concerning libel chill when Zeran was decided (see Ciolli’s work for a discussion) there were none dealing with chilling effects in online contexts.

Though these two claims are framed slightly differently—one speaks to tort liability more generally while the other concerns liability on notice—the central point of both is that OSPs, when faced with liability concerns arising from the activities of users of their services, will take steps to limit their exposure to liability by restricting those activities. Here, this would mean restricting and limiting internet speech, thus “chilling” it. Put succinctly, the OSP, through its liability concerns, is the main source for any chilling effects on internet speech.

As with many chilling effect claims, this assertion is difficult to assess because it would involve testing counterfactuals—how does one test the proposition that but for the broad §230 immunity for online service providers found in Zeran there would have been a chilling effect on speech due to OSP restrictions? Or that but for removing liability even on notification, OSPs would have taken steps to limit and thus chill speech?

Fortunately, recent research on the Digital Millennium Copyright Act (DMCA) arguably allow us to do just that. As with the defamatory content in Zeran, OSPs in the 1990s faced liability for copyrighted materials users posted on their services without authorization. But rather than §230’s blanket immunity approach to deal with this challenge, Congress instead enacted the DMCA, which employs a notice-and-takedown system to enforce and police copyright online. Someone who believes their copyrights are being infringed can send a DMCA “takedown” notice to an OSP to have the content removed. Put simply, like the liability-on-notice schemes rejected in Zeran because they would likely lead to a “chill” on internet speech, the DMCA provides OSPs with immunity so long as they remove infringing content promptly upon notice. In other words, the DMCA has, in ways, created the counterfactual regulatory state of affairs to test the chilling effect claims in Zeran.

So, is there any evidence or empirical support for the Fourth Circuit’s chilling effects concerns? On this count, the Fourth Circuit in Zeran was right to raise chilling effect concerns, but was wrong to predict that OSPs would pose the real threat to speech.

The Zeran court, as noted, was primarily concerned about OSPs restricting speech through a “liability on notice” regime and the “spectre” of liability it constitutes. There is certainly some evidence on this count, but the case is largely circumstantial. For example, there has long been anecdotal evidence of DMCA “abuse” whereby invalid, false or improper DMCA notices lead to content removal online, especially as automation is increasingly used for enforcement. Moreover, the “compliance” rate for DMCA notices, that is, the reported rate at which an OSP report removing content in response to notification is fairly high at various well known and popular OSPs. Google, for example, removes websites or content either fuller or partially in response to DMCA takedown notices in 98 percent of cases. Twitter, from January to June 2017, complied with 75 percent of DMCA notices. WordPress reports removal in 61percent of cases. Those rates are not necessarily a problem by themselves, though when combined with studies that have documented substantial percentages of invalid or problematic DMCA notifications — like this 2016 study by Jennifer Urban, Joe Karaganis and Brianna Schofield finding that 30 percent of DMCA notices had potential problems — then these rates and anecdotal instances may suggest OSPs are opting for removal, and thus speech restrictions, to avoid liability. Still, there is no “smoking gun” here, and more research would need to be done on OSP practices to substantiate chilling effect concerns like those of Fourth Circuit in Zeran.

But this is not the end of the story. In fact, there is reason to suspect “liability on notification” schemes can have a noteworthy chilling effect on online activities, but the culprit is not the OSPs receiving the notifications, but the notifications themselves.

This is among the key findings I discuss in my new chilling effects research paper, published earlier this year, based on an empirical case study from my doctorate at the University of Oxford. The study involves an original first-of-its-kind survey, administered to over 1,200 U.S. based adult internet users, designed to explore different dimensions of chilling effects, threats and concerns online by comparing and analyzing participant responses to hypothetical scenarios that, in theory, may cause chilling effects or self-censorship. The study’s findings suggested, among other things, that once internet users received a personal legal notice for content they had posted online, noteworthy percentages of internet users were less likely to speak or write about certain things online, less likely to share personally created content, less likely to engage with social media, and more cautious in their internet speech or search. In other words, there was a clear chilling effect. And among all the scenarios studied, responses suggested receiving a personal legal notice like this would have the greatest comparative chilling effect on people’s online activities. This is important as under the DMCA, and similar liability-on-notice regimes, the user posting the alleged illegal content, in addition to the OSP, receives a copy of the legal notice. These findings offer insights into the impact these legal notices, and the legal threat therein, have on individual internet users.

For example, in terms of online speech, 75 percent of respondents in the study indicated they would be “much less likely” (40 percent) or “somewhat less likely” (35 percent) to “speak or write about certain topics online” after receiving a personal legal notice about something they had previously posted online. Similarly, 81 percent of respondents indicating they “strongly agreed” (50 percent) or “somewhat agreed” (31 percent) with a statement that they would be more cautious or careful about their online speech after receiving such a personal legal notice. There were similar findings suggesting a chilling effect on other activities beyond speech, including online search, content sharing, and social network engagement. I also found evidence of a form of indirect chilling effects where internet users suggested they would be less likely to speak or share when a friend in their online social network had received a personal legal notice for content they had posted online.

When you combine these empirical insights as to the impact of these “liability notifications” like DMCA (or libel) notices with the reality that literally tens of millions of these notices are now being sent weekly due to automation, a starker picture emerges of a substantial and noteworthy chilling effect on internet speech, and a range of other online activities, likely stemming from this broader regulatory ecosystem. Moreover, the notion that fear of legal or similar threats may “chill” online activities is consistent with a range of recent and comparable chilling effect studies in different contexts.

The chilling effect concerns raised in Zeran were essential to the Fourth Circuit’s reasoning as they were employed to justify rejecting alternatives to blanket immunity — like liability on notice. Years on, in light of new empirical studies on chilling effects, including my own, we are better situated to assess those claims. Today, the evidence suggests that the court’s concerns about chilling effects associated with “liability on notification” alternatives were sound. The court was just wrong on how they would arise.

Jonathon Penney is a research fellow at the Citizen Lab at the University of Toronto and teaches law at Dalhousie University. He will be a research affiliate at Princeton’s Center for Information Technology Policy starting in fall 2017.

This essay is part of a larger collection about the impact of Zeran v. AOL curated by Eric Goldman and Jeff Kosseff.