Do Big Data and advances in statistical inference threaten to replace trial lawyers? No. That’s because rhetoric, the essence of the trial lawyer’s craft, always will define the starting point of any inquiry into cause and effect.
The lawyer of the future would be “the man of statistics,” predicted Oliver Wendell Holmes Jr. in his classic law review essay, “The Path of the Law.” More than a century later, after many missteps, false starts, appalling injustices and fierce controversies, the future undeniably has arrived.
Data-mining is going on everywhere, and the tools to do it grow cheaper and more powerful each year. Among other things being unearthed are: new claims, e.g., finding links between a drug and an illness; new avenues of defense, e.g., risk quantification; new ways to show malfeasance, e.g., turning a failed clinical trial into a successful one by exploding statistics; and new ways to uncover the prevarications of expert witnesses, e.g. hacking data until a statistically significant association emerges.
In the midst of the Big-Data, statistical-analysis and machine-learning revolution, lawyers must wonder whether they’ll have a role in the future. Will it lead to the sort of justice that consists of dumping all the evidence into a computer, hitting “enter” and reading the verdict off the screen?
While lawyers have been fretting about their jobs, some leading experts in subjects like statistics, machine learning and causal inference have dug deeply into legal topics like evidence, proof and preponderance of evidence. To their surprise, they have found that forensic rhetoric is indispensable and fundamental to solving the sort of problems posed in lawsuits. Equally surprising, they are finding that the charts developed by John Henry Wigmore (of evidence hornbook fame) closely resemble modern Bayesian causal networks (of machine-learning fame). Indeed, before evidence became little more than rules and their exceptions, it was presciently engaged in trying to find algorithms to do the same things as today’s experts in artificial intelligence.
Updating Holmes, “the trial lawyer of today needs to be a person of statistics,” wrote statistics professor Philip Dawid in an appendix, titled “Probabilities and Proof,” to the book “Analysis of Evidence.” Dawid was, of course, the expert who, on appeal, helped secure a reversal of the notorious conviction of Sally Clark.
That today’s trial lawyer needs to be a person of statistics is hardly how the future looked four decades ago when law professor Lawrence Tribe wrote “Trial by Mathematics,” expressing a deep skepticism towards trial by statistics. Early uses of statistics in trials had led to so many absurd results that a widespread belief emerged that statistical evidence was too easily abused and too difficult for jurists and jurors to grasp ever to serve in the cause of justice. If the cottage industry that has sprung up to publish books about outrageous judgments based on laughable statistical ignorance is any indication, the skeptics were right.
Yet, here we are in a judicial system awash in statistical data, and lawyers have nothing to do but learn to deal with it. Besides, difficult questions about causation in litigation are nothing new. Making sound inferences about causation when faced with concurrent or superseding causes isn’t just a potential question on a first year torts final. It’s very much a here-and-now question, as evidenced by all the attention focused on a case currently before the Texas Supreme Court, Bostic v. Georgia Pacific.
So, if most courts have little hesitation about wading into the thicket of profoundly intellectually challenging debate about causation, why has it been so hard to get right statistical evidence and the conclusions drawn from it?
One reason may be that Dawid suggests, which he gleaned from his dealings with lawyers. He believes some attorneys fear numbers, while others actually take pride in their innumeracy. I think the more likely cause is that trial lawyers and judges, thanks to their training in the art of persuasion and experience with juries, have heightened awareness about how real people make decisions in the face of uncertainty. In other words, they appreciate how readily statistical data can trigger cognitive biases, the elucidation of which won Daniel Kahneman a Nobel Prize in economics several years ago.
But whatever the reason for avoiding statistics in the past, nowadays lawyers can’t escape them. In this age of Big Data, an attorney can’t test an opponent’s evidence if she doesn’t know that most published, peer-reviewed and statistically significant biomedical research findings are probably false—and why that’s the case. A lawyer who can’t articulate why a woman probably doesn’t have breast cancer even though she’s had a positive mammogram that’s 95 percent accurate and only produces 5 percent false positives ignores severe cognitive blind spots. And counsel who can’t explain “p-hacking” (i.e. how p-values and confidence intervals can easily be hacked) miss out on a potentially devastating avenue for cross-examination.
By definition, statistical data merely is a sample of a much larger set of data. It is therefore not the sum of all knowledge but rather just some of the knowledge. So, just like any other piece of evidence has to be assessed as relevant and reliable. Far from threatening to replace trial lawyers with an algorithm, the advances in Big Data in statistical inference cry out for skilled lawyers to interrogate statistical information for relevance and credibility the same way they would any other piece of evidence.
The good news is that it’s not really all that hard. Watch the first 10 Khan Academy videos on statistics or take one of many introductory massive open online courses (MOOCs) about probability and statistics. Besides, if the presence of a young man I saw on a jury panel recently who was wearing a “Bayes Rule!” T-shirt is any indication, your audience may soon come to expect it.
David A. Oliver of Houston, a partner in Vorys Sater Seymour and Pease, is board-certified in personal-injury trial law by the Texas Board of Legal Specialization. His litigates allegations of injuries due to exposure to chemicals or pharmaceuticals. He is the editor of the blog Mass Torts: State of the Art.