Stanford Law School | King of Hearts via Wikimedia Commons
PALO ALTO — Ashkon Farhangi doesn’t have a name yet for the software project he’s been working on. But it might just be the next thing to shake up the legal research industry.
A Stanford computer science masters student who also works full-time at Google, Farhangi has been developing a machine-learning based “citator.” It’s a tool that lets lawyers and legal researchers know when a judicial decision they’ve found is still good law, or has been overturned.
And here’s the kicker: Once it’s finished, he’s planning to make the software free and open source.
The juggernauts of the legal research industry, LexisNexis and Westlaw, employ hundreds of attorney-editors to offer citator services as a core part of their offerings. Many lawyers are no doubt familiar with the yellow and red “flags” on court opinions found through their research tools, indicating that later rulings have either reversed or somehow affected that decision.
Those services may be well and good for “big, fancy law firms” that can afford the freight, Farhangi said, presenting his software at an event hosted by Stanford’s Center for Legal Informatics (CodeX) on a recent afternoon. But because it’s so critical for lawyers to know whether they’re citing valid cases, Farhangi said he wants to make that same tool more widely accessible. And some in the legal research industry say the project is poised to have a big impact.
“There’s no question that the prospect of an open source citator is tremendously exciting and would shift the competitive landscape,” said Pablo Arredondo, co-founder at AI legal research company Casetext. (Farhangi said he got much of the idea from conversations with Arredondo; Casetext is located a stone’s throw from Stanford’s campus.) “Having that ability outside of the duopoly of Lexis and Westlaw is a very big deal,” Arredondo added.
Ed Walters, CEO of Washington, D.C.-based Fastcase—which has its own algorithm-driven citator called Bad Law Bot—said “I love it,” when asked how the company felt about Farhangi’s project. “Anything that makes the traditional legal research services less inevitable is good for Fastcase.”
Farhangi cautions that the program is still a “work in progress.” He’s used his algorithm so far on only a limited sample of case decisions pulled from CourtListener, the free court opinion database.
Ashkon Farhangi (Courtesy photo)
The numbers so far, however, sound promising. Of the decisions that his algorithm flagged as containing “negative treatment” of a prior judicial opinion, it was accurate 89 percent of the time. The algorithm missed 18 percent of the negative treatments in the opinions it scanned.
The program is more sophisticated than the current iteration of Fastcase’s Bad Law Bot. Bad Law Bot currently cannot tell when one decision has directly overturned another, but only determines that a case is no longer good law by how it’s cited in later decisions. Farhangi’s project is able to pull language directly out of a decision to make the call that it is overturning another.
Walters said his company is incorporating algorithms that will catch when a ruling directly overturns another and hopes to make the system even more sophisticated in the future. The thing about Farhangi’s tool being open source is that—rather than being a competitor—it could be something that Fastcase could build on, helping it bolster its own citator, Walters noted. The same holds true for any other legaltech startup that may try to enter the space.
That’s part of the reason Farghangi’s project could have such a disruptive impact on the legal research industry. Providing free access to a citator that’s disaggregated from a larger suite of online legal research services could also give attorneys more freedom to pick and choose the services they want.
Still, even with all the advances in machine learning, others in the industry stress that interpreting judicial opinions is notoriously difficult and subjective.
“You’re never going to be in a position where that software is sufficiently accurate,” said Itai Gurari, who founded San Francisco-based legal research startup Judicata. “Reading this language is very hard, and it’s way beyond the scope of technology today.”
Gurari said his small company has built a version of an algorithmic citator that currently only includes California state court cases. But Judicata still also uses a human review process to categorize how one judicial decision is affecting another. It’s a labor-intensive process.
The need to employ humans to review legal decisions for a citator is, of course, a big part of the reason why the two main legal research services tend to be more costly.
Thomson Reuters, which owns Westlaw, sees the human element as essential to ensuring the reliability of its citator service, called KeyCite. According to Leann Blanchfield, vice president of content, strategy and editorial at Thomson Reuters Legal, the company employs some 600 attorney-editors.
“I believe attorney-editors’ ability to understand the nuance of the law will always be needed to be a premier product,” said Chris Fenne, the company’s director of product management for judicial content and KeyCite. Fenne also rebuffed the notion that Westlaw’s legal research services are out of reach cost-wise to smaller firms and solo practitioners, saying that it offers different packages tailored to the needs of the lawyer or firm.
Lexis’ citator, called Shepard’s Citations Service, also uses attorney-editors in its review process, according to the company’s website. Lexis declined to comment for this story through a spokesman.
Farhangi admits his program isn’t yet able to identify more nuanced treatment of earlier caselaw. The algorithm puts decisions into binary categories of either being overturned entirely—red-flagged—or not. “In a nutshell, my project provides a simpler output than the output of the big legal research services’ citators,” he explained.
But it’s a start. And as for why he doesn’t plan to commercialize the product? “There’s a little bit of altruism there,” Farhangi said.
