Credit: Andrea Cobb

The billable hour has plenty of enemies, but Wendy Rubas wasn't always among them.

As an associate at McDermott Will & Emery's Chicago office from 1999 to 2004, she felt that, even if it was a taskmaster, it served another purpose—a measure of her value to the firm.

Even before there was a billable hour target to hit, Rubas' grades in law school served as a barometer of her hard work. Moving her way through the great sorting of law school and Big Law associate life, there was always a reliable metric to tell her where she stood.

Of all the changes that would occur when Rubas moved in-house to a Chicago-based health-care system in 2004, perhaps none was as destabilizing as losing a cut-and-dry number to prove her worth. In business meetings at her hospital, doctors discussed how many lives they had saved and businesspeople showed how much spending they had cut. Winning a summary judgment didn't seem to stack up.

“I didn't have any goal that is expected and nothing to say I'm meeting it,” Rubas said. “It's not defined. So I started to just define it.”

Rubas is now the general counsel of VillageMD, a tech-focused company helping primary care physicians transition to a new value-based payment system. In her nearly 15 years as an in-house lawyer, she has gone further than most toward capturing and modeling data that shows her clients the costs associated with legal events.

Wendy Rubas, general counsel of VillageMD.

Through automated reports, she has wound up tracking the relative value of legal work. And that has taken her to an even more unexpected place: changing the way she thinks about what lawyers should be doing. For instance, she's not all that interested in spending time negotiating software licenses or nondisclosure agreements. They rarely are “a cause of loss,” she says.

“What would happen if you signed a terrible NDA?” she asks, before whispering the answer: “Probably nothing. So what are we doing all this negotiating for? That's the behavior change that has been the most radical for me.”

Rubas says, in order for this idea to proliferate through the legal market, there needs to be a common language for legal work and outcomes that can be shared across industries and law firms to provide a detailed, comparable understanding of what is happening in legal departments and law firms across the country. That would help the industry know what it should be spending its time on, rather than finding places it where it can spend it.

This idea is separately picking up steam in Big Law firms, corporate legal departments and technology companies. Creating a data-tracking system that could measure legal events and outcomes—in addition to the costs associated with doing legal work—could have radical consequences for the industry. The long-term impact of a legal market that competes on the quality of outcomes rather than quantity of effort could be an upending of the traditional metrics that lawyers like Rubas have long relied on to validate their places in the legal ecosystem: namely, law school and law firm brand names. When clients can judge outcomes, performance is more likely to be rewarded than credentials.

But first, a reliable data-capturing system must be put in place, which raises a few questions. What would that system measure—and how? Who would do the measuring? Answers are slowly coming into view. And beyond that horizon, the profession will confront the question that may be at the heart of it all: Will lawyers be willing to overcome professional biases to apply statistics-based decision-making to their practices?

StatCast for Lawyers

One of the most visible examples of how a new data-tracking system can change an industry's self-evaluation comes, strangely enough, from the baseball diamond.

Babe Ruth may have made the home run famous, but it took a century after he first stepped into the batter's box for baseball teams to properly value the home run. Major League Baseball teams hit 6,105 home runs in 2017, setting an all-time record and blowing away the previous high of 5,693 set in 2000, the heart of the steroid era.

There are a number of reasons for this increasingly feast-or-famine approach, including theories that baseballs themselves have been “juiced,” but most anybody in the game will credit the introduction of a slew of new data points and statistics gathered by a system called StatCast that MLB brought online in 2015. StatCast is a series of cameras and sensors in all 30 baseball stadiums that measure the physics of the sport.

As with any data point, these new numbers mean nothing until they're put into context. Blending a mixture of all its measurements, StatCast knows how frequently any batted ball will be caught by a defender. That is a statistic now known, simply, as “catch probability.” It is a finite number that hitters attempt to minimize. The way to do that, the numbers show, is to hit the ball higher and harder. It's not so much a focus on home runs that has resulted in their spike, but rather, in the new language of baseball, a focus on the ball's “exit velocity” off the bat.

Andrew Baker, a senior director at HBR Consulting who has helped law firms and clients like Rubas develop data-capturing systems, says he sees a number of similarities between baseball and the legal profession. He expects there would be similar changes to lawyers' behaviors if there were a better way to measure the impact of their work.

“We're silly to assume that there wouldn't be an insight like [exit velocity] in legal,” Baker says. “You have a guild that has operated in almost exactly the same way for a very long time, and it has done so pretty much exclusively relying on subjective measurements. There are no studies that suggest how we practice law right now is optimal.”

Persuasion Rate

Make no mistake, practicing law is nowhere near as amenable to data analysis as baseball. There are no home runs for lawyers. Even if there were, they would differ from litigation to regulatory to transactional practices by each matter.

But by no means is the law incompatible with data analysis. Physics-tracking cameras will play no role in that analysis, but one technology that will is computers' increasing ability to understand text. That is the technology underlying LexisNexis Context. Launched in late 2018, Context has analyzed rulings from each U.S. district court judge on 100 different types of motions. The analysis can tell lawyers how likely a given judge is to grant or deny a motion for dismissal, as well as how frequently the judge cites any given case in deciding that motion.

Judge William Alsup of the U.S. District Court for the Northern District of California, for instance, has ruled on 864 motions to dismiss, granting 52 percent and tossing 25 percent of them. In doing so, he is most likely to cite Bell Atlantic v. Twombly (425 appearances), a watershed 2007 U.S. Supreme Court case that raised the bar a plaintiff must meet in order to proceed.

Nik Reed of Ravel Law.

The data gives lawyers insights that they previously could only access by anecdote and through speaking with a judge's clerk, says Nik Reed, co-founder of Ravel Law, which LexisNexis purchased and refitted to create Context.

“If you read 1,000 documents, you're probably not going to notice that one case was cited 50 times,” Reed says. “But that's something that computers are really good at spotting.”

Applying a similar analysis to individual lawyers' briefs could be the context that turns that data into something powerful enough to change purchasing decisions. Matching up lawyers' briefs with judges' decisions could tell clients how frequently a lawyer's case citations are persuasive for any given judge. “Persuasion rate” could be the new defining measure of a litigator's success.

“When the technology is there, when the process is there and when the data is right, it shows that the possibility is ­really limitless,” says Mark Koussa, director of product ­management at LexisNexis.

Lawsuits as Data Points

LegalMation is another company hard at work turning legal documents into data points. Today, the company's software can generate automated responses to lawsuits in as little as two minutes. The product is being used by companies as big as Walmart and law firms including Ogletree, Deakins, Nash, Smoak & Stewart.

But spitting out responses to complaints is just one product that LegalMation's software generates. Its larger ability is to turn lawsuits into data points that will allow for a new level of analysis that James Lee, the company's co-founder, compares to a “digital fingerprint” for a lawsuit. It is called an “entity relationship” report that tracks about 500 sortable data points.

For a personal injury lawsuit, it can determine what body part is involved in an injury, how many times a defendant has been accused of the same wrongful act, whether a substance is said to have caused a slip-and-fall and more.

The software also can decipher aspects of a lawsuit that don't seem as straightforward. For instance, it can tell when a client issued an apology as a result of the alleged behavior. With enough data, it could determine whether apologies are helpful or harmful as it relates to payouts.

Lee says he believes this type of analysis will ultimately change the competitive landscape for law firms, allowing firms and clients alike to anonymize data as a way to create benchmarks in the industry. Clients could make purchasing decisions based on that data.

“That is what this is all about: getting better, more precise information to everybody in the marketplace, so we can all become better lawyers and deliver better outcomes,” Lee says.

A Standard Taxonomy

While efforts to turn legal documents into data could introduce new ways of analyzing legal issues, there also is a broader effort underway to turn the actual work lawyers do into data that is comparable across law firms. But firms may not be tracking the most useful data to best conduct that analysis.

Most of today's legal data pertains to time and billing. That data is often next to useless, many lawyers say, because bad time-keeping habits and the current system of task-based coding is not granular enough to measure what actually happened in a given matter.

Toby Brown, chief practice management officer at Perkins Coie.

Toby Brown, chief practice management officer at Perkins Coie, says he became a “great task-code basher” in presentations at legal industry conferences. He's set out to solve this problem by creating a standard taxonomy of legal work that he hopes will be adopted industrywide. It is the same potential solution to the problem that Rubas, the VillageMD general counsel, had separately theorized.

The standards organization Brown helped to create is called the SALI Alliance, which stands for Standards Advancement for the Legal Industry. The alliance has a long list of prominent members and supporters, including large law firms like Perkins Coie, Holland & Knight and Greenberg Traurig. Major clients supporting the initiative include Shell, GlaxoSmithKline, Citigroup and Deutsche Bank.

The alliance wants to create a common language to describe legal matters, similar to the way products are itemized by “stock-keeping units.” In legal, that could present itself as a string of letters and numbers that characterize work by matter type, industry, jurisdiction and other aspects. Ready for an example? Well, a labor and employment class action dispute based in New York might be coded something like this: LEM-EMP/D-CCD(CLA)/USA-NY-NY.

If this language were baked into technology and billing platforms and used by different law firms, clients would be able to compare their firms' work, the founders of the standards initiative say.

“Right now it's a guessing game,” says James Hannigan, senior manager of product development and project management at Allen Matkins, who is a member of SALI's standards committee. “We spent $1 million on this patent lawsuit. What happened? I don't know. This would let you analyze outcome to price and inputs.”

SALI launched a test batch of standard codes at the Legalweek New York conference in late January.

“Standards come about when an industry realizes not having standards is hampering innovation,” Brown says. “Take electrical outlets. If everything had a differently shaped plug, people wouldn't use electricity the way they do now. It would be too much of a hassle.”

Uber for Law

The legal industry is painfully slow to adopt major changes, and lawyers often say work can't be easily categorized or simplified. SALI has a long way to go to garner adoption throughout the industry. If it is successful, there is the potential for dramatic follow-on effects once the data is put into context.

One executive at an Am Law 100 firm, who asked not to be identified, says lawyers are able to charge higher rates due to a lack of understanding of the work they do. That dynamic won't last forever, the executive says, and he has developed a theory about how the market will transform once there is a uniform way to describe and price legal work and success. Fair warning: It reads like a dystopian vision of Big Law's future.

If there were marketwide understanding of, say, how many mergers and acquisitions each law firm handles, how much they charge for those services, how quickly they completed them and how successful they are, then clients could select law firms based on much more relevant metrics, the executive says. Nobody should be upset by the idea of clients making informed purchasing decisions.

But it likely wouldn't stop there, the executive says. It is likely that, if such information existed, the company that owned it would serve as a marketplace for lawyers. It would match its knowledge of lawyers' work and price history with clients' desires.

“Take the Uber analysis and imagine 'Lawber,'” the executive says. “That's the way our clients will, in a few years from now, buy our legal services. They will say, 'Here is my problem, and here are my levers: price, quality, safety.' There is a mixture there that they can select, and then out comes a law firm or a legal team that is assigned the work.”

The executive suggests that consulting firms like the Big Four or billing technology providers are best positioned to serve this role in the market. The key is having a platform that has analyzed a vast enough swath of legal purchases and prices to set the market.

Even if the “Lawber” theory sounds extreme, plenty of legal market insiders see valid points in certain aspects of the analogy. Raj Goyle, a co-founder of the legal database company Bodhala, says his company, which analyzes billing data, can already go a long way toward optimizing legal spend to efficient law firms.

“It is crazy that the buy-side of this market does not use its data to actually know precisely how much they are willing to pay or what is the fair-market wage with a lawyer getting some margin on these services,” Goyle says. “That is the ultimate destination.”

Professional Bias

The Lawber concept is far from inevitable, says William Henderson, an Indiana University Maurer School of Law professor. Lawyers have spent decades pushing back against the idea that a statistical analysis of the practice of law would lead to better decisions than those made by lawyers themselves.

“I'm very, very bearish on the uptake of data, because the lawyers operate so much on the belief that they can outperform the machine,” Henderson says. “There is a huge cultural constraint here.”

One lawyer who has seen this response firsthand is Randall Kiser, a former real estate lawyer in California for 20 years who left the practice to study business and rational decision-making. In 2008, he published a paper in the Journal of Empirical Legal Studies that showed lawyers, as a group, had largely been making bad decisions when taking civil cases to trial.

The paper was the result of comparing the final settlement offers to the trial outcomes in 2,054 cases in New York from 2002 to 2005. The findings showed that plaintiffs made the wrong decision in roughly 60 percent of cases, taking home an average of $70,000 less as a result of trial than the last settlement they were offered. Defendants made the wrong decision to go to trial about a quarter of the time; in those cases, they paid $1.4 million more than they would have given away under the final settlement offer.

The findings also showed that some of the strongest signals of success in the legal market—experience, law school pedigree and the size of a lawyer's law firm—were not predictive of better decision-making, Kiser says.

“When I have personally presented this data to law firms, there has been a tremendous amount of pushback,” Kiser says. “I've had managing partners scream at me and walk out of meetings calling it bullshit.”

Growth Mindset

One potential solution to the fear of statistics can be seen, yet again, on the baseball diamond. In his book “Astroball: The New Way to Win It All,” Ben Reiter details how the Houston Astros weaponized a statistical approach to the game to win a World Series. The Astros' executives, led by a former McKinsey consultant and a former NASA data scientist, looked for a trait in players they referred to as a “growth mindset.”

In practice, a growth mindset in a player meant they were more likely to be willing to change their approach to the game based on what data was telling them. Perhaps the best example of this was Dallas Keuchel, a 5-foot-9-inch pitcher who couldn't hit 90 mph on a radar gun. The Astros had tried to trade him before he ever made it to the Major Leagues.

Keuchel, frustrated with his early-career results, took to a group of data analysts the team employed. They showed him where certain hitters were most likely to hit the different pitches he threw. They told him how to rearrange the defense behind him to be in the best position to field those balls. In 2015, he won the Cy Young Award as the best pitcher in the American League.

VillageMD's Rubas is a prime example of how unsettling it can be for professionals to shine the light of data on their work. She was often discouraged during her journey as she searched for ways to use data to limit clients' legal costs. Lawyers often get defensive when a client pokes holes in the data they present or when a legal department points out their problems. The metrics present a hard truth that can be tough to swallow.

Still, Rubas remains optimistic that lawyers will someday harness the outcomes of their work to make them more informed decision-makers—and to save money for their clients.

“We're going to have to change,” Rubas says of the legal industry. “Data models plus experts is definitely going to make us better—if that's our goal, to be better. Or is it just to be busy?”

Email: [email protected]