Artificial Intelligence

Perhaps legal has been going about this artificial intelligence (AI) question all wrong.

“AI is a hot topic in legal technology, but the question gets asked in the wrong way,” said Brad Blickstein, head of Blickstein Group. “It's become a top-down thing: What are we doing about AI? It's like asking, what are we doing about databases? It's a crazy question. The question should be, what problems do we have, how do we solve them, and is AI or some semblance of AI a potential solution for that?”

It's that idea of finding a specific problem to solve that is at the center of the next project for the Law Department Operations Survey publisher. Blickstein Group is currently developing its first “Blickstein Group Legal AI Efficacy Report,” which will analyze AI solutions currently in the legal tech market for how effective they are at solving their intended problem.

The report aims to cover around 60 solutions in its first iteration and will be released in Q3 2019—by ILTACON 2019 at the latest. The report is geared towards current and potential buyers of legal AI products, including both corporate legal departments and law firms, and will be available via the Blickstein Group website. Research into the AI solutions began in earnest last week and will continue for the next roughly three months.

Importantly, Blickstein did note as well that the report is independent and will not be accepting advertising, and AI solution companies do not need to pay to be covered. He added, “You can't buy your way into the report; you can't buy your way into us covering you.”

The project has been gestating for about a year, but kicked into high gear recently with the addition of Erin Harrison to the project. Both Blickstein and Harrison will conduct research on the different solutions, while Harrison, a former editor-in-chief of Legaltech News (full disclosure: I worked with her at both LTN and InsideCounsel magazines), will be the project manager and write most of the final report.

Harrison told LTN that she “jumped at the opportunity because it is a pivotal time to educate the market while also hopefully affecting the progress of legal AI from conception to implementation.”

She added, “Our primary goal is to evaluate each technology based on how 'effective' a product is at solving a given problem for specific use cases. As for the criteria, it varies based on what problem the technology purports to solve: For example, a legal research technology may claim to increase user speed or produce more relevant results, while a prediction tool may say it helps mitigate risk or more accurately predicts cost.”

Blickstein Group's research will begin with the AI solution provider filling out a survey detailing the product and the problem they intend to solve. Following those initial steps will come a briefing, where Blickstein explained, “we'll come from the position of skeptical people that say, 'OK, you said all this. Make us believe. Show us, tell us, convince us that what you say is true.”

But this all begs the question: What exactly is the definition of AI the report will be using? Blickstein said he tends to be over-inclusive rather than provide strict guidelines, since the point is to judge problem solving efficacy rather than analyze the technology itself. “Whether it's machine learning or natural language processing—some claim those aren't really AI, but I don't have a taste for that argument. I don't think it adds much value,” he added.

To help guide the report, Blickstein has enlisted an advisory board of legal AI buyers to help guide the project. Dera Nevin, an attorney in Baker McKenzie's information governance and e-discovery group, told LTN that having a report like this is crucial for those interested in AI to even know where to begin.

“It's like walking into Home Depot. You can be overwhelmed by the sheer range, but if you know you're going to electrical rather than plumbing, it helps you pick the right aisle,” Nevin said.

She said she views the AI market currently as being in a similar place as e-discovery a little more than a decade ago—solutions existed in the marketplace, but quality was not consistent, there was little interoperability between systems, and pricing all over the map. Then came the EDRM model, which helped establish guidelines and a common language to help pull the entire market towards maturity.

Similarly with AI solutions today, she said, “There's enough people in the market that have gone down this pathway that we can collectively pull our expertise and come up with something that we can all agree on.”

Jeff Marple, director of innovation for Liberty Mutual's legal department and also on the advisory board, agrees that the time is now for an AI report, as refinement is coming into these solutions.

“A couple of years ago, we were beaten about the face and neck with the term AI,” Marple said. “But as the buyers have become more educated and we've come to understand what artificial intelligence means, the very different facets of it and how it might be applied, I think you're getting more and more true product development as compared to technology development.”

And this development isn't going away. Blickstein said he intends the report to be a living document—solutions will be added or updated as major changes are made in quarterly updates, and each solution that has not been updated for a full year will be given a new evaluation.

That's important, because Marple noted that for buyers, learning these distinctions between solutions will be continue to be crucial: “As the marketplace becomes more and more technologically advanced, there is more need for, if you haven't adopted that technology, to start to because you're going to be left behind. … I don't know if it's exactly there, but it's close.”