Active machine learning — an element of technology assisted review — is a disruptive technology, as Maura Grossman and Gordon Cormack observed in the preamble to their Glossary of Technology-Assisted Review. TAR (a.k.a. predictive coding) is not just marketing hype, although most executives in charge of legal software companies wish it were.
Indeed, those executives are clueless in the face of real change. They stand still, or at best, make slow, cautious upgrades. Most seem oblivious of the fact that their once proud companies are rapidly becoming e-discovery BlackBerrys. Their sales staff and experts must be telling them to act, but they do not hear, do not lead. They are empty suits.
Yes, things are changing very rapidly in legal review technology — too fast for most established companies to keep up. It takes visionary management to do that, plus they must have the ability to execute quickly. It is not enough to know what to do, you actually have to do it, and do it quickly. That is the key to success with high-tech, a fact Silicon Valley entrepreneurs know well. But most corporations outside of the youth-oriented tech bubble lack that kind of leadership.
The e-discovery industry is no exception. Most CEOs have never even heard of Mark Zuckerberg’s “Hacker Way,” much less learned how to manage in a bold way. (See “Impactful, fast, bold, open, values: Guidance of the ‘Hacker Way’” and “‘The Hacker Way’ — What e-Discovery Can Learn From Facebook’s Culture and Management.”)
Without understanding the importance of change and decisive action, there is no way to stay relevant in the software business today, to attract the best talent, and create insanely great high-tech products. Moving slowly and taking little steps is the exact opposite of the management style that works in today’s high-tech world. Learn from the successful start-ups in Silicon Valley, encourage a culture of hands-on meritocracy, of continuous improvement.
When are the CEOs of legal software companies going to join the 21st century? When are they going to take bold, decisive action? What happened to the courage of the suits in charge? Are there no hackers in the positive sense of that word in legal search software? No Steven Jobs among us? Is it all just Dilbert?
We need management to embrace the values of the legal justice system, not treat lawyers like another huckster market. We need leadership in the legal software C-suite. We need CEOs to follow “The Hacker Way” and act fast, act bold and go for the maximum possible impact on society. We need CEOs who encourage this kind of open culture — and products that are driven by true value, not “value propositions.” We need software companies where code wins arguments, not pointy-haired spreadsheet pushers.
We need executives to grasp what Grossman, Cormack and many other experts have been telling us — that we are dealing with a bona fide paradigm shift here in review software. (See Baron, J. & Paul, G., “Information Inflation: Can the Legal System Adapt?”) Because most of the C-suite is asleep in last century slumber, it is especially important for buyers to beware. I am talking about the buyers in the corporate law departments and law firms who decide what software to use or purchase for search and review, who choose outsourcing options.
• The message for my fellow lawyers and technologists who are in charge of such decisions is simple: Look out, be careful what you buy, or you could end up investing in a falling star. You could end up with a review tool that looks more like a BlackBerry than an iPhone.
• My message to e-discovery vendors who do not make their own software is essentially the same. Pay for the scientific and other specialist advisors you need to help you identify the best software to offer to your customers and the best methods for them to use that software.
• My message to everyone interested in legal search and review is also simple. Predictive coding software is not a commodity that you can buy based on price or brand alone and get pretty much the same product. Like any other new, complex technology, some predictive software is far better than others. Some relevance ranking systems work better than others. Some artificial intelligence code works better than others. Some of the software on the market has solid science behind it. Some does not. And do not think that just because there is a patent with a fancy name that there is solid science behind the software, or that the software works as advertised. There is more to it than that.
Some predictive coding software programs on the market today were cheaply built, and built in a hurry. Some were not. Some were well designed — and as a consequence are intuitive to use. Some were not and are difficult to use. Some of the software on the market is effective, getting better fast, and leverages the skills of experienced searchers. Some software houses understand the importance of human/computer information retrieval. Some do not. Some software is multimodal and harnesses the power of all search methods, not just predictive coding or blind chance. Some do not.
It gets worse. No one in the software industry yet provides effective, in-depth training on the new methods to use the tools. Hopefully that will change soon because the best tools in the world are useless without knowing how to use them. If you create a disruptive technology that works differently than anything available before, you must offer training. Otherwise the market will just look and talk, not buy and use. It is only by using this kind of breakthrough technology, and using it properly, that you will understand how good it is. It is one thing to read Enron search narratives and hear about review speeds of thousands of files per hour, and costs of less than a penny a document. It is quite another to try it yourself. Only then will the Kool-Aid really kick in.
Due to all of these failings — not of technology, not of science, but of management — there are now significant differences in vendor quality. All predictive coding software is not created alike. Do your homework, learn about predictive coding. But beware of vendor white papers; they have too much skin in the game to be objective.
A few vendors understand what is happening. Fewer still are able to make the investments and changes needed to create the insanely great products that our legal profession needs in order to survive the data deluge. No vendor has established effective training, but that should come soon. Some leaders are embracing Facebook’s Hacker values. The few who do will prosper and be around to help our children. The rest will soon retire and dissolve. You do not want to be left holding the bag of their obsolete technology. •