The Georgetown Law Advanced E-Discovery Institute, held in December in Tysons Corner, Va., was an excellent conference. It boldly addressed the numerous issues that surfaced last year around the concepts of technology-assisted review — aka computer-assisted review or predictive coding, all variants of lurching moves towards more efficient search and review of data.
If you polled the 600 attendees for a definition of these terms, you would no doubt get 600 (or more) answers. The conference reinforced the undisputed fact that we are in the very early stages of using advanced technology for electronic data discovery. Few legal professionals believe that discovery can actually be provided in a box, or that it can possibly be anywhere near simple.
By the conclusion of the conference, I was convinced that confusion and uncertainty are the most prevalent emotions. Why? Comedian Stephen Wright once said, "If you were speeding through space at the speed of light, would you know if your headlights were on?" As of today, a small segment of the judiciary and lawyers are indeed speeding along in attempting to use TAR on a broader basis, but even this group is a bit uncertain as to exactly where the spaceship is heading.
The old ways of searching for documents have seemingly been discredited. It is now claimed by a small but expanding group of judges that if you are not using some variation of TAR, you are — or soon may be — falling below professional standards. There is a slight problem, however, because there are no common standards or clear protocols governing how and when these technologies should be applied.
It was quite vexing that several of the judges at the conference made it clear that they do not know how these technologies actually function, and likely never will. This, of course, is not a critical defect — because they can rely on technology experts to clarify particular issues as they arise. But does this not potentially bring us back to where the focus of discovery reverts to a contest of experts, creating a new set of issues regarding the reliability of various technologies and workflow to occupy the court?
I am very much in favor of TAR. In most variations, if applied properly and for the right types of cases, it should increase the accuracy of data searches. Implementing sampling and workflow efficiencies should result in a cost-effective result for the client. Unfortunately, surveying the response to the use of TAR, even within this select group of lawyers, we are not there yet. Only a very small segment of the bar is currently relying upon TAR.
While recent case law has permitted or required the use of TAR, there is no consensus among the judiciary as to when and how these technologies should be applied. Clearly, we are at a nascent point in the development of case law. These technologies are based on math, science, and statistics; this area remains a moving target, with little common ground on how data should be collected, sampled, searched, or reviewed. Because most of us do not understand the complicated concepts "behind the curtain," there is a strong reluctance to simply accept the "trust me, it's all (or enough of it) there" assurances of our adversary.
For example, there are no acceptable standards yet as to what constitutes a valid response to a discovery request using technology. The discussion inevitably turns towards the right of an attorney to examine the adversary's seed (sampling) sets, workflow, and non-responsive document collections. Of course, then comes a discussion of how large must a seed set be, why that size, etc.
We are becoming familiar with the concepts of "precision" and "recall," and are now being presented with "rules-based" and "machine-learning algorithms," TAR training set workflows, and confidence levels/intervals.
But we still are very much at the "trust but verify" stage of TAR, and attorneys still worry that we may be giving up the ability to obtain what we seek in discovery. And we have yet to see the consequences of not selecting the proper tool for the data set in a case.
Subscribe to Law Technology News
