You have survived Step 1, Step 2 and Step 3 and are now tasked with actually reviewing all of the data you preserved. Your first job will be to determine what information is actually relevant. Then, of course, you will need to protect any privileged or confidential material. Electronically stored information is multiplying every day, and this directly impacts your litigation, most particularly the amount of data you will need to review. The more data you have to review, the more expensive and time consuming it will be. In fact, because traditional linear review can account for upwards of 73 percent of e-discovery costs, technology and other cost-saving methods must be considered.


The search & review step will require your consideration and application of some technology you are not familiar with and of which you might be skeptical. But as Mr. Ralph Losey says, “Our current linear, confrontative, one-dimensional, largely manual, Bates stamp approach to discovery must be replaced with a multidimensional, cooperative, iterative, largely automated, hash value approach.”

One of the themes in this articles series has been the need for lawyers to change the way they practice law and adopt new practices to allow for proper handling of e-discovery. The importance of embracing change is perhaps most applicable to the search and review process. The Sedona Conference agrees:

The legal profession is at a crossroads: the choice is between continuing to conduct discovery as it has “always been practiced” in a paper world – before the advent of computers, the Internet, and the exponential growth of electronically stored information (ESI) – or, alternatively, embracing new ways of thinking in today’s digital world. Cost-conscious clients and over-burdened judges are demanding that parties now undertake new approaches to solving litigation problems.


Before you do anything with the data you have preserved and collected, the first step should be bulk culling, which is the elimination of files from your review. There are several ways to cull data; however, you might start first by custodian. Best practices recommend you rank your custodians by the key players versus those who may have relevant evidence. The more complex the case, the more custodians you will likely have. Further bulk culling will then take place by date range, de-duplication, file types and keywords.

The use of keyword searching to cull an ESI data set prior to review has long been part of the typical e-discovery workflow. Keyword searching can narrow both the set of data that requires review as well as the data that you can discard without review. Courts will also encourage the parties to agree on a set of keyword searches at the outset of the case — remember cooperation — in an attempt to limit the data set and decrease litigation costs. However, with the advent of technology-assisted review (TAR), also referred to as computer-assisted review (CAR), practitioners need to re-examine the importance of keyword searching in the e-discovery workflow.

Keep in mind that bulk culling to remove system files and duplicates is a necessary first step; however, keyword searches are only as good and as thorough as the person who crafted it. Additionally, know that most of the bulk culling is technical work and will need to be done by an e-discovery service provider. However, it cannot be stressed enough that all of this work, whether technical, legal or a mixture of both, needs to be supervised by merits counsel, i.e., the attorney signing the pleadings under Rule 26(g).

Computer-assisted review

In the era of Big Data, the old ways of linear, paper review are being left behind and replaced by new approaches that involve iterative methods that include computer-assisted review and predictive coding. You would be hard-pressed not to have heard of predictive coding, but you may not fully understand what it is or why it is useful. You may also wonder if it can be trusted.

Computer-assisted review means much more than simply reviewing and coding documents on a computer — it suggests a methodology by which attorneys use computer software as a search tool to find relevant evidence from a larger universe of documents. Even though you have already culled the data set down through the bulk culling process, it is not reasonable to then review all of the documents that might be relevant, particularly in a very large data set. Mr. Ralph Losey is clear about this point: “Individual attorney review of all documents that survive bulk culling is not a best practice.” Further, remember that keyword searching is only as good as the person who crafted the search. Therefore, CAR will not only be used for the review and coding of documents, but also to further search and cull down the dataset. A predictive coding platform, when applied and managed correctly, can and will do a much better job of finding relevant documents within your data set than a keyword search possibly could. With a properly trained assisted review tool, you can review a portion of your data set and verify with about 95 percent confidence that you have found all of the relevant documents.

Not only does the use of assisted review eliminate the practical need for keyword culling, but the use of keyword culling in an assisted review workflow can be problematic. Predictive coding itself serves as the best culling tool by scanning data to determine what is likely responsive and what is not, to arrive at a set of data that is at least worthy of review. To properly train the assisted review tool, you need to provide the system with good examples of both responsive and non-responsive documents. Culling a large percentage of your non-responsive documents prior to training the system will yield fewer examples of the various types of non-responsive documents that exist in your data collection. This means that your seed documents are less likely to include examples of each type of non-responsive document, and your system will not be properly trained to categorize all of the non-responsive documents in your data set.

To ensure the best results, you must take the time to create a proper and defensible workflow. You will need to work with an expert but supervise the process at each step so that you are making the legal decisions and using your vast knowledge of the case to inform all of the searches and inputs.


The protections phase includes the quality control necessary to ensure a defensible and accurate review, as well as the protection of privileged information from inadvertent disclosure. The latter is a significant concern particularly with large data sets. However, some judge will as a matter of course issue an order pursuant to Fed. R. Evid. 502(d) that allows for clawback of inadvertently produced privileged information. If your judge does not do this as a matter of course, it is considered a best practice to move the court for such an order.

Other obvious protections include redactions and privilege logs, both of which are completed by attorneys during the review phase. However, besides these obvious legal obligations, best practices also require that during the review you follow an established protocol that is observed and managed by a legal project manager. Successful project management depends upon leadership, tailoring, expertise, adaptability, measurement, documentation and transparency. By continually measuring and documenting through statistical sampling and metrics-based quality control systems, you can establish a defensible and reliable review process. Such a process allows for cost-effective and proportional discovery.

As we discussed in the cooperation phase, one of your main goals of discovery should be proportionality. Proportional discovery means cost-efficient discovery. Therefore, e-discovery best practices require that attorneys apply the latest technology and adopt new methodologies to realize the goal of proportionality.