Is keyword search a filter prefatory to review or a proxy for review? It’s a question you’ll soon face. When counsel hammer out agreed search terms and queries to run against electronically stored information, requesting parties often expect that any responsive item (that is, all "hits") must be produced unless withheld as privileged.

Put another way, requesting parties frequently believe that by agreeing to the use of a set of keywords as a proxy for attorney review, those agreed searches serve as a de facto request for production and define responsiveness per se, requiring production if not privileged.

That stance upsets producing parties, triggering reactions such as:

• "We always have the right to review items hit for responsiveness!"

• "Requests for Production, not keyword hits, define scope of discovery!"

• "Nothing in the rules or the case law requires us to produce nonresponsive items!" [Expletives omitted]

Perhaps. But there’s sufficient ambiguity surrounding the issue to prompt prudent counsel to address the point explicitly when negotiating keyword search protocols and drafting memorializing agreements. Different expectations flow from different incentives behind agreements to confine the scope of search using queries and keywords:

• Federal law requires producing parties to search all reasonably accessible sources of ESI that may hold responsive information — a broad mandate. It’s no wonder producing parties seek agreements to limit the scope of review to only items with keyword hits. It eases their burden, trims costs, and (best of all) affords them cover from complaints about scope and methodology.

• By swapping keyword culling for human review, requesting parties seek to diminish reliance on an untrusted opponent’s self-interested assessment of the material. Their quid pro quo for limiting the scope of search is the expectation that, if it’s hit by the agreed-upon keywords, the item will be produced unless privileged.

In effect, requesting parties regard an agreement to use queries as an agreement to treat those queries as requests for production. Producing parties who reject this thinking would nevertheless be wise to plan for opponents (and judges) who embrace it.

Competing Ambitions

The use of negotiated electronic search and culling criteria is a relatively new development in the law, yet it’s become a de rigueur approach to dealing with ESI. Driven by competing ambitions to narrow the scope of discovery yet protect its integrity, opposing counsel are moved to negotiate objective search and culling criteria as benign as date intervals and file types, as contentious as custodians and queries and as complex as sampling methodologies and search algorithms.

Coincident with the growth of agreed search is the fall off in attorney eyes-on linear review of collections. Brute force review is deemed too costly by those who believe human review works and too wasteful by those who know how poorly human reviewers perform. Both contingents expect technology to solve the problems technology wrought. Technology can almost certainly do so, but not without re-examining cherished notions of legal counsel’s role.

American discovery law is founded on the premise that lawyers supervise and mediate efforts to identify information that will be produced. Implicit in this are the elements of lawyer discretion and professional responsibility; that is, courts accord substantial deference to the process and judgment (i.e., to attorney work product) presumed to have been employed to identify relevant evidence and to produce or withhold same.

Objective culling, keyword search, and emerging technologies such as predictive coding make clear that the idealized view of counsel as ultimate arbiter of relevance is mostly myth. Consequently, as more parties forge detailed agreements establishing objective evidentiary identifiers such as dates, sources, custodians, circulation, data types and lexical content, litigants and courts grow impatient with the cost and time required for attorney review and are reluctant to give it deference.

Broader use of mechanized search and technology-assisted review brings us closer to the day that counsel’s act of countermanding a machine’s characterization of a document as "responsive" will be viewed with suspicion. In the face of multiple objective indicia of relevance, producing counsel may be required to defend subjective decisions not to produce on grounds of relevance — assessments once immune from scrutiny.

But ambiguity and unmet expectations aren’t just problems for requesting parties. Responding parties may contend that their agreement to deploy a requester’s keyword searches frees them from a further obligation to review and cull documents for responsiveness (in much the same way as they currently assume use of agreed-upon keyword searches frees them from an obligation to review documents not turned up by keyword search). In turn, requesting parties may object to productions rife with noisy, irrelevant keyword hits as obstructive "data dumps."

So, is keyword search a filter prefatory to review or a proxy for review? Reasonable minds may differ, such that the best course is to set expectations by expressly setting out the respective roles of search and review when agreements to use keywords are reached. Don’t simply recite, "The responding party will search its collections for documents containing the following keywords," without also stating whether the responding party is permitted or obliged to review the results of keyword search with an eye toward their responsiveness to formal requests for production.


Responding Party shall be obliged to interrogate reasonably accessible, electronically searchable sources using the agreed-upon search terms and queries, review the search results and produce only such non-privileged items as Responding Party deems responsive to outstanding Requests for Production, subject to objections previously lodged to such Requests.

Compare it to:

Responding Party shall be obliged to produce all nonprivileged items identified using the agreed-upon search terms and queries and may not withhold nonprivileged items on the grounds that they are not relevant or not responsive to a request for production. The use of agreed-upon search terms and queries does not relieve the Responding Party of its obligation to act with reasonable diligence to identify potentially responsive information by other means.

In the first formulation, keyword search is simply a filter, limiting the scope of review but not dispensing with subjective assessment of responsiveness to requests for production. In the second, keyword search serves as an objective criterion for production, uncoupled from any right or duty to measure responsiveness against requests for production. Neither formulation is superior to the other, but each implicates markedly different duties and expectations.

Negotiated protocols are upending traditional notions respecting the limits and duties of discovery. Producing counsel cannot expect opponents to devote significant effort to negotiating objective search and culling criteria and then voice no objection when documents with hits aren’t produced. Likewise, producing counsel has little incentive to run an opponent’s proposed search unless doing so reins in some significant element of cost, burden, or risk.

When parties negotiate agreements largely without precedent in discovery jurisprudence, the old assumptions no longer apply. Accordingly, counsel negotiating objective culling and search criteria must think through and spell out the consequences of objective culling and search.

Practice Tip

If you enter into an agreement with the other side to use keywords and queries for search, be clear about the disposition of items hit by queries. Assuming the items aren’t privileged, are they deemed responsive because they met the criteria used for search, or is the producing party permitted or obliged to further cull for responsiveness based on the operative Requests for Production? You may think this is clear to your opponent, but don’t count on it. Likewise, don’t assume the court shares your interpretation of the protocol. Just settling upon an agreed list of queries may not assure a meeting of the minds. •