[co-author: Ronald J. Hedges]
The integration of AI, including agentic AI, into eDiscovery workflows represents a significant evolution in how legal teams manage the review and production of electronically stored information (ESI). As with any technological advancement, the use of AI introduces both opportunities and challenges, particularly when it comes to defensibility. Legal practitioners must be prepared to justify their use of AI tools under both ideal and adversarial conditions. This article explores some of the core elements shared by all eDiscovery workflows and examines how AI fits within existing legal standards and expectations.
By analyzing best and worst-case scenarios, this article provides a framework for understanding the defensibility of AI review. In favorable circumstances, the result of AI is considered no different from human reviewers, allowing for efficient and confidential workflows. In more contentious situations, however, the producing party may face heightened scrutiny and demands for transparency. We also consider what practical cooperation looks like between parties, emphasizing the importance of reasonableness, transparency, and adherence to established principles such as those outlined by The Sedona Conference, and what judges expect from Rule 26(f) meet and confers.
What All eDiscovery Workflows Have in Common
Regardless of the specific tools or technologies employed, all eDiscovery workflows share a foundational structure that could include negotiation with opposing counsel. This negotiation may involve the scope of discovery, custodians, date ranges, or the format of production, among many other factors. Even when advanced technologies like AI are used, the underlying requirement to engage with the opposing party remains. These discussions, starting with Rule 26(f) meet and confers, are essential to ensuring that both sides understand the parameters of discovery and can raise concerns early in the process.
Under Federal Rule of Civil Procedure 26(f), parties are required to meet early in the litigation process to discuss a discovery plan, including issues related to ESI. This meet and confer is a critical opportunity to address the role of technology in the discovery process, such as the use of AI agents for assistance with identification and preservation, AI review, traditional technology-assisted review (TAR), search terms, or other advanced tools. Counsel are expected to be competent in these technologies or to consult with experts who are, as courts now view technological literacy as part of a lawyer’s duty of competence. Discussions should cover the sources and formats of ESI, methods for data preservation and collection, and protocols for search and review. Proactively addressing these topics helps ensure that discovery is conducted in a manner that is reasonable, proportional, and defensible, while also minimizing the risk of disputes or sanctions later in the case.
Another commonality is the potential for mistakes. Whether review for responsiveness and privilege is conducted manually by lawyers or assisted by technology, including AI, errors such as misclassification, missed documents, or overproduction often occur. Similarly, agentic AI and humans may both introduce errors during the identification phase of implementing a legal hold. These risks necessitate quality control measures and documentation. Importantly, the presence of mistakes does not inherently undermine the defensibility of a workflow; what matters is how those mistakes are addressed and whether the overall process was reasonable and proportional.
Finally, all review workflows, whether traditional linear or technology-assisted review, can be evaluated using similar metrics and standards. Precision, recall, and elusion rates are applicable across methodologies and workflows. This consistency allows courts and parties to assess the effectiveness of a review process without bias toward the technology used. The key is transparency and the ability to demonstrate that the process was thoughtfully designed and executed.
Best Case Scenario: AI Facilitated Review Is Held to the Same Standard As Human Review
In the best-case scenario, courts and opposing parties treat AI review with the same level of trust and scrutiny as traditional human review. Under this framework, there is no obligation to share prompts used to guide the AI, just as there is no requirement to disclose a human reviewer’s decision-making rubric or protocol. The producing party retains discretion over its internal processes, provided the results meet the agreed-upon standards of completeness and accuracy.
This approach aligns with Sedona Principle 6, which states that the producing party is best situated to determine the appropriate technologies and methods for its production. This principle supports the idea that AI, like other tools, can be used effectively without external micromanagement. It reinforces the notion that defensibility is rooted in reasonableness and good faith, not in the specific mechanics of the review as dictated by opposing counsel.
Moreover, there is no requirement to disclose evaluation metrics such as precision and recall, although calculating them is strongly recommended. These metrics provide internal assurance that the review process is functioning as intended. Just as with human review, the absence of shared metrics does not imply a lack of rigor, only that the producing party is not compelled to expose its internal quality control measures unless a deficiency is alleged.
Similarly, there is no need to negotiate a TAR protocol when using AI, just as one would not negotiate the details of a linear review workflow. The producing party maintains control over its methods, and the focus remains on the outcome: whether the production is complete and responsive. This best-case scenario fosters efficiency and innovation while maintaining fairness and accountability.
We note, however, that use of AI for the purposes of searching documents in place and designating them for preservation is risky, whether disclosed to opposing counsel or not. This is primarily due to the underlying indexing technology agentic AI[1] may have access to, rather than the application of AI on its own. That said, agentic AI may prove to be very useful in the identification of custodians and data stores subject to legal hold, so long as no filtering beyond date ranges is applied. If, for any reason, there is a loss of ESI that should have been subject to a legal hold, then the producing party must demonstrate that it took reasonable steps to avoid the loss and sanctions.
Worst Case Scenario: The Use of AI Is Scrutinized in Every Possible Way
In the worst-case scenario, the opposing party challenges the adequacy of the production and demands full transparency into the AI review process. This includes requests to disclose the prompts used to guide the AI under the argument that these prompts influence the scope and nature of the review. While this level of scrutiny is uncommon, it may arise in high-stakes litigation or when there is a history of discovery disputes. However, while these may be requested in discovery, there is currently no precedent in place that compels a party to share the prompts used for review.
The producing party may also face demands to disclose every step of the workflow, including preprocessing, filtering, and post-review validation. This level of detail is rarely required in traditional workflows but could be argued as necessary to assess the reliability of a novel technology. The burden of such disclosure can be significant, both in terms of time and potential exposure of privileged or strategic information.
Another potential demand is access to “the AI reasoning or decision-making process.” While this may be technically infeasible given the opaque nature of large language models (LLMs), many tools log the chain of thought “reasoning” output by the underlying LLM in use. Ultimately, courts may balance the requesting party’s need for transparency with the producing party’s right to maintain confidentiality over its tools and methods.
Bias evaluation may also become a focal point. Opposing counsel could argue that the AI may have inherent biases that skew the review results, particularly in cases involving sensitive topics or diverse data sets. In such cases, the producing party may need to demonstrate that steps were taken to assess and mitigate bias, even if such evaluations are not typically required in human review.
Finally, the producing party may be asked to share evaluation metrics, and possibly even go beyond standard measures like recall and precision. Even qualitative assessments may be requested. While these demands can be burdensome, they underscore the importance of maintaining thorough documentation and being prepared to defend the process if challenged.
Throughout every step of discovery, it is critical to maintain audit trails. Audit trails in eDiscovery are detailed records that document every action taken throughout the discovery process, including data collection, processing, review, and production. These logs capture who accessed what data, when, and what changes or decisions were made, creating a transparent and traceable history of the workflow. In litigation, audit trails serve as a critical safeguard by demonstrating that discovery was conducted in a consistent, defensible, and good-faith manner. If questions arise about the integrity of the process, such as allegations of spoliation or improper filtering, a well-maintained audit trail can provide evidence that appropriate procedures were followed. This not only helps protect against sanctions but also reinforces the credibility of the producing party’s efforts in the eyes of the court.[2]
Evaluating a Review Workflow or Methodology
Evaluating the review process in eDiscovery requires the application of well-established metrics to ensure the process is both effective and defensible. Three of the most critical metrics are unbiased estimates of recall, precision, and elusion for the review population. These three metrics should be used to evaluate any review workflow, whether technology-assisted or otherwise.
Recall measures the proportion of relevant documents that were successfully identified. High recall indicates that the processes and tools used are capturing most of the documents that should be produced, which is essential for completeness. In legal contexts, failing to identify relevant documents can lead to accusations of spoliation or inadequate production, so maintaining strong recall is a cornerstone of defensibility.
Precision, on the other hand, measures the proportion of documents identified as relevant that are actually relevant. High precision means the party is not overproducing irrelevant documents, which helps reduce review costs and protects against unnecessary disclosure of sensitive or privileged information. In practice, there is often a trade-off between precision and recall, and the optimal balance depends on the case context and the volume of data. For example, in high-stakes litigation, higher recall may be prioritized to ensure nothing is missed, even if it means reviewing more false positives.
Elusion is a lesser known but sometimes important metric that evaluates the proportion of relevant documents missed by the review process. It allows for an estimate of what might have been overlooked and allows the producing party to provide an interval estimate for recall when all the predicted relevant documents have been reviewed and received gold standard labels. In datasets with very low prevalence, elusion testing can be particularly valuable because it allows the producing party to reduce the volume of data reviewed for the purposes of evaluation alone.
These metrics are generally estimated from an unbiased random sample of the entire review population or by using stratified sampling to assess certain sub-populations of the overall review population. Again, these metrics should be used to evaluate the adequacy of any review workflow or methodology, but in practice, they are often considered more critical for TAR workflows despite studies that indicate humans tend to produce less consistent annotations than LLMs.
In Practice: What Does Cooperation Look Like?
In real-world scenarios, cooperation between parties is essential to avoid disputes and delays. For the producing party, this means being transparent about the use of AI and providing reasonable assurances about the quality and completeness of the production. Sharing high-level information about the technology used and the evaluation metrics obtained can help build trust and avoid conflict.
At the same time, the requesting party must act reasonably. Just as it would be inappropriate to demand the opposing party’s review protocol or seed set in a traditional TAR workflow, it is equally unreasonable to demand exhaustive details about a AI review process without a specific basis for concern. Cooperation requires both sides to recognize the balance between transparency and strategic confidentiality.
Ultimately, defensibility in eDiscovery, whether using AI or not, rests on principles of reasonableness, proportionality, and good faith. By adhering to these principles and fostering open communication, parties can navigate the evolving landscape of AI review while minimizing the risk of disputes and ensuring fair outcomes.
Conclusion
As AI is more integrated into legal workflows, its role in eDiscovery will continue to evolve. The key to defensibility lies not in the specific technology used but, in the reasonableness, cooperative nature, and consistency of the process. Whether using human reviewers, traditional TAR, or AI, the producing party must be able to demonstrate that its approach was thoughtfully designed, properly executed, and aligned with legal standards. Courts are increasingly open to innovative methods, provided they are applied in good faith and produce reliable results.
The contrast between best and worst-case scenarios highlights the importance of preparation. In the best case, AI is treated as a natural extension of existing review practices, requiring no additional disclosures or negotiations beyond what is standard for human review. In the worst case, however, the producing party may be required to defend every aspect of its workflow, from prompt design to bias evaluation. Anticipating these challenges and maintaining thorough documentation can help mitigate risk and ensure that the review process remains defensible under scrutiny.
Ultimately, cooperation between parties remains a cornerstone of effective eDiscovery. Transparency from the producing party and reasonableness from the requesting party are essential to maintaining trust and avoiding disputes. As the legal community continues to adapt to the capabilities of AI, a shared commitment to fairness, efficiency, and defensibility will be critical in shaping the future of discovery.
[1] A type of AI powered by AI and more traditional software engineering.
[2] Note that we do not address the implications of proportionality here, which may be a topic for a future paper.
[View source.]
