Guidance on Generative AI for Judges and Confidentiality Risks

Guidance for Judges on Generative AI

eDiscovery Today reported on 31 August that a new guide, titled 'The Judicious Judge’s Guide to Generative AI and LLMs: Artificial Intelligence Best Practices', is now available. This guide aims to provide direction for judges when dealing with generative artificial intelligence (GenAI) and large language models (LLMs).

The availability of such guidance indicates a growing recognition within the legal sector of the need to address the practical implications of AI technologies. While this specific guide is aimed at judges, its existence highlights the broader requirement for clear protocols and understanding across all parties involved in legal processes where AI may be employed or encountered.

Confidentiality Risks with Public LLMs

Also on 31 August, eDiscovery Today discussed the risks to confidentiality associated with using public LLMs. The article, referencing Cimplifi, noted that while using public LLMs on opponents’ documents might be tempting, it increases risks to confidentiality.

This observation underscores a critical concern for practitioners. The convenience and perceived efficiency of public LLMs must be weighed against the potential for inadvertent disclosure or compromise of sensitive information. Unlike proprietary or securely hosted models, public LLMs often involve data being processed or stored on third-party servers, which may lack the necessary security protocols or contractual safeguards required for legal data.

AI and Office Return Trends

Separately, eDiscovery Today reported on 27 August on a trend related to AI and working practices. According to an article, AI is leading to consultants sending junior staff back to the office more frequently.

While this development does not directly relate to the technical aspects of disclosure or forensic practice, it touches upon the evolving landscape of legal work and how AI's integration might influence team structures and physical presence. The article suggests an unexpected 'by-product' of AI, indicating that the impact of these technologies extends beyond purely technical applications to affect operational models.

What this means in practice

The emergence of guidance for judges on GenAI and LLMs signals that UK courts will increasingly expect practitioners to understand and articulate their use of these technologies. For UK disclosure and review workflows, this means:

• Transparency is paramount: If GenAI or LLMs are used in any stage of the disclosure process, practitioners should be prepared to explain how they were used, what data was input, and what safeguards were in place. This includes explaining the specific model, its version, and any fine-tuning applied.
• Data security and confidentiality: The warning regarding public LLMs is critical. Practitioners must ensure that any LLM used for processing client or opponent documents adheres to strict confidentiality and data protection standards. This typically means avoiding public, general-purpose LLMs for sensitive data. Instead, consider secure, private, or on-premise solutions, or LLMs with explicit contractual guarantees regarding data handling and deletion.
• Risk assessment: Before deploying any AI tool, a thorough risk assessment should be conducted. This assessment should consider the nature of the data, the potential for hallucination or bias, and the security implications. For disclosure, this includes assessing the risk of producing privileged or irrelevant material, or failing to identify responsive documents.
• Human oversight: Even with advanced AI tools, human oversight remains essential. Review workflows should incorporate checks and balances to validate AI outputs, particularly for critical tasks such as privilege review or responsiveness tagging. The 'Judicious Judge’s Guide' likely reinforces the expectation that AI is a tool to assist, not replace, human judgment.
• Training and competence: Practitioners and their teams need to be adequately trained on the capabilities and limitations of AI tools. This includes understanding how to prompt effectively, interpret results, and identify potential errors or biases.

For forensic practice, the implications are similar, with an added emphasis on data integrity and chain of custody:

• Forensic soundness: Any AI tool used in forensic analysis must be demonstrably sound and not alter the original evidence. The process must be repeatable and verifiable.
• Validation of AI outputs: If AI is used for tasks like anomaly detection or pattern recognition in forensic data, the outputs must be rigorously validated by human experts. The methodology for validation should be documented.
• Ethical considerations: The use of AI in investigations raises ethical questions, particularly concerning bias and fairness. Forensic practitioners must be aware of these and ensure their use of AI does not compromise the integrity or impartiality of an investigation.