Computer Forensics Lab, e-discovery.uk
e‑discovery.uk
Computer Forensics Lab eDiscovery
§ Guide

Deduplication Explained

This guide explains deduplication, including how hashes identify identical documents, the impact of scope choices, and why "the same document" is a legal question.

Ref · E-D · 2026 · §LIBClass · ConfidentialJuris · England & WalesStatus · Active

Guide · 17 pages · 20 min read · Published 2026-08-30

Aimed at UK legal practitioners navigating modern e-discovery cost pressures, this analysis addresses how redundant document copies inflate review budgets and complicate disclosure obligations. It explores the technical mechanics of hashing to establish exact document identity, contrasting strict identity with mere resemblance, while detailing the strategic scope decision between custodian-level and matter-level deduplication. The material examines email quirks, near-copies, and thread layers, explaining how to suppress duplicate items to cut expenditure without losing metadata, counterpart sources, or recipient evidence. Legal teams gain practical guidance on maintaining custodian tables, using hashing as forensic copy-tracing evidence, and mapping wider source architecture. Beyond core technical principles, it provides actionable wording for a disclosure protocol, targeted interrogatories for clients, opponents, and e-discovery providers, worked examples, and a checklist of red flags detailing common technical limitations. Aligned with EDRM processing practice and London ISO 17025 procedures, it assists lawyers in determining precisely when to instruct a digital forensic specialist laboratory.

Share
LinkedIn X / Twitter WhatsApp Email

Read this guide on your phone, browse guides by topic or go back to the full PDF library.

§ Credit and source

Published by Computer Forensics Lab on 2026-08-30. Original material of the practice, free to read, cite and download. The authority behind this subject is NIST Computer Forensics Tool Testing programme, which you should read alongside this guide. See every guide's author and source.

§ Read Deduplication Explained

Download the PDF

Prefer a PDF that matches this page exactly? Download the current text as a PDF, generated from the current wording of the guide, including any later corrections.

Page 1

DEDUPLICATION · A GUIDE FOR UK LAWYERS Deduplication Explained Hashes, Scope Choices and Why "the Same Document" Is a Legal Question COMPUTER FORENSICS LAB

§ ABOUT THE AUTHOR PREPARED BY COMPUTER FORENSICS LAB E-DISCOVERY TEAM ESTABLISHED 2007 · LONDON ISO 17025-ALIGNED PROCEDURES EDRM PROCESSING PRACTICE

§ CONTENTS In this guide 01 Executive summary 02 The problem in plain English: the copies that eat the budget 03 How it works: hashes and what counts as identical 04 The scope decision: custodian-level vs matter-level 05 Email's special problems: near-copies that are not twins 06 Keeping what suppression hides: custodian tables and copy evidence 07 The forensic cousin: hashing as copy-tracing evidence 08 Source architecture: where else the evidence lives 09 Worked examples 10 Common mistakes and technical limitations 11 Questions to ask · Suggested wording 12 Checklist and red flags · When to involve a digital forensic expert 13 Frequently asked questions 14 Glossary · References · Disclaimer · How a specialist laboratory can assist

§ 01 · ORIENTATION Executive summary THE HEADLINE POINT: DEDUPLICATETOREVIEWONCE, BUTRECORDWHOELSEHELD EVERY COPY: SUPPRESSIONMUSTLOSECOST, NOT EVIDENCE

§ 02 · FIRST PRINCIPLES The problem in plain English: the copies that eat the budget

§ 03 · THEMECHANICS How it works: hashes and what counts as identical 83) handles resemblance; dedup handles identity only.

Page 2

§ 04 · THESCOPEDECISION The scope decision: custodian-level vs matter-level

§ 05 · EMAIL ' SQUIRKS Email's special problems: near-copies that are not twins

§ 06 · WHATSUPPRESSIONMUSTKEEP Keeping what suppression hides: custodian tables and copy evidence

§ 07 · THECOUSINDISCIPLINE The forensic cousin: hashing as copy-tracing evidence

§ 08 · THEWIDERMAP Source architecture: where else the evidence lives QUESTION SUPPRESSION PRESERVED INSTANCE DUP / COUNTERPART FORENSIC TABLE SOURCES METADATA THREAD PRODUCTION LAYER NEAR- LAYER

§ 09 · IN THE WILD Worked examples EXAMPLE1 · THEMEMOTHATELEVENPEOPLEHELD EXAMPLE2 · THESCOPEFIGHTSETTLEDBYARITHMETIC EXAMPLE3 · THE " DUPLICATE " THATWASTHEFORGERY ' STWIN

Page 3

§ 10 · WHEREITGOESWRONG Common mistakes and technical limitations Common mistakes Technical limitations

§ 11 · INTERROGATORIES & DRAFTING AIDS Questions to ask · Suggested wording Ask your client Ask your opponent Ask your e Discovery / forensic provider SUGGESTED WORDING · DEDUPLIC AT ION PA R AG RAPHFORTHEDISCLOSUREPROTOCOL

§ 12 · QUICK CONTROL Checklist and red flags · When to involve a digital forensic expert The deduplication checklist Red flags When to involve a digital forensic expert

§ 13 · COMMON QUESTIONS Frequently asked questions Does deduplication delete documents? Custodian-level or matter-level: which should we agree? Can we still prove who received a document if their copy was suppressed? The same email seems to appear several times in the review set anyway. Why? Two productions contain "the same" contract. Do we need to check? Can deduplication data itself be evidence?

§ 14 · REFERENCE Glossary Sources and authoritative references DISCLAIMER

§ HOW A SPECIALIST LABORATORY CAN ASSIST Working with Computer Forensics Lab Speak to a forensic examiner, not a salesperson. INSTRUCTTHELAB NEWENQUIRIESEMAILE - DISCOVERY

§ Related documents
Instruct the practice

Bring us in early. Defensibility is built, not retrofitted.

Whether you are responding to a regulator, preparing for disclosure, or scoping an internal investigation, start the chain of custody with a short, confidential conversation.

Open a confidential matter→+44 (0)20 7164 6971Direct lineinfo@e-discovery.ukEncrypted
WhatsApp