Page 1
DEDUPLICATION · A GUIDE FOR UK LAWYERS Deduplication Explained Hashes, Scope Choices and Why "the Same Document" Is a Legal Question COMPUTER FORENSICS LAB
§ ABOUT THE AUTHOR PREPARED BY COMPUTER FORENSICS LAB E-DISCOVERY TEAM ESTABLISHED 2007 · LONDON ISO 17025-ALIGNED PROCEDURES EDRM PROCESSING PRACTICE
§ CONTENTS In this guide 01 Executive summary 02 The problem in plain English: the copies that eat the budget 03 How it works: hashes and what counts as identical 04 The scope decision: custodian-level vs matter-level 05 Email's special problems: near-copies that are not twins 06 Keeping what suppression hides: custodian tables and copy evidence 07 The forensic cousin: hashing as copy-tracing evidence 08 Source architecture: where else the evidence lives 09 Worked examples 10 Common mistakes and technical limitations 11 Questions to ask · Suggested wording 12 Checklist and red flags · When to involve a digital forensic expert 13 Frequently asked questions 14 Glossary · References · Disclaimer · How a specialist laboratory can assist
§ 01 · ORIENTATION Executive summary THE HEADLINE POINT: DEDUPLICATETOREVIEWONCE, BUTRECORDWHOELSEHELD EVERY COPY: SUPPRESSIONMUSTLOSECOST, NOT EVIDENCE
§ 02 · FIRST PRINCIPLES The problem in plain English: the copies that eat the budget
§ 03 · THEMECHANICS How it works: hashes and what counts as identical 83) handles resemblance; dedup handles identity only.
Page 2
§ 04 · THESCOPEDECISION The scope decision: custodian-level vs matter-level
§ 05 · EMAIL ' SQUIRKS Email's special problems: near-copies that are not twins
§ 06 · WHATSUPPRESSIONMUSTKEEP Keeping what suppression hides: custodian tables and copy evidence
§ 07 · THECOUSINDISCIPLINE The forensic cousin: hashing as copy-tracing evidence
§ 08 · THEWIDERMAP Source architecture: where else the evidence lives QUESTION SUPPRESSION PRESERVED INSTANCE DUP / COUNTERPART FORENSIC TABLE SOURCES METADATA THREAD PRODUCTION LAYER NEAR- LAYER
§ 09 · IN THE WILD Worked examples EXAMPLE1 · THEMEMOTHATELEVENPEOPLEHELD EXAMPLE2 · THESCOPEFIGHTSETTLEDBYARITHMETIC EXAMPLE3 · THE " DUPLICATE " THATWASTHEFORGERY ' STWIN
Page 3
§ 10 · WHEREITGOESWRONG Common mistakes and technical limitations Common mistakes Technical limitations
§ 11 · INTERROGATORIES & DRAFTING AIDS Questions to ask · Suggested wording Ask your client Ask your opponent Ask your e Discovery / forensic provider SUGGESTED WORDING · DEDUPLIC AT ION PA R AG RAPHFORTHEDISCLOSUREPROTOCOL
§ 12 · QUICK CONTROL Checklist and red flags · When to involve a digital forensic expert The deduplication checklist Red flags When to involve a digital forensic expert
§ 13 · COMMON QUESTIONS Frequently asked questions Does deduplication delete documents? Custodian-level or matter-level: which should we agree? Can we still prove who received a document if their copy was suppressed? The same email seems to appear several times in the review set anyway. Why? Two productions contain "the same" contract. Do we need to check? Can deduplication data itself be evidence?
§ 14 · REFERENCE Glossary Sources and authoritative references DISCLAIMER
§ HOW A SPECIALIST LABORATORY CAN ASSIST Working with Computer Forensics Lab Speak to a forensic examiner, not a salesperson. INSTRUCTTHELAB NEWENQUIRIESEMAILE - DISCOVERY
Cite as: Joseph Naghdi, Deduplication Explained, Computer Forensics Lab, https://e-discovery.uk/library/deduplication-explained/pdf.