Organising a personal photo archive: capture date from EXIF, exact duplicates by checksum, and an inventory that is checked yearly

Goal

Turn scattered image files into one archive in which each photo's date has a stated origin, exact duplicates are removed with a record of what was removed, near-duplicates are flagged rather than merged, and the whole is inventoried and periodically checked.

Prerequisites

A working copy of all sources on one disk (originals untouched until the end); an EXIF reader; a checksum tool (sha256sum or an equivalent); the file-naming convention from the naming article. The ExifTool tag list documents DateTimeOriginal as the date and time when the original image was taken, CreateDate as the EXIF DateTimeDigitized, and OffsetTimeOriginal as the time zone for DateTimeOriginal; the last was added to EXIF later than the other two and is absent from older files.

Steps

1. Inventory sources: for each device, card, cloud export and chat export, record name, date range and file count before copying.
2. Copy everything into incoming/<source>/, preserving file dates, and write a SHA-256 list of the whole tree to a file kept with the archive.
3. Exact duplicates: for each hash with more than one path, keep one (the copies are byte-identical, so prefer the path that best records the source) and move the others to duplicates/, with a removal-log line: hash, kept path, removed paths.
4. Dates: record per file where its date came from: DateTimeOriginal with OffsetTimeOriginal; DateTimeOriginal without offset (local time of an unknown zone); CreateDate; or file-system time only, which for a file received through a messaging app is commonly the download time, not the capture time. Store the origin as a column, not in the file name.
5. Rename and file by year and month per the naming convention, keeping the original name in the inventory.
6. Near-duplicates: files that differ in bytes but show the same picture (re-encoded, resized, screenshots) are found by grouping on date and camera and comparing by eye; tag the set with one group id and leave every member in place.
7. Write the inventory: one line per folder with date range, count and sources, plus a description of the structure as the Library of Congress suggests.
8. Keep at least two copies in different places; once a year, open a sample from each copy and re-run the checksum check, logging date and outcome.

Expected result

An archive whose dates, removals and groupings each trace to a log line, with a dated integrity check.

Limits and test basis

Proposed method; no counts or failure rates are claimed. A checksum finds byte-identical files only; visual comparison is manual and subjective, so it flags, not deletes. A camera with a wrong clock produces wrong dates that the origin column records but cannot fix.