Measured, not marketed

Linux — internal HDD (XFS) → SSD

Setup: 12,347 files / 110.9 MB (avg 9 KB — a seek-bound, many-small-files tree) · rotational HDD source (XFS) → SSD destination · page cache dropped before every run (sync; echo 3 > /proc/sys/vm/drop_caches) · timing includes sync on the destination (durable write) · 3 interleaved rounds per tool · best-of-3 reported.

Commands

# before EVERY run:
sync; echo 3 > /proc/sys/vm/drop_caches

blitcp.py SRC DST                          # default (dedup + verify)
blitcp.py SRC DST --no-dedup --no-cache    # pure copy path
cp -ar SRC/. DST/
rsync -a SRC/ DST/
# timing wraps each command plus `sync -f DST`

Windows — USB 2.0 external HDD → SSD

Setup: 9,578 files / 890.8 MB (avg 95 KB) · USB 2.0 external HDD source → system SSD · standby list + working sets purged with Sysinternals RAMMap (-Ew, -Es) before every run · Windows Defender active for all tools · single manual run per tool · blitcp was the only tool that also verified the copy afterwards. A USB 2.0 source is latency-bound (~10 ms per file op over the BOT protocol), so per-file efficiency decides the race — raw bandwidth cannot win it.

The two workloads are different datasets — compare tools within the same workload row group, not across groups.

Commands

# before EVERY run (Administrator):
RAMMap64.exe -Ew ; RAMMap64.exe -Es

blitcp-windows.exe E:\src C:\dst --no-dedup --no-cache --include-node-modules
robocopy E:\src C:\dst /E /R:0 /W:0 /MT:1 /NFL /NDL /NJH /NP
robocopy E:\src C:\dst /E /R:0 /W:0 /MT:8 /NFL /NDL /NJH /NP
FastCopy.exe /cmd=force_copy /auto_close /no_confirm_stop /verify=FALSE /error_stop=FALSE /balloon=FALSE E:\src /to=C:\dst

Windows — vs TeraCopy: combined small + large (real-world mix)

Hardware: 13th Gen Intel Core, 16 GB RAM · HDD over USB 2.0 source → internal SSD (NTFS) · Windows 11 · blitcp v3.12.3 · TeraCopy 4.0.3.2. Workload: 11,878 files / 11.4 GB, 2 source folders → 1 destination in a single copy job. Deliberately bimodal: one folder contributes 9,578 small files (890 MB, avg ~95 KB); the other 2,300 large files (10.5 GB, avg ~4.7 MB) — 92% of the total bytes. Both regimes are exercised in the same run.

Methodology

• Destination folder deleted before each run.
• Full system reboot, then 5–10 min idle until background disk I/O settles.
• Cold cache verified (RAMMap: standby list + system working set empty).
• Single timed run per cell under identical conditions; wall-clock time. No other significant I/O or workloads running.
• Both tools copied the identical source set in a single invocation/transfer job. File counts shown are files only; folder counts noted where tools display them (e.g. TeraCopy reports files + folders combined).
• Source on USB 2.0 deliberately: many-small-file transfers from external spinning disks are the worst case for every copy tool and the workload blitcp targets. On fast NVMe-to-NVMe transfers, differences between tools shrink. Sparse-file savings are a separate feature and are not part of these numbers.
• Reproduce it: run the same protocol on your own hardware and open an issue with your results. Benchmark scripts and raw run reports are being added to the GitHub repository.

Break-even — at what storage speed does hashing/dedup stop paying for itself?

Setup: asked by a reader (thanks!), measured August 2026. Source tree in tmpfs so reads are free — destination write speed is the only variable: raw SATA SSD, then the same SSD capped at 100 MB/s and 30 MB/s with cgroup v2 io.max (systemd-run -p IOWriteBandwidthMax), plus tmpfs→tmpfs as the “infinitely fast” limit. Dataset: 2 GB in 4 × 512 MB files, all unique and same-size — the worst case for dedup: full hashing cost, zero payoff. Page cache dropped before every run, sync inside every timing, best of two rounds.

The overhead is a roughly flat cost, not a percentage: ~0.9 s interpreter startup + 0.2 s scan + ~1 s xxHash-128 per 2 GB. Flat costs sink into slow I/O and dominate fast I/O — so the break-even for duplicate-free data lands around 100–200 MB/s destination write speed. Faster than that with no duplicates: use cp — our own comparison page says exactly that. On the other side of the ledger, a single duplicated 512 MB file repays the entire hashing bill several times over, so on real datasets the line moves right.

Two honest limits of this rig. A bandwidth cap models slow-but-seekless storage (USB 2, network shares) — it cannot simulate a rotational disk’s seek penalty, which is where blitcp’s small-file wins come from (see the HDD benchmark above — bandwidth-capped small-file runs favour cp, seek-bound ones favour blitcp). And the measurement flushed out real waste on our side: the extent-map probe runs (and fails) on filesystems without FIEMAP, burning ~0.7 s — that skip is on the fix list, which is exactly what this page is for.

Commands

# dataset: 4 x 512 MB unique, same size (worst case for dedup)
for i in 0 1 2 3; do head -c 536870912 /dev/urandom > SRC/big$i.bin; done

# before EVERY run:
sync; echo 3 > /proc/sys/vm/drop_caches

# throttled tiers wrap each command with:
systemd-run --scope -p "IOWriteBandwidthMax=/dev/sda 30M" ...

cp -a SRC DST                              # + sync -f DST, inside the timing
blitcp.py SRC DST --no-dedup --no-verify --no-cache
blitcp.py SRC DST --no-verify --no-cache

Honest notes & caveats

• Dataset shape matters. These are many-small-files workloads — where copy tools actually differ. With a few large files, every tool saturates the disk or the USB link and the results converge.
• Linux: best-of-3 interleaved cold rounds; cp / rsync are the untouched system binaries (coreutils 9.10, rsync 3.4.3).
• Windows: single run per tool; every time is the tool’s own reported total. Windows Defender real-time protection was active for all tools equally.
• blitcp default mode reads each source file twice on a first copy (hash pass + copy pass). On high-latency sources (USB) this costs real time — on repeat/incremental runs the hash cache eliminates it entirely, which is the mode’s purpose.
• Copies were content-verified after the runs (all files identical to source).

Download blitcp