haveibeenfiltered

What is this?

haveibeenfiltered is an npm package for checking passwords against breach datasets locally. It uses ribbon filters to compress 2 billion SHA-1 hashes from Have I Been Pwned into a 1.79 GB file with a 0.78% false positive rate and zero false negatives.

Install & Usage

Install

npm install haveibeenfiltered

Download the filter data

npx haveibeenfiltered download

Use in your app

const hbf = require('haveibeenfiltered')

const filter = await hbf.load()

filter.check('password123')   // true  — breached
filter.check('Tr0ub4dor&3')   // false — not found

filter.checkHash('5BAA61E4C9B93F3F0682250B6CF8331B7EE68FD8')  // true

filter.close()  // release memory when done

How Ribbon Filters Work

A ribbon filter is a probabilistic data structure for set membership testing, similar to a Bloom filter but roughly 20% more space-efficient. Given a set of keys, it stores compact fingerprints in a system of linear equations that can be solved via Gaussian elimination. Querying evaluates the system for a given key and compares the result to the expected fingerprint.

Query flow

The filter file is divided into 256 shards based on the first byte of the SHA-1 hash. Within each shard, the MurmurHash3 (x64, 128-bit) of the hex hash is computed using a seed from the file header. This produces three values:

• start — index into the solution array
• coefficient — a 64-bit bitmask selecting which solution rows to XOR (ribbon width = 64)
• fingerprint — a 7-bit value to compare against

The query XORs the solution entries at positions where the coefficient bits are set, starting from the start index. If the result matches the fingerprint, the key is considered present. A small number of keys that couldn't be inserted during construction ("bumped" keys) are stored in a per-shard overflow table and checked separately.

Benchmarks

Comparison with HIBP API

Downloads

Filter binaries are hosted on Cloudflare R2 at https://files.haveibeenfiltered.com/v0.1/.

Integrity is verified via SHA-256 after download. The CLI handles downloading automatically:

npx haveibeenfiltered download

False Positives & False Negatives

A ribbon filter is a probabilistic data structure. It has two possible error types:

• False positive (FP): A safe password is incorrectly reported as breached. This can happen because the filter stores compressed fingerprints, not exact hashes. The filter uses 7-bit fingerprints, giving a rate of 1/128 (~0.78%).
• False negative (FN): A breached password is missed and reported as safe. This cannot happen. If a password is in the dataset, the filter will always detect it.

In practice this means: if check() returns false, the password is definitely not in the dataset. If it returns true, there is a ~0.78% chance it's a false alarm. For security applications this is the right tradeoff — you never miss a breached password.

FAQ