Checksums Are Not Hashes: CRC-32, Adler-32 and What They Are For

Three jobs that all produce a short number

The confusion starts with the output. CRC-32, Adler-32, FNV-1a, MurmurHash3, MD5 and SHA-256 all take arbitrary bytes and return a fixed-size number, so they look interchangeable in an API. They are not. They were designed against three completely different threat models, and picking the wrong family produces failures that are silent until they are catastrophic.

A checksum answers: did this data change by accident on its way here? Its adversary is a cosmic ray, a marginal cable, a truncated write, a disk sector going soft. That adversary is random and does not adapt. CRC-32 and Adler-32 are checksums. A cryptographic hash answers a harder question: can anyone, given as much time and hardware as they can buy, find a second input with this same output? Its adversary is a person with a budget. MD5, SHA-1 and SHA-256 are attempts at this — with varying success, which the companion piece on hash choice goes into.

The third job is the one people forget. A hash-table hash answers: how do I turn this key into a bucket index quickly and evenly? Its adversary is nominally nobody — until the keys come from HTTP request parameters, at which point the adversary is whoever is sending them. FNV-1a and MurmurHash3 live here. They are excellent at their job and offer no protection whatsoever at the other two.

CRC-32 is linear, and here is the collision

CRC-32 is polynomial division over GF(2), and division is linear. Concretely, for any three messages of the same length, crc(a XOR b XOR c) equals crc(a) XOR crc(b) XOR crc(c). Testing that on five random 32-byte triples gave an exact match every time — for instance one triple produced 8975e151 on both sides. No cryptographic hash has any such identity, and that single algebraic fact is the entire difference between the two families.

Linearity means you can solve for a collision instead of searching for one. Take a message an attacker wants to alter, give it a few bytes of slack anywhere — padding, a comment field, a reserved header, trailing whitespace — and the required slack value is the solution of a 32-unknown linear system over GF(2). Gaussian elimination handles that in microseconds.

Done for real: the original message was "config: mode=safe, retries=3, pad=......" with CRC-32 78aa94ad. The forgery was to read "config: mode=open, retries=9, pad=", followed by six pad bytes to be determined. Solving those bytes produced "config: mode=open, retries=9, pad=!9[noI" — same 40-byte length, 11 bytes different, and the identical CRC-32 78aa94ad. Twenty-five candidate pads were tried before one came out fully printable; the whole program ran in 0.11 seconds. The two messages' SHA-256 digests begin d9cddeec and 6b7bdc0c, which is what a function without a linearity identity looks like.

Nothing here required cryptanalysis, a GPU or a wordlist. It required knowing that CRC-32 is a linear map and owning six bytes of the message. That is why a CRC accompanying a file over an untrusted channel proves nothing about tampering: an attacker who can change the file can change the CRC to match, and even if the CRC is delivered separately and cannot be touched, they can still craft a different file that produces it.

What CRC-32 is genuinely excellent at

None of the above makes CRC-32 a bad function. It makes it a function doing a different job, and at that job it is close to optimal. Its guarantees are not statistical, they are proved: it detects every single-bit error, every two-bit error within an enormous message length, every error affecting an odd number of bits, and every burst error up to 32 bits — the length of the checksum itself.

Measured rather than asserted: taking a 200-byte message, all 1,600 possible single-bit flips changed the CRC, and all 1,279,200 possible two-bit flips changed it too — not one escaped. On a 1500-byte Ethernet-sized frame, 200,000 random burst errors at each of six widths (8, 16, 32, 33, 40 and 64 bits) were all detected. Bursts longer than 32 bits are not guaranteed, only overwhelmingly likely: the escape probability is about 2 to the minus 32, or one in 4.29 billion, which is why 200,000 trials found nothing.

This is why CRC-32 is in Ethernet frames, in gzip trailers, in PNG chunks, in ZIP entries and in SATA. Those are all channels whose failure mode is a physical glitch producing a contiguous run of corrupted bits — precisely the error class CRC polynomials are constructed to catch with certainty. A cryptographic hash would also catch them, but at several times the cost and with no proved guarantee, only a probabilistic one.

Adler-32: cheaper to compute, weaker at detecting

Adler-32, defined in RFC 1950 for the zlib format, is two running sums modulo 65,521: a plain sum of the bytes, and a sum of those partial sums. It was designed to be much cheaper than a CRC while detecting most of the same errors — no lookup table, just addition. In practice the modulo operation costs enough that the promised advantage often evaporates: measured in the same JavaScript runtime on the same 64 MB buffer, Adler-32 ran at 174 MB/s against CRC-32's 262 MB/s. Adler-32 was slower.

It also has a blind spot you can prove rather than estimate. The modulus 65,521 is the largest prime below 65,536. If you increase one byte by d and decrease another byte by d, the first sum is unchanged, and the second sum changes by d times the distance between them — which vanishes modulo 65,521 exactly when that distance is 65,521. So any message longer than about 64 kilobytes has compensating pairs of changes that Adler-32 cannot see at all.

Demonstrated on a 70,000-byte buffer: raising byte 100 by 7 and lowering byte 65,621 by 7 left Adler-32 at 3fee717c, byte-for-byte identical to the clean value, while CRC-32 moved from abc586b8 to 43c209f4. Adler-32 is also poor on short inputs — 200,000 random four-byte inputs produced only 152,364 distinct Adler-32 values, where an ideal 32-bit function would have produced about 199,995. RFC 1950 itself notes the weakness for short messages, which is why zlib streams carry it over whole streams rather than tiny records.

FNV-1a and MurmurHash3: the third category

FNV-1a and MurmurHash3 are neither checksums nor cryptographic hashes. They are non-cryptographic hashes designed for hash tables, bloom filters and sharding, where the requirement is uniform distribution at the lowest possible cost per byte. They deliver on that. In the same runtime and on the same buffer, MurmurHash3 ran at 730 MB/s and FNV-1a at 520 MB/s, against 262 MB/s for CRC-32.

Collisions in them are trivial to find, and the exercise takes under two seconds. Enumerating seven-character strings, FNV-1a 32 collided at "7yzlaaa" and "e6apaaa", both hashing to 15111984, after 700,997 candidates and 680 milliseconds. MurmurHash3 with seed 0 collided at "rynbaaa" and "ciaabaa", both hashing to e5407f96, after 1,679,907 candidates and 1.6 seconds. That is expected — 32 bits means a birthday collision around 77,163 items — and it is not a defect. It becomes a defect when someone chooses the keys.

Hash flooding is the resulting attack, and it is easy to reproduce. Collecting 20,000 keys whose FNV-1a value lands in bucket 0 of a 4,096-bucket table cost a fraction of a second of trial division. Inserting them collapsed the table into a single 20,000-entry chain, where ordinary keys gave a longest chain of 12. Twenty thousand lookups then took 1,397.9 milliseconds instead of 11.5 — a 121-fold slowdown, from constant time to linear. Every request that touches such a table becomes an amplifier, which is exactly the denial-of-service class that pushed language runtimes to randomly seeded SipHash for their built-in dictionaries.

Choosing, in one question

Ask who benefits if two different inputs produce the same value. If the answer is nobody — you are catching truncated copies, flaky cables, corrupted archives, a bit rot on a backup drive — a checksum is correct and CRC-32 is the sensible default. It is small, it is everywhere, it has proved guarantees against exactly the error shapes hardware produces, and native implementations are extremely fast: node's zlib CRC-32 hit 2,248 MB/s on the same buffer, three times SHA-256's 763 MB/s.

If the answer is somebody — a download over a network you do not control, a signature, a licence file, an update payload, deduplication of user-supplied blobs, a cache keyed by anything a user can influence — you need a cryptographic hash, and today that means SHA-256. The cost is real but small: 763 MB/s is still faster than most disks and most networks, and it is the only family in this comparison where a second input with the same output is not something anyone can simply solve for.

And if the value never leaves your process — a bucket index, a bloom filter, a shard selector — use a hash-table hash, but ask a follow-up question: can an attacker choose the keys? If they can, you want a keyed and randomly seeded function such as SipHash, not a fixed-seed FNV-1a. Most modern language runtimes already do this for their built-in maps; the danger is a hand-rolled table in application code that does not.

Frequently asked questions

Is CRC-32 a hash function?

In the loosest sense yes — it maps arbitrary input to a fixed 32-bit output — but calling it one invites the mistake this whole article exists to prevent. CRC-32 is a linear function, computed as the remainder of a polynomial division over GF(2). That linearity gives it the identity crc(a XOR b XOR c) = crc(a) XOR crc(b) XOR crc(c), verified here on random triples, and from that identity a collision follows by solving a 32-unknown linear system rather than by searching. A cryptographic hash is specifically engineered so that no such algebraic shortcut exists; that is what the word cryptographic is doing. So CRC-32 is a checksum, and the useful mental model is a very good error-detecting code rather than a weak hash. If a library, an API or a code review calls it a hash, check what property is actually being relied upon: uniqueness against an adversary is the one it cannot supply, and it is the one people assume.

Can I use a CRC-32 to verify a download?

It depends entirely on what you are verifying against. If you are checking that the bytes that arrived match the bytes that left — that the connection did not drop mid-transfer, that the archive is not truncated, that the disk wrote what it was given — a CRC-32 is exactly the right tool and will catch any realistic transport fault. That is why every ZIP entry and every gzip stream carries one. If instead you are asking whether the file is the file the publisher intended, a CRC-32 answers nothing. An attacker who can substitute the file can substitute the CRC too, and even where the CRC is published separately and cannot be altered, they can construct a different file matching it — this article did exactly that in 0.11 seconds. For publisher verification you need a cryptographic digest, published over a channel the attacker does not control, and ideally a signature over that digest rather than the digest alone.

Why does gzip use CRC-32 while zlib uses Adler-32?

Both formats wrap the same DEFLATE compressed data and differ mainly in their container. RFC 1952 specifies a CRC-32 in the gzip trailer; RFC 1950 specifies an Adler-32 in the zlib trailer. The reasoning at the time was speed: Adler-32 needs only additions and a modulo, no 256-entry lookup table, so on the processors of the early 1990s it was meaningfully cheaper per byte, and zlib was aimed at contexts where the checksum cost mattered relative to the compression. That advantage has largely evaporated. Modern CRC-32 implementations use slicing-by-8 tables or dedicated CPU instructions, and in the measurements here node's native CRC-32 reached 2,248 MB/s while a straightforward Adler-32 in the same JavaScript runtime managed 174 MB/s against CRC-32's 262 MB/s. The formats remain as specified because changing a checksum algorithm breaks every existing reader, and both are strong enough for their job of catching accidental corruption in a compressed stream.

How likely is an accidental CRC-32 collision?

For a single corrupted message the answer is excellent: bursts up to 32 bits are never missed, and beyond that the escape probability is about one in 4,294,967,296. For a collection of files it is much worse than intuition suggests, because the birthday effect applies. Two random 32-bit values collide with probability one in 4.29 billion, but a set of n values contains n(n−1)/2 pairs, so the 50% mark arrives at 77,163 items. Ten thousand files already carry a 1.16% chance of some pair sharing a CRC-32, and a hundred thousand carry 68.8%. That matters if you are using CRC-32 as a deduplication key or a content-addressed identifier over a large corpus, where a collision silently discards one of two different files. For a per-file integrity check against transport damage the birthday effect is irrelevant, because you are comparing one value against one expected value, not searching a population for matches.

Are FNV-1a and MurmurHash3 safe to use on user-supplied keys?

Not without a random seed, no. Both are unkeyed by default, so their output is a public function anyone can compute offline. That lets an attacker precompute keys landing in the same bucket and send them all at once, which is hash flooding. Reproduced here on a 4,096-bucket table: 20,000 crafted keys all hashed into bucket 0, turning a longest chain of 12 into a single chain of 20,000 and making 20,000 lookups take 1,397.9 milliseconds instead of 11.5 — 121 times slower, and the collection of those keys took under a second of attacker effort. The fix is not a stronger unkeyed hash; it is a keyed one with a per-process random seed, which makes offline precomputation impossible. SipHash is the standard choice and is what most language runtimes now use internally for dictionaries. If your keys come from configuration, from your own database, or from anywhere an attacker cannot influence, seedless FNV-1a and MurmurHash3 remain perfectly good and very fast.

What should I use for cache keys and deduplication?

Decide by who supplies the content and by how much a wrong answer costs. For an in-memory cache whose keys you generate — a query shape, a rendered template name, an internal identifier — a fast non-cryptographic hash is right, and MurmurHash3 at 730 MB/s is a good pick. For deduplication across a corpus you control, where a collision means silently keeping one of two different objects, 32 bits is too narrow: the 50% birthday point arrives at 77,163 items. Move to a 64-bit or 128-bit non-cryptographic hash, or to a truncated SHA-256. For anything where a user supplies the content — uploaded files, user-generated blobs, a content-addressed store, a shared cache serving multiple tenants — use SHA-256 in full. There a collision is not an accident but a capability, letting someone place a chosen object under an identifier that already exists, and only a cryptographic hash makes that infeasible. The cost is modest: 763 MB/s measured here is faster than the storage layer you are writing to.