English Carries About One Bit Per Letter and the Rest Is Insurance
Gepubliceerd op 13-11-2026 · 3 min leestijd · Ontwikkelaarstools
Daniel Okonkwo — Front-enddeveloper en techredacteur bij OneKitly
Webprestaties · Bestandsformaten
Getoetst aan 2 bronnen
Shannon entropy measures how much a symbol tells you, in bits, and it is largest when every outcome is equally likely. Four equally likely symbols carry exactly 2 bits each; skew them to 50, 25, 12.5 and 12.5 per cent and the average drops to 1.75 bits, a redundancy of 12.5%. Written English is far more skewed than that: with real letter frequencies each letter carries 4.18 bits against a maximum of 4.70, and once the surrounding words are taken into account Shannon's own experiments put the figure near 1.1. Roughly three quarters of every English text is therefore predictable — which is what makes both compression and reading in a noisy room possible.
26 equally likely letters would carry 4.70 bits each. Real letter frequencies give 4.18, and context cuts it to about 1.1 — a redundancy of roughly 77%.
Redundancy is a feature, not waste
A language with no redundancy would be unreadable the moment a letter was smudged, because every possible string would be a valid word and no error could be detected. Human languages sit around three quarters redundant, which is why a sentence survives a bad phone line, a foreign accent and a page of handwriting. The same principle is designed deliberately into machines: error-correcting codes add redundancy on purpose, and a QR code still scans with a third of its area destroyed for exactly the reason a smudged word is still readable.
What a compressor is actually removing
Entropy sets a hard floor: no lossless compressor can average fewer bits per symbol than the source's entropy, and every one of them is a machine for approaching that floor. Text compresses well because there is a great deal of redundancy to take out; already-compressed files, encrypted files and random noise compress not at all, since their entropy is already at the maximum. That is a useful field test — anything that refuses to shrink is either already compressed or is not the kind of data you thought it was.
| Distribution | Entropy | Redundancy |
|---|---|---|
| 25, 25, 25, 25 | 2.00 bits | 0% |
| 50, 25, 12.5, 12.5 | 1.75 bits | 12.5% |
| English letters | 4.18 of 4.70 bits | 11.2% |
Uitgewerkt met onze eigen calculator
Calculator shannon-entropie
Gegeven
- Kansen (tellen op tot 1)
- 0.25, 0.25
Resultaat
- Entropie (bits)
- 1
- Maximale entropie (bits)
- 1
- Redundantie
- 0%
Deze cijfers komen uit de calculator hieronder, ze zijn niet met de hand ingetypt — ze worden opnieuw berekend telkens als de tool verandert.
Reken het door met je eigen cijfers →Veelgestelde vragen
- Why measure information in bits?
- Because a bit is the answer to one yes-or-no question, and any uncertainty can be resolved by asking enough of them. Two bits identify one of four equally likely options, ten bits identify one of 1024. Entropy is simply the average number of such questions needed, which is why it comes out fractional: 1.75 bits means that a long run of symbols can be pinned down with 1.75 questions each on average, even though no individual question can be three quarters asked.
- Do other languages come out the same?
- Close, once the alphabet size is accounted for. Languages with more letters start from a higher maximum and end up in a similar place, because the redundancy is imposed by how humans speak and listen rather than by the writing system. Where the measurements diverge is with scripts that encode syllables or whole words, since a symbol then carries far more information and the per-symbol entropy rises accordingly — comparisons across writing systems have to be made per unit of meaning, not per character.
Artikelen die je misschien interesseren
Alle gidsen →Vergelijkbare tools
Bronnen
Een fout in dit artikel gezien?