Skip to content
OneKitly

English Carries About One Bit Per Letter and the Rest Is Insurance

Published 11/13/2026 · 3 min read · Developer tools

Daniel Okonkwo

Daniel Okonkwo — Front-end developer and tech writer at OneKitly

Web performance · File formats

Checked against 2 sources

View profile →
In short

Shannon entropy measures how much a symbol tells you, in bits, and it is largest when every outcome is equally likely. Four equally likely symbols carry exactly 2 bits each; skew them to 50, 25, 12.5 and 12.5 per cent and the average drops to 1.75 bits, a redundancy of 12.5%. Written English is far more skewed than that: with real letter frequencies each letter carries 4.18 bits against a maximum of 4.70, and once the surrounding words are taken into account Shannon's own experiments put the figure near 1.1. Roughly three quarters of every English text is therefore predictable — which is what makes both compression and reading in a noisy room possible.

Advertisement

26 equally likely letters would carry 4.70 bits each. Real letter frequencies give 4.18, and context cuts it to about 1.1 — a redundancy of roughly 77%.

Redundancy is a feature, not waste

A language with no redundancy would be unreadable the moment a letter was smudged, because every possible string would be a valid word and no error could be detected. Human languages sit around three quarters redundant, which is why a sentence survives a bad phone line, a foreign accent and a page of handwriting. The same principle is designed deliberately into machines: error-correcting codes add redundancy on purpose, and a QR code still scans with a third of its area destroyed for exactly the reason a smudged word is still readable.

What a compressor is actually removing

Entropy sets a hard floor: no lossless compressor can average fewer bits per symbol than the source's entropy, and every one of them is a machine for approaching that floor. Text compresses well because there is a great deal of redundancy to take out; already-compressed files, encrypted files and random noise compress not at all, since their entropy is already at the maximum. That is a useful field test — anything that refuses to shrink is either already compressed or is not the kind of data you thought it was.

Redundancy
Entropy of four symbols under two distributions
DistributionEntropyRedundancy
25, 25, 25, 252.00 bits0%
50, 25, 12.5, 12.51.75 bits12.5%
English letters4.18 of 4.70 bits11.2%

Worked with our own calculator

Shannon entropy calculator

Given

Probabilities (sum to 1)
0.25, 0.25

Result

Entropy (bits)
1
Maximum entropy (bits)
1
Redundancy
0%

These figures are produced by the calculator below, not typed in by hand — they are recomputed whenever the tool changes.

Run it on your own figures →
Advertisement

Frequently asked questions

Why measure information in bits?
Because a bit is the answer to one yes-or-no question, and any uncertainty can be resolved by asking enough of them. Two bits identify one of four equally likely options, ten bits identify one of 1024. Entropy is simply the average number of such questions needed, which is why it comes out fractional: 1.75 bits means that a long run of symbols can be pinned down with 1.75 questions each on average, even though no individual question can be three quarters asked.
Do other languages come out the same?
Close, once the alphabet size is accounted for. Languages with more letters start from a higher maximum and end up in a similar place, because the redundancy is imposed by how humans speak and listen rather than by the writing system. Where the measurements diverge is with scripts that encode syllables or whole words, since a symbol then carries far more information and the per-symbol entropy rises accordingly — comparisons across writing systems have to be made per unit of meaning, not per character.

Articles you may find interesting

All guides →
ExplainerPassword Entropy: What a Strength Meter Cannot KnowEntropy measures the process that produced a password, not the characters in it. H = L x log2(R) is only true when every character was chosen at random — which is exactly why a meter scoring a human-invented password on its character classes is measuring the wrong thing.ExplainerA Four-Digit PIN Has 10 000 Values and People Use About TwentyTen choices four times over is 10 000. Adding two digits reaches a million; using letters as well as digits reaches 1 679 616 with the same four slots.ExplainerA Poll of a Thousand Is Just as Accurate in a Country of Sixty Million500 out of 1000 gives 50% with a margin of 3.10 points. Cut the sample to 100 and the margin becomes 9.80 — three times wider, from a tenth of the interviews.ExplainerIf Three People Each Name Three Contacts Among Four, Someone Is WrongThe list 3, 3, 3, 1 cannot describe any real network. 3, 3, 2, 2 can, with five links. Two centuries-old rules catch survey errors before any analysis starts.ExplainerThree to One Means One Chance in Four, Not ThreeFractional odds count the ways to lose against the ways to win. 3/1 is 25%, 1/3 is 75%, and 5/2 is 28.57% — the fraction is never the probability.ExplainerA Secret Santa Draw Fails 63 % of the Time, Whatever the Group SizeOf the 120 ways five people can draw names, 44 give nobody their own — 36.7 %. With ten people it is 36.788 %, and with twenty it is 36.788 %. The proportion stops moving almost immediately.

Related tools

Sources

Spotted a mistake in this article?