What Is UTF-8 and Unicode? Code Points, Byte Encoding, and Why UTF-8 Won

Code points versus bytes

It helps to separate two ideas. A code point is an abstract number that identifies a character in the Unicode catalog, independent of how it is stored. An encoding is the concrete recipe that turns that number into bytes on disk or on the wire. Unicode currently defines code points from U+0000 up to U+10FFFF, more than a million slots, of which around 150,000 are assigned.

The U+ notation is just hexadecimal, base 16. U+0041 is decimal 65, the letter A, and U+20AC is decimal 8,364, a currency symbol. Converting between the hex form and decimal or binary is exactly the kind of base change a number-base converter handles, and it is how you check by hand which byte pattern a code point should produce.

One to four bytes, decided by size

UTF-8 is a variable-length encoding. Code points from U+0000 to U+007F, the original ASCII range, are stored in a single byte whose top bit is zero, so plain English text is byte-for-byte identical to old ASCII files. Code points from U+0080 to U+07FF take two bytes, U+0800 to U+FFFF take three, and everything up to U+10FFFF, including most emoji, takes four.

The leading byte announces the length through its high bits: a two-byte sequence starts with 110, three-byte with 1110, four-byte with 11110, and every continuation byte starts with 10. That self-describing pattern lets a decoder resynchronize after a corrupt byte and makes it impossible to mistake a continuation byte for the start of a character.

Why UTF-8 beat the alternatives

The main rivals were UTF-16 and UTF-32. UTF-32 stores every code point in a fixed four bytes, simple to index but wasteful, quadrupling the size of ordinary English text. UTF-16 uses two or four bytes and once dominated Windows and Java, but it needs a byte-order mark to say which end comes first and still uses surrogate pairs for large code points, reintroducing the variable length it tried to avoid.

UTF-8 avoids all of that. It has one canonical byte order, needs no mark, and stays byte-compatible with the vast installed base of ASCII tools, so a UTF-8 file full of plain English opens correctly in software that predates Unicode. Those practical virtues, not committee decree, made it the default of the web, where it now covers the overwhelming majority of pages.

Frequently asked questions

Is UTF-8 the same as Unicode?

No. Unicode is the catalog that assigns each character a code point. UTF-8 is one way to encode those code points as bytes. Unicode says what the number is; UTF-8 says how to store it.

How many bytes does an emoji use in UTF-8?

Most single emoji sit above U+FFFF and take four bytes. Some visible emoji are actually several code points joined by zero-width joiners, so on disk they can occupy many more bytes than one character suggests.

Why does my text show garbled characters like é?

That is a mismatch: bytes written as UTF-8 are being read as a single-byte encoding like Latin-1, so a two-byte sequence shows up as two wrong characters. Declare and read the file as UTF-8 on both ends to fix it.

Does UTF-8 make files bigger for non-English text?

Sometimes. Latin-script languages stay near one byte per character, but scripts like Chinese or Japanese use three bytes per character in UTF-8 versus two in UTF-16. For mixed or mostly-Latin content, UTF-8 is usually the most compact overall.