Character Limits That Actually Bite: Code Units, Code Points and Graphemes

Three units, all of them called a character

A UTF-16 code unit is 16 bits of storage. Anything above U+FFFF - which is every emoji, every historic script and a good deal of CJK - needs two of them, called a surrogate pair. JavaScript strings are defined as sequences of UTF-16 code units, so .length counts those, and so does Java's String.length() and C# for a .NET string.

A code point is one entry in the Unicode character database, written U+0041 or U+1F44D. This is the unit Python 3's len() returns and the unit that spread syntax and for-of iteration use in JavaScript. It is closer to intuition than a code unit, but still not what a reader sees, because several code points routinely combine into one visible mark.

A grapheme cluster - the standard calls it an extended grapheme cluster, defined in Unicode Annex 29 - is the user-perceived character. It is what a caret jumps over, what a backspace deletes, and what a person counts. Swift's String.count returns graphemes; so does Intl.Segmenter in JavaScript with granularity set to grapheme. No other mainstream language returns them by default, which is the root cause of nearly every emoji bug you have seen.

One string, four numbers

Take a short, entirely ordinary message: Shipping to, a French flag emoji, today, a thumbs-up with a medium skin tone, an em dash, and thanks with an exclamation mark. Measured in Node 22 it is 37 UTF-16 code units, 33 code points, 31 grapheme clusters and 47 bytes in UTF-8. A person reading it would say it is 31 characters long. JavaScript will tell you 37. A byte-limited database column will see 47.

The gaps come from two places. The flag emoji is a pair of regional indicator symbols - two code points, four UTF-16 units, one visible flag - and the thumbs-up is a base emoji plus a skin-tone modifier, again two code points and four UTF-16 units for one visible mark. Between them they account for the 37 against 31. Nothing exotic is happening; this is what a normal message from a normal phone looks like.

The same visible text can have two different lengths

The word Cafe with an acute accent, followed by a space, a skin-toned thumbs-up and an exclamation mark, measures 10 UTF-16 units, 8 code points, 7 graphemes and 15 UTF-8 bytes when the accented letter is the single precomposed code point U+00E9. Type the identical-looking string on a Mac that emits e followed by a combining acute instead, and it becomes 11 UTF-16 units, 9 code points, still 7 graphemes, and 16 UTF-8 bytes.

Same pixels, different length, and a byte-count check that passes in one case fails in the other. The fix is to normalise on input: apply Unicode Normalization Form C, which composes e plus combining acute back into U+00E9, before you measure, store, compare or hash. Almost every duplicate-detection bug involving accented names traces back to comparing an NFC string with an NFD one.

Truncation is where the abstraction breaks in public

Cut that Cafe string at six UTF-16 units with a plain slice and you get C, a, f, the accented e, a space and then U+D83D on its own - the high half of a surrogate pair with nothing to pair with. That lone surrogate is not a valid character. Renderers show a replacement box, JSON encoders emit an escape that some parsers reject, and databases with strict UTF-8 validation refuse the write outright.

Truncating on code points instead avoids the lone-surrogate crash but still cuts a family emoji into two adults and a child, or strips the skin-tone modifier off a thumbs-up so it renders yellow. The only truncation that is safe in front of a user is grapheme-based: segment with Intl.Segmenter, take the first n segments, join them. If your limit is expressed in bytes, do both - count graphemes to decide where a cut is legal, and count UTF-8 bytes to decide how many you can afford.

Which platform counts which unit

X does not count characters at all: it computes a weighted length in which characters in the Latin, Latin-1 supplement and general punctuation ranges count 1, and everything else - CJK, Arabic, Cyrillic beyond the basic block, and every emoji - counts 2. A 280-limit post therefore holds 280 Latin letters or 140 emoji. Bluesky enforces two limits at once on the same field: 300 grapheme clusters and 3,000 bytes, so a post of 300 flag emoji is legal on graphemes and fails on bytes. Mastodon's default 500 is counted in code points, with any URL charged a flat 23 regardless of its real length.

The practical consequence is that a single counter widget cannot serve every destination. If you schedule the same copy to four networks, you need four counts, and a preview that shows the string as the reader will see it rather than as the storage layer measures it. When in doubt, count graphemes for the human-facing limit and UTF-8 bytes for the machine-facing one, and treat any single number labelled characters with suspicion until you know which unit produced it.

SMS: 160 becomes 70 the moment one character leaves GSM-7

An SMS body is 140 octets. Encoded in the GSM 7-bit default alphabet defined in 3GPP TS 23.038, seven bits per character, that gives 160 characters. But the GSM-7 alphabet has only about 128 slots plus a small extension table, and if even one character in the message is outside it, the whole message must be re-encoded in UCS-2 at 16 bits per character: 140 octets divided by 2 is 70 characters. Not 70 for the offending character - 70 for the entire message.

Work an example. A 145-character booking confirmation fits in one GSM-7 segment, because 145 is under 160. Paste it out of a word processor that has silently converted the straight apostrophe to a typographic one, or add an em dash, or an emoji, and the message becomes UCS-2. Now 145 is over 70, so it is split into concatenated parts, and concatenation steals six octets per part for the segmentation header, leaving 67 characters each. 145 divided by 67 rounds up to 3. One invisible substitution turned one billable message into three.

Which characters are safe is not intuitive. Several accented letters are in the GSM-7 basic set - lowercase e-acute, e-grave, a-grave, u-grave, the German sharp s and the umlauted vowels among them - so a French or German SMS is not automatically UCS-2. But a-acute, i-acute, o-acute and u-acute are not there, which means Spanish and Portuguese accents usually do force the switch. The euro sign lives in the extension table and costs two septets rather than one. Curly quotes, en and em dashes, the ellipsis character and every emoji are simply absent.

VARCHAR(255) is not 255 of anything in particular

In PostgreSQL, varchar(255) is 255 characters, meaning code points, and the storage grows to whatever UTF-8 needs. In MySQL, VARCHAR(255) has counted characters rather than bytes since version 5.0, so under the utf8mb4 character set the same column can occupy up to 1,020 bytes. In Oracle, VARCHAR2(255) means 255 bytes unless you write VARCHAR2(255 CHAR) or change NLS_LENGTH_SEMANTICS, which is why a 255-limit field in an Oracle-backed system silently rejects a 200-character name written in an accented alphabet. In SQL Server, VARCHAR is bytes in a single-byte collation and NVARCHAR is UTF-16 code units at two bytes each.

There is a second-order trap in MySQL specifically. The old InnoDB index prefix limit of 767 bytes meant a VARCHAR(255) column in utf8mb4 - up to 1,020 bytes - could not be fully indexed, which is where the folklore of VARCHAR(191) comes from: 191 times 4 is 764, just under the limit. Modern InnoDB with DYNAMIC row format raises the prefix limit to 3,072 bytes and the workaround is obsolete, but the 191-length columns it created are still in production schemas everywhere.

Frequently asked questions

Why does JavaScript say my emoji is 2 characters long?

Because .length counts UTF-16 code units, and every emoji sits above U+FFFF, so it needs a surrogate pair - two units. Add a skin-tone modifier and you get four. Use [...str].length to count code points, or Intl.Segmenter with granularity grapheme to count what a reader would count. The one-liner is: new Intl.Segmenter(undefined, { granularity: 'grapheme' }).segment(str) gives you an iterable of visible characters.

How many bytes does one character take in UTF-8?

One to four, decided by the code point. ASCII up to U+007F takes 1 byte. Latin accents, Greek and Cyrillic up to U+07FF take 2. Most of the Basic Multilingual Plane, including CJK, takes 3. Everything above U+FFFF, which is all emoji, takes 4. So a skin-toned thumbs-up is 8 bytes because it is two code points of 4 bytes each, and the four-person family emoji is 25 bytes: four emoji at 4 bytes plus three zero-width joiners at 3 bytes each.

Why did my 145-character SMS get billed as three messages?

Because one character in it was not in the GSM 7-bit alphabet, so the whole message switched to UCS-2 at 70 characters per segment. Above 70, segments carry a concatenation header that costs six octets, leaving 67 characters each, and 145 divided by 67 rounds up to 3. The usual culprits are a typographic apostrophe, an em dash, an ellipsis character or an emoji, all of which word processors and phone keyboards insert automatically. Run the copy through a byte-and-encoding counter before you send a campaign, not after.

Should I store a limit in characters or in bytes?

Enforce two limits, not one. Use a grapheme limit for what you show the user, because that is the number they can verify by looking. Use a byte limit for storage and transport, because that is what the column, the payload and the protocol actually constrain. If you can only have one, make it bytes and make the UI honest about it, since a single byte overflow is a write failure while a grapheme overflow is only an aesthetic problem.

Do the same rules apply to a meta description or title tag?

No, and this is the one place where counting characters is the wrong instinct entirely. Search engines truncate titles and descriptions by rendered pixel width, not by character count, so a title of 60 narrow letters can fit where 50 wide ones do not, and an emoji in a title occupies far more width than its single grapheme suggests. Character targets like 60 and 155 are rules of thumb that approximate a pixel budget. Write for the pixel budget, verify in a SERP preview, and use the character counter only to stop yourself drifting into obviously unrecoverable lengths.