UTF-8 and character sets
Why 'one character = one byte' is wrong, what code points are, and how UTF-8 became the default for the web.
Characters are not bytes
A code point is a number assigned to a character by the Unicode standard (for example, A is U+0041, Γ© is U+00E9, π is U+1F600). An encoding decides how that number is stored as bytes. The same character can be stored differently by UTF-8, UTF-16, or UTF-32.
β οΈ
Mixing encodings is the classic cause of mojibake (garbled text like
ΓΒ©). Always declare and agree on one encoding end to end.How UTF-8 works
UTF-8 is a variable-width encoding: ASCII characters (0β127) take exactly one byte and are identical to ASCII, which is why legacy English text keeps working. Other characters take 2β4 bytes.
| Code point range | UTF-8 bytes |
|---|---|
| U+0000 β U+007F | 1 byte (same as ASCII) |
| U+0080 β U+07FF | 2 bytes |
| U+0800 β U+FFFF | 3 bytes |
| U+10000 β U+10FFFF | 4 bytes |
A β 0x41 (1 byte)
Γ© β 0xC3 0xA9 (2 bytes)
δΈ β 0xE4 0xB8 0xAD (3 bytes)
π β 0xF0 0x9F 0x98 0x80 (4 bytes)UTF-8 is the web default
- Declare it:
<meta charset="utf-8">in HTML, andContent-Type: text/html; charset=utf-8over HTTP. - For APIs, JSON text is defined by the spec to be UTF-8; do not add a BOM.
- Read files with an explicit encoding:
open(f, encoding='utf-8')in Python, not the platform default.
π‘
The Byte Order Mark (BOM) is meaningless for UTF-8 and often breaks parsers (JSON, shell scripts). Save UTF-8 without BOM.
FAQ
Is UTF-8 the same as Unicode?
No. Unicode is the character table (code points); UTF-8 is one way to encode those code points into bytes.
Why do I get Γ’β¬Β’ instead of β’?
The β’ (U+2022) was written as UTF-8 but read as Latin-1. Re-decode as UTF-8 to fix it.
Related
Base64 encoding URL encoding (percent-encoding)
Last refreshed 2026-09-17.