A | B | C | D | E | F | G | H | CH | I | J | K | L | M | N | O | P | Q | R | S | T | U | V | W | X | Y | Z | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9
| Standard | Unicode Standard |
|---|---|
| Classification | Unicode Transformation Format, extended ASCII, variable-length encoding |
| Extends | ASCII |
| Transforms / Encodes | ISO/IEC 10646 (Unicode) |
| Preceded by | UTF-1 |
UTF-8 is a variable-length character encoding standard used for electronic communication. Defined by the Unicode Standard, the name is derived from Unicode Transformation Format – 8-bit.[1]
UTF-8 is capable of encoding all 1,112,064[a] valid Unicode code points using one to four one-byte (8-bit) code units. Code points with lower numerical values, which tend to occur more frequently, are encoded using fewer bytes. It was designed for backward compatibility with ASCII: the first 128 characters of Unicode, which correspond one-to-one with ASCII, are encoded using a single byte with the same binary value as ASCII, so that valid ASCII text is valid UTF-8-encoded Unicode as well.
UTF-8 was designed as a superior alternative to UTF-1, a proposed variable-length encoding with partial ASCII compatibility which lacked some features including self-synchronization and fully ASCII-compatible handling of characters such as slashes. Ken Thompson and Rob Pike produced the first implementation for the Plan 9 operating system in September 1992.[2][3] This led to its adoption by X/Open as its specification for FSS-UTF,[4] which would first be officially presented at USENIX in January 1993[5] and subsequently adopted by the Internet Engineering Task Force (IETF) in RFC 2277 (BCP 18)[6] for future internet standards work, replacing Single Byte Character Sets such as Latin-1 in older RFCs.
UTF-8 results in fewer internationalization issues[7][8] than any alternative text encoding, and it has been implemented in all modern operating systems, including Microsoft Windows, and standards such as JSON, where, as is increasingly the case, it is the only allowed form of Unicode.
UTF-8 is the dominant encoding for the World Wide Web (and internet technologies), accounting for 98.2% of all web pages, 99.1% of the top 100,000 pages, and up to 100% for many languages, as of 2024[update].[9] Virtually all countries and languages have 95% or more use of UTF-8 encodings on the web.
Naming
The official name for the encoding is UTF-8, the spelling used in all Unicode Consortium documents. Most standards officially list it in upper case as well, but all that do are also case-insensitive and utf-8 is often used in code.[citation needed]
Some other spellings may also be accepted by standards, e.g. web standards (which include CSS, HTML, XML, and HTTP headers) explicitly allow utf8 (and disallow "unicode") and many aliases for encodings.[10] Spellings with a space e.g. "UTF 8" should not be used. The official Internet Assigned Numbers Authority also lists csUTF8 as the only alias,[11] which is rarely used.
In Windows, UTF-8 is codepage 65001[12] (i.e. CP_UTF8 in source code).
In MySQL, UTF-8 is called utf8mb4[13] (with utf8mb3, and its alias utf8, being a subset encoding for characters in the Basic Multilingual Plane[14]). In HP PCL, the Symbol-ID for UTF-8 is 18N.[15]
In Oracle Database (since version 9.0), AL32UTF8[16] means UTF-8. See also CESU-8 for an almost synonym with UTF-8 that rarely should be used.
UTF-8-BOM and UTF-8-NOBOM are sometimes used for text files which contain or do not contain a byte-order mark (BOM), respectively.[citation needed] In Japan especially, UTF-8 encoding without a BOM is sometimes called UTF-8N.[17][18]
Encoding
UTF-8 encodes code points in one to four bytes, depending on the value of the code point. In the following table, the x characters are replaced by the bits of the code point:
| First code point | Last code point | Byte 1 | Byte 2 | Byte 3 | Byte 4 |
|---|---|---|---|---|---|
| U+0000 | U+007F | 0xxxxxxx | |||
| U+0080 | U+07FF | 110xxxxx | 10xxxxxx | ||
| U+0800 | U+FFFF | 1110xxxx | 10xxxxxx | 10xxxxxx | |
| U+010000 | [b]U+10FFFF | 11110xxx | 10xxxxxx | 10xxxxxx | 10xxxxxx |
The first 128 code points (ASCII) need 1 byte. The next 1,920 code points need two bytes to encode, which covers the remainder of almost all Latin-script alphabets, and also IPA extensions, Greek, Cyrillic, Coptic, Armenian, Hebrew, Arabic, Syriac, Thaana and N'Ko alphabets, as well as Combining Diacritical Marks. Three bytes are needed for the remaining 61,440 codepoints of the Basic Multilingual Plane (BMP), including most Chinese, Japanese and Korean characters. Four bytes are needed for the 1,048,576 codepoints in the other planes of Unicode, which include emoji (pictographic symbols), less common CJK characters, various historic scripts, and mathematical symbols.
A whole graphic character can take more than 4 bytes, because it is made of more than one code point. For instance, a national flag character takes 8 bytes since it is "constructed from a pair of Unicode scalar values" both from outside the BMP.[19][c]
Examples
- In the following examples, red, green, and blue digits indicate how bits from the code point are distributed among the UTF-8 bytes. Additional bits added by the UTF-8 encoding process are shown in black.
- The Unicode code point for the euro sign € is U+20AC.
- As this code point lies between U+0800 and U+FFFF, this will take three bytes to encode.
- Hexadecimal 20AC is binary 0010 0000 1010 1100. The two leading zeros are added because a three-byte encoding needs exactly sixteen bits from the code point.
- Because the encoding will be three bytes long, its leading byte starts with three 1s, then a 0 (1110...)
- The four most significant bits of the code point are stored in the remaining low order four bits of this byte (11100010), leaving 12 bits of the code point yet to be encoded (...0000 1010 1100).
- All continuation bytes contain exactly six bits from the code point. So the next six bits of the code point are stored in the low order six bits of the next byte, and 10 is stored in the high order two bits to mark it as a continuation byte (so 10000010).
- Finally the last six bits of the code point are stored in the low order six bits of the final byte, and again 10 is stored in the high order two bits (10101100).
The three bytes 11100010 10000010 10101100 can be more concisely written in hexadecimal, as E2 82 AC.
The following table summarizes this conversion, as well as others with different lengths in UTF-8.
| Character | Binary code point | Binary UTF-8 | Hex UTF-8 | |
|---|---|---|---|---|
| $ | U+0024 | 010 0100 | 00100100 | 24 |
| £ | U+00A3 | 000 1010 0011 | 11000010 10100011 | C2 A3 |
| И | U+0418 | 100 0001 1000 | 11010000 10011000 | D0 98 |
| ह (Devanagari letter HA) | U+0939 | 0000 1001 0011 1001 | 11100000 10100100 10111001 | E0 A4 B9 |
| € | U+20AC | 0010 0000 1010 1100 | 11100010 10000010 10101100 | E2 82 AC |
| 한 | U+D55C | 1101 0101 0101 1100 | 11101101 10010101 10011100 | ED 95 9C |
| 𐍈 | U+10348 | 0 0001 0000 0011 0100 1000 | 11110000 10010000 10001101 10001000 | F0 90 8D 88 |
| Suppl Private Use Area B | U+1096B3 | 1 0000 1001 0110 1011 0011 | 11110100 10001001 10011010 10110011 | F4 89 9A B3 |
The Vietnamese phrase Mình nói tiếng Việt (𨉟呐㗂越, "I speak Vietnamese") is encoded as follows:
| Character | M | ì | n | h | n | ó | i | t | i | ế | n | g | V | i | ệ | t | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Code point | 4D | EC | 6E | 68 | 20 | 6E | F3 | 69 | 20 | 74 | 69 | 1EBF | 6E | 67 | 20 | 56 | 69 | 1EC7 | 74 | ||||||
| Hex UTF-8 | C3 | AC | C3 | B3 | E1 | BA | BF | E1 | BB | 87 | |||||||||||||||
| Character | 𨉟 | 呐 | 㗂 | 越 | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Code point | 2825F | 5450 | 35C2 | 8D8A | |||||||||
| Hex UTF-8 | F0 | A8 | 89 | 9F | E5 | 91 | 90 | E3 | 97 | 82 | E8 | B6 | 8A |
Codepage layout
The following table summarizes usage of UTF-8 code units (individual bytes or octets) in a code page format. The upper half is for bytes used only in single-byte codes, so it looks like a normal code page; the lower half is for continuation bytes and leading bytes and is explained further in the legend below.