NDEF Text Records: UTF-8 vs UTF-16 and Language Codes

An NDEF Text record stores its text as either UTF-8 or UTF-16, and the very first byte of the payload says which. The same byte also holds the length of the language code that follows. If a tag reads as garbage on one phone and fine on another, that first byte is the place to look.

The payload layout: status byte, language, text

A Text record is a Well Known record with type T. Android names that type RTD_TEXT and uses it with TNF_WELL_KNOWN, as Android's NFC basics guide shows in its sample code. The payload has three parts, in this order:

1. One status byte.
2. The language code, as US-ASCII bytes.
3. The text itself, in the encoding the status byte names.

Take the word "Hello" in English, as UTF-8. The payload starts 02 65 6E, which is the status byte followed by the ASCII letters "en", and then the text bytes 48 65 6C 6C 6F. Nothing else sits between the parts. There's no terminator after the language code and no length field for the text, because the record's own payload length covers it.

The status byte: bit 7 and the language length

The Android sample builds the status byte by adding a UTF flag to the language length. The flag is 0 for UTF-8 and 1 << 7 for UTF-16. In plain terms:

• Bit 7: 0 means UTF-8, 1 means UTF-16.
• Bit 6: reserved. The open-source ndeflib reference code never sets it on write and ignores it on read.
• Bits 0 to 5: the language code length. ndeflib reads them with the mask 0x3F.

So an English UTF-8 record starts with 0x02. The same record in UTF-16 starts with 0x82. With a two-letter language code, those are the two values you'll see most. The ndeflib documentation says a Text record carries "a UTF-8 or UTF-16 encoded text string", and those are the only two encodings it accepts.

We haven't quoted the NFC Forum's own Text record specification here, so treat the bit positions above as what Android's sample and ndeflib both do, not as a quotation from the standard.

Language codes

The language is an IANA or ISO identifier such as en or de, written as ASCII. ndeflib's decoder rejects a language length of zero and a length longer than the payload, and its setter accepts 1 to 63 characters. It does not check the code against the IANA registry, so a typo like ne for en is stored without complaint.

On Android, you can leave the language out. The NdefRecord reference says createTextRecord uses "the language code corresponding to the current default locale" when you give none. That's convenient, but a tag written on a phone set to German will say de even if the text is English. If your readers filter or display by language, set it yourself.

UTF-16 and the byte order mark

UTF-16 stores each character in two bytes, so you need to know which byte comes first. A byte order mark (BOM) answers that: FE FF for big-endian, FF FE for little-endian. According to Wikipedia's byte order mark article, UTF-16 "may or may not begin with a BOM", and when there is none, Unicode says to assume big-endian, though the article adds that this presumption is "widely ignored".

That's why UTF-16 is the fragile choice for a tag. A writer might include a BOM, another might leave it out, and a reader might guess the other byte order. ndeflib, for example, uses Python's generic UTF-16 codec, which writes a BOM when encoding and relies on it when decoding. UTF-8 has none of this: bytes are read in order, and plain ASCII text is the same bytes in UTF-8 as in ASCII.

Why Android defaults to UTF-8

Android's own helper, NdefRecord.createTextRecord(languageCode, text), is documented as creating "a new NDEF record containing UTF-8 text data". The longer sample in the NFC basics guide lets you choose UTF-16, but the one-line helper does not.

A reported iPhone UTF-16 problem

Two developers on Apple's forums, in a thread called "coreNFC write NDEF with utf8", say that NFCNDEFPayload.wellKnownTypeTextPayload(string:locale:) writes UTF-16 and that the result was unreadable in an Android app. The first post is from 2019 and a second developer asks the same question in 2020. No Apple engineer replied, and the thread offers no workaround or error message.

That thread can't tell us what current iOS versions do. Treat it as a reason to test, not as a confirmed rule: write a tag with your iPhone tool, read it on an Android phone, and look at the first byte.

Checking what your tag actually holds

Read the raw payload of the record and look at the first byte. 02 followed by 65 6E is UTF-8 with "en". 82 means the text is UTF-16, and you should expect a BOM right after the language code. If you only want to move the message to another tag, see copying the NDEF message rather than a raw dump, and if a rewrite fails, check whether the tag is read-only first.

An NFC app that shows raw records makes this a ten-second check. NFCore reads and writes NDEF on iPhone and Android. If a tag shows 82, rewrite it as UTF-8 with a tool that lets you pick the encoding.

The short version

Look at bit 7 of the first byte: 0 is UTF-8, 1 is UTF-16. Prefer UTF-8, set the language code on purpose, and when a tag reads wrong on one phone, check the encoding before you blame the chip.

Frequently Asked Questions

Does an NDEF Text record have to be UTF-8?

No. The record allows UTF-8 or UTF-16, chosen by bit 7 of the status byte. UTF-8 is the safer choice because it is what Android's own text-record helper writes.

How long can the language code be?

The length sits in the low bits of the status byte. ndeflib reads six bits, so up to 63 characters, and rejects zero. In practice codes such as en or de are a few ASCII characters.

Why would a Text record read fine on one phone and not another?

One common cause is the encoding bit. A reader that assumes UTF-8, or one that mishandles a UTF-16 byte order mark, can show garbage. Check the first payload byte: 02 means UTF-8 with a two-letter language, and 82 means UTF-16.