Protocol Buffers: field numbers, unknown fields and the rules for evolving a message

What it is

A .proto file declares messages whose fields have a type, a name and a field number. The language guide (cited) states that the number identifies the field in the wire format, must be unique within the message, lies between 1 and 536,870,911 (19,000 to 19,999 are reserved for the implementation) and cannot be changed once the message is in use: "changing" a number is the same as deleting the field and adding a new one. Numbers 1 to 15 encode in one byte and 16 to 2047 in two, which is why the most frequently set fields should get the low numbers.

Why it matters

The binary format carries only numbers and wire types, never names. If a number is reused for a different field, a parser cannot tell which definition wrote the data; the guide lists parse errors, leaked data and corruption as consequences, and the best-practices page puts it bluntly: never re-use a tag number. The evolution rules are the only thing that keeps old and new binaries talking.

How to apply

• Adding fields is wire-safe: old code parses new messages and keeps the new fields as unknown fields (proto3 preserves them and re-serialises them); new code reading old messages sees the default value, so choose defaults that mean "not set".
• Removing a field is safe only if its number goes into a reserved statement, and its name too if JSON or text formats are in use: reserved 2, 15, 9 to 11; and reserved "old_name";. Alternatively keep the field and rename it with an OBSOLETE_ prefix.
• Adding enum values is safe; changing a field's number or moving fields into an existing oneof is not.
• int32, uint32, int64, uint64 and bool are wire-compatible but lossy: a value above the 32-bit maximum read as int32 is truncated. Widen a type only after every reader is deployed, and never in a schema published outside your control.
• Enforce the rules mechanically: the compiler rejects use of reserved numbers, and a registry or schema linter can refuse the remaining unsafe edits before they reach a build.

Pitfalls

Unknown fields survive binary round trips but are lost when a message is converted to JSON or copied field by field into a new message. The rules above are for the binary format; ProtoJSON (cited) has its own list of safe and unsafe changes because field names appear on the wire there, and it writes int64 values as strings so that no parser silently loses precision on large values. Renumbering fields to tidy up the file is a full incompatible change.