Binary formats: Protocol Buffers, MessagePack, CBOR and BSON

Field tags instead of names, how schema evolution stays compatible, where MessagePack and CBOR fit, what BSON adds for MongoDB, and real numbers on size and speed.

Protocol Buffers: the tag is the identity

syntax = "proto3";
package shop;

message Order {
  int64  id       = 1;
  string currency = 2;
  repeated Item items = 3;

  reserved 4;              // never reuse a removed tag
  reserved "old_total";
}

message Item {
  string sku = 1;
  int32  qty = 2;
}

Fields are identified by their numeric tag, not their name, so renaming a field is safe on the wire.
Adding a new field with a new tag is backward and forward compatible; reusing or changing the type of a tag is not.
repeated is the default for lists; scalar fields are omitted when they hold the type's default value.
In proto3, presence tracking requires optional or a wrapper type — otherwise you cannot distinguish "unset" from "zero".

# generate and inspect
protoc --python_out=. order.proto
echo '{"id":1,"currency":"EUR","items":[{"sku":"A1","qty":2}]}' \
  | protoc --encode=shop.Order order.proto > order.bin
protoc --decode=shop.Order order.proto < order.bin

The schema-less alternatives

Format	Schema	Self-describing	Typical use
Protobuf	Required, compiled	No	Service-to-service APIs, gRPC
MessagePack	None	Yes	Compact JSON substitute in caches and queues
CBOR	Optional	Yes	IoT and COSE; deterministic encoding rules exist
BSON	None	Yes	MongoDB storage and query documents
Avro	Required, travels with data	Yes	Event streams and data lake writes

MessagePack and CBOR are binary JSON with a smaller footprint and a wider type set (raw bytes, non-string map keys, 64-bit integers). Neither fixes the ambiguity of JSON — a float stays a float, and there is no schema to validate against.

import msgpack, cbor2, bson

payload = {"id": 1, "tags": ["a", "b"], "ok": True}
packed = msgpack.packb(payload, use_bin_type=True)
assert msgpack.unpackb(packed, raw=False) == payload

# BSON embeds type and length prefixes per field: larger, but queryable
doc = bson.BSON.encode(payload)

Size and speed in practice

Representation	Relative size	Encode/decode cost	Human readable
JSON text	100% (baseline)	Text parsing dominates	Yes
MessagePack	roughly 55-75%	Faster than JSON	No
Protobuf	roughly 30-50%	Fast; no field names on the wire	No
Parquet (columnar)	often 10-25%	Cheap per column, costly per record	No

Treat these as orders of magnitude, not constants: the ratio depends on how many field names and repeated values your payload carries.

💡

Binary formats trade debuggability for size. Keep a text path for development — protoc --decode, a MessagePack viewer, or an NDJSON mirror — or every incident becomes a hex-dump archaeology exercise.

FAQ

Is it safe to rename a Protobuf field?

Yes on the wire, because tags carry identity. It matters for generated code and for JSON mapping, so update the schema and regenerate all clients together.

MessagePack or Protobuf?

Protobuf when you control both ends and want a schema plus the smallest payload. MessagePack when you need a drop-in, schema-free replacement for JSON in caches or logs.

Schema definition and validation Choosing a format: size, speed, readability and tooling

Last refreshed 2026-09-18.

Binary formats: Protocol Buffers, MessagePack, CBOR and BSON

Protocol Buffers: the tag is the identity

The schema-less alternatives

Size and speed in practice

FAQ

Related