IoT Data Formats: JSON, CBOR, MessagePack, Protobuf

Every telemetry message, command, and config payload in an IoT system has to be serialized before it hits a radio or socket, and that choice ripples through everything downstream: airtime on a constrained link, RAM for buffering, CPU cycles for parsing on an MCU, and how much friction developers face wiring up a cloud app. Picking JSON everywhere because "it's simple" can cost 2–4× the airtime of a binary format on a battery-powered LPWAN node, while picking a compact binary format everywhere can make debugging and schema evolution painful. Understanding the actual trade-offs — encoding size, parse cost, schema requirements, tooling — is a foundational decision that shapes device firmware, gateway logic, and cloud ingestion pipelines alike.

The Four Formats at a Glance

All four represent the same conceptual data — numbers, strings, arrays, maps — but differ in how much of the structure is described inline (self-describing) versus predefined in a schema.

JSON: The Universal Baseline

JSON (JavaScript Object Notation) is UTF-8 text with keys, values, arrays and nesting:

{"dev":"sensor-14","temp":23.7,"hum":48,"ts":1718000000}

• Self-describing: any consumer can parse it without prior knowledge of field types.
• Ubiquitous tooling: every language, every cloud IoT service (AWS IoT Core, Azure IoT Hub, MQTT brokers, REST APIs) accepts JSON natively.
• Cost: field names are repeated in every message, numbers are stored as ASCII digits rather than binary, and parsing involves character-by-character tokenizing — expensive relative to binary formats on a Cortex-M0 with no FPU.

The line above is 57 bytes for four fields. Over a low-power radio at a few kbps, that overhead is real: at LoRaWAN's ~0.3–5 kbps effective rate, every extra byte is measurable airtime and energy.

CBOR: Binary JSON, Same Data Model

CBOR (Concise Binary Object Representation, RFC 8949) was designed specifically for constrained nodes and keeps JSON's flexible, schema-less data model but encodes it in a compact binary tag+length+value structure.

• Each item starts with a major type (3 bits) and additional info describing length, so a small integer or short string encodes in 1–2 bytes instead of ASCII digits.
• Supports data types JSON lacks natively: byte strings, tagged values (e.g., timestamps, bignums), indefinite-length streaming.
• No schema needed — a decoder can walk the structure generically, same as JSON, just faster and smaller.

Encoding the example above in CBOR: the map header, short text keys, and a small integer/float values typically come to ~35–40 bytes versus 57 for JSON — roughly a 30–40% reduction with zero schema overhead. This is why CBOR is the payload format specified alongside CoAP in many constrained-device stacks (it's explicitly referenced in IETF constrained-RESTful-environments work).

MessagePack: The Pragmatic Alternative

MessagePack has essentially the same goal and mechanism as CBOR — binary, self-describing, JSON-compatible data model — developed slightly earlier and popular in web/gaming/logging ecosystems before CBOR was standardized.

• Byte-for-byte sizes are close to CBOR's for most payloads; neither has a decisive size advantage in general.
• CBOR has an IETF RFC and is the format of choice in standards-track IoT protocols (CoAP, SenML); MessagePack has broader ad-hoc library support in scripting languages (Python, Ruby, JS) and is common where a team just wants "smaller JSON" without adopting an IETF-aligned stack.
• Practical choice is often driven by ecosystem fit rather than technical difference: if your protocol stack (CoAP, OMA LwM2M) already specifies CBOR, use CBOR; if you're gluing together services with existing MessagePack libraries, that's fine too.

Protobuf: Schema-Driven Compactness

Protocol Buffers (Google) take a different approach: the message structure is defined once in a .proto schema, compiled into language-specific code, and the wire format contains almost no structural metadata — just field numbers and values.

message Telemetry {
  string dev = 1;
  float temp = 2;
  uint32 hum = 3;
  uint64 ts = 4;
}

• Field tags, not names, on the wire: dev becomes tag 1, so the encoded field is a 1-byte tag + varint length + payload — no string "dev" repeated in every message.
• Varint encoding: small integers use fewer bytes (e.g., hum = 48 encodes in 1 byte instead of 2 ASCII digits or a fixed-width int).
• Result: the same telemetry example typically encodes in ~20–25 bytes, roughly 2.5× smaller than JSON and noticeably smaller than CBOR/MessagePack, because field names never appear on the wire at all.
• Cost: both ends need the compiled schema. Adding a field is backward-compatible (old code ignores unknown tags), but you lose the "just read it in a browser console" debuggability that JSON offers, and schema versioning/build tooling becomes part of the firmware release process.

Worked Comparison

Same logical message — device ID string (9 chars), float temperature, integer humidity, 32-bit Unix timestamp — encoded four ways:

JSON:         57 bytes   {"dev":"sensor-14","temp":23.7,"hum":48,"ts":1718000000}
CBOR:        ~38 bytes   binary map, 4 key/value pairs, short keys
MessagePack: ~36 bytes   binary map, similar layout to CBOR
Protobuf:    ~22 bytes   tag+varint fields, no key names on wire

Check: Protobuf's savings come almost entirely from omitting the four field names ("dev", "temp", "hum", "ts" = 12 bytes of key text alone, plus quoting/colons in JSON) and using varints for the small integers. That accounts for the bulk of the ~35-byte gap between JSON and Protobuf, consistent with the numbers above.

At 1,000 messages/day from one device, JSON costs ~57 KB/day of payload; Protobuf costs ~22 KB/day. Multiply by a fleet of 10,000 devices: 10,000 × 57 KB/day ≈ 570 MB/day for JSON versus 10,000 × 22 KB/day ≈ 220 MB/day for Protobuf — a difference that is real cellular/LPWAN data cost and battery drain from radio-on time.

Design Implications

• Constrained Class 0/1 devices (RFC 7228, kilobytes of RAM): favor CBOR or Protobuf — both avoid the tokenizing overhead of text JSON, and both have C libraries with small footprints (nanopb for Protobuf, tinycbor for CBOR).
• Debuggability during development: JSON is often kept at the gateway or cloud boundary (e.g., MQTT topic payloads translated to JSON for a dashboard) even if the device itself speaks CBOR/Protobuf on the wire, so engineers can mosquitto_sub and read messages directly.
• Schema evolution: if devices in the field will receive firmware updates at different times, Protobuf's tagged-field backward compatibility is safer than a bespoke binary format, but require strict field-number discipline (never reuse a retired tag).
• Interoperability with standards: CoAP-based stacks and OMA LwM2M lean on CBOR/SenML; MQTT-based stacks are format-agnostic and commonly carry JSON or Protobuf depending on the vendor.
• Gateway role: gateways frequently act as translators — decoding a compact binary format from constrained nodes and re-encoding to JSON for cloud APIs that expect it, trading a small amount of CPU time at the gateway for compatibility on both sides.

Key Takeaways

• JSON is self-describing and universally tooled but carries the heaviest per-message overhead — field names and ASCII-encoded numbers repeated in every payload.
• CBOR and MessagePack keep JSON's schema-less flexibility while cutting size by roughly 30–40% through binary tag+length+value encoding; CBOR has IETF standardization behind it (RFC 8949) and ties into CoAP/LwM2M stacks.
• Protobuf achieves the smallest payloads (often 2–2.5× smaller than JSON) by moving field names into a compiled schema and using varint encoding, at the cost of requiring schema management and losing raw human-readability.
• The right choice depends on constraints: RAM-limited Class 0/1 devices and metered LPWAN links favor binary formats; debugging convenience and quick integration favor JSON at least at some boundary in the pipeline.
• Real systems often mix formats — compact binary on the constrained link, JSON at the cloud API boundary — with gateways doing the translation.