Specs, decoders and raw data

A device sends bytes. A spec is what decides how long they are kept, how they are read, and what the dashboard draws. A device with no spec still works — its bytes are simply stored as they arrived.

What a spec is

A spec is the description a group of devices shares. Devices that send the same kind of payload sit on one spec, and it carries four things:

  • Transport — how its devices report: HTTPS, HTTP, MQTT, MQTTS, TCP or UDP.
  • Security level — how much a device must prove before a message is accepted. See Security levels.
  • History — how many days of messages are kept.
  • A decoder, and dashboard controls — how the bytes become named values, and what appears on the device page.

Changing a spec changes it for every device on it. The setup card says how many other devices that is before you change anything.

What a decoder adds

A decoder is a small script on the spec. It runs at the moment a message is stored and turns the payload into named fields — temp_c 21.4 instead of two bytes of hex. Nothing else about the message changes: the raw bytes are always kept too.

Decoding happens once, on arrival. Adding a decoder does not go back and read the messages that are already stored, so the first decoded reading you see is the next one the device sends.

A board that sends JSON needs the shortest decoder there is. If it sends {"temp_c": 21.4, "door": "open"}, this is the whole script:

def decode(payload, meta):
    return json.decode(payload)

json.loads is the same function, as in Python. A field is a number, text or true/false, so from JSON with objects inside objects, pick out the values you want: d = json.decode(payload), then return {"temp_c": d["env"]["t"]}. A packet that is not JSON is kept raw, and the device page’s message history shows why the decoder could not read it.

Three different outcomes

What you seeWhat it meansWhat to do
A reading, with namesA decoder ran and read the payload.Nothing.
Data received and stored as raw bytesNo decoder ran — the device has no spec, or its spec has none.Attach a spec on the device page, then write its decoder.
Data received, but the decoder could not read itA decoder ran and failed on this payload.Open the spec's decoder and check it against the stored bytes.

The middle row is not a failure. Arriving raw is a legitimate way to start: you get real bytes from real hardware, and you write the decoder against them instead of against a guess.

Working from raw data

Every stored message keeps its raw payload, whether or not anything decoded it. On the device page you can read those bytes, put the device on a spec, and open that spec's decoder editor with this device's own payload already loaded.

Setup never attaches a decoder to bytes it cannot read. When you install your own firmware, or keep firmware setup did not write, the honest answer is no spec — and the card says so rather than pairing your payload with somebody else's decoder.

A demo installs its own spec, decoder and dashboard controls with it, so a demo board arrives decoded on its first message. A demo that comes with a routine adds that too, switched on; it is listed on the Routines page like any other.