Using ABCL to work with binary payloads

When you want to to get data from your device over an LPWAN network (such as LoRaWAN), you cannot use a plain JSON format over HTTP(S) or MQTT. Given their contrained nature, LPWAN networks expect that your device sends over a binary payload. The payload should be as compact as possible to minimise the Time-on-Air and reduce power consumption (for battery operated) devices.

This guide will help you to encode and decode different types of data in as little bytes as possible.

Get started with ABCL

ABCL stands for AllThingsTalk Binary Conversion Language , and it’s a JSON-based, domain specific language, used for encoding and decoding of AllThingsTalk asset data to and from binary payloads. It gives you the freedom to specify your own decoding scheme on a per device level.

Mapping statements

Mapping statements enclosed within a sense block define how the binary data received from the edge, will be decoded into understandable asset format.

{"sense": [
  // mapping statements
]}

Byte

Set the byte to parse. When bytelength is set, this is the first byte we look at from the left.

Note that selectors are zero-indexed. The first byte is {"byte": 0 }

{"sense": [
  {"asset": "temperature", "value": {"byte": 0}}
]}

Bytelength

By using the selector bytelength we can specify how many bytes make up our value.

{"sense": [
  {"asset": "temperature", "value": {"byte": 0, "bytelength": 2, "type": "integer"} }
]}

Byteorder

Set the preferred byteorder with this keyword. Big endian or Little endian. Supported options are

• little
• big

By default big endian is used

{"sense": [
  {"asset": "big", "value": {"byte": 0, "bytelength": 2, "byteorder": "big", "type": "integer"}},
  {"asset": "little", "value": {"byte": 0, "bytelength": 2, "byteorder": "little", "type": "integer"}}
]}

Bit

Select a specific bit within a byte. With just a single bit, the output value can of course only be _0_ or _1_.

Same as byte, bit is zero-indexed

{"sense": [
  {"asset": "test1", "value": {"byte": 0, "bit": 6}}
  {"asset": "test2", "value": {"byte": 0, "bit": 5}}
]}

Bitlength

Select multiple consecutive bits with the bitlength selector.

Bitorder is always Most Significant Bit (MSD) first

{"sense": [
  {"asset": "test1", "value": {"byte": 0, "bit": 3, "bitlength": 2}},
  {"asset": "test2", "value": {"byte": 0, "bit": 3, "bitlength": 3}},
  {"asset": "test3", "value": {"byte": 0, "bit": 2, "bitlength": 3}}
]}

Signed value

When signed is

• false, values range from 0 to 255
  • 0x00 = 0
  • 0xFF = 255
• true, values range from -128 to 127 (values are represented by their two’s complement)
  • 0x00 = 0
  • 0x7F = 127
  • 0x80 = -128
  • 0xFF = -1

By default, values are signed

{"sense": [
  {"asset": "test1", "value": {"byte": 0}},
  {"asset": "test2", "value": {"byte": 0, "signed": true}},
  {"asset": "test3", "value": {"byte": 0, "signed": false}}
]}

Fixed value

We can also send a fixed value, instead of parsing some bytes. This is particularly useful in a switch statement. Instead of setting bits and bytes to parse, simply set a fixed String in the value field. The example below shows how to use this.

Switch case

Using a switch/case statement, we can parse payloads differently depending on the value of certain bytes.

{"sense": [
  {"switch": {"byte": 0}, "on": [
    {"case": 0, "do": [ {"asset": "message", "value": "Running"} ]},
    {"case": 1, "do": [ {"asset": "message", "value": "Debug mode"} ]}
  ]}
]}

Or if we also want the raw value, we can simple parse it as well

{"sense": [
  {"asset": "raw", "value": {"byte": 0}},
  {"switch": {"byte": 0}, "on": [
    {"case": 0, "do": [ {"asset": "message", "value": "Running"} ]},
    {"case": 1, "do": [ {"asset": "message", "value": "Debug mode"} ]}
  ]}
]}

Type

• integer an integer whose bytelength is specified with bytelength and its sign by signed
• number an IEEE 754 floating point number. Its precision is determined by bytelength
  • bytelength of 2 indicates half precision
  • bytelength of 4 indicates single precision
  • bytelength of 8 indicates double precision
• string a UTF-8 string
• boolean a boolean value. If all bits in a chunk are zero, the value is false. Otherwise, it’s true
• datetime seconds since UNIX epoch. The chunk is read as a signed integer and converted to seconds
• hexstring a UTF-8 string presentation of provided hexadecimal binary (this type is only available for converting hex to json).

{"sense": [
  {"asset": "text", "value": {"byte": 0, "type": "string"}},
  {"asset": "toggle", "value": {"byte": 0, "type": "boolean"}},
  {"asset": "date", "value": {"byte": 0, "bytelength": 4, "type": "datetime"}}
]}

{"sense": [
  {"asset": "int", "value": {"byte": 0, "bytelength": 2, "type": "integer"}},
  {"asset": "half", "value": {"byte": 0, "bytelength": 2, "type": "number"}}
]}

{
    "sense": [
        {
            "asset": "hardware_unique_id",
            "value": {
                "byte": 0,
                "bytelength": 16,
                "byteorder": "big",
                "signed": "false",
                "type": "hexstring"
            }
        }
    ]
}

Calculation

Add simple calculations to your values before sending them to the cloud. Use val as the value placeholder in your calculation.

Supported operators are

• addition +
• subtraction -
• division /
• multiplication *
• exponent **
• logatirthm log
• square root sqrt

{"sense": [{
    "asset": "temperature",
    "value": {"byte": 0, "bytelength": 2, "type": "integer", "calculation": "val/100"}
}]}

Note that while the value is an integer at first, it becomes a number after the calculation. Make sure the asset profile in AllThingsTalk is set to number.
{ "type": "number", "unit": " °C" }

Comment

Using the keyword comment, you can add comments anywhere in the json.

{
  "comment": "a very basic weather station",
  "sense": [{
    "comment": "temperature in degrees Celcius",
    "asset": "temperature",
    "value": {"byte": 2, "type": "integer"}
  }]
}

* Composite asset

Sometimes an asset needs a more complex structure than just a single value, for example coordinates of a GPS or the x, y and z values of an accelerometer.

{
  "sense": [{
    "asset": "Angle",
    "comment" : "3-axis accelerometer",
    "value": {
      "x": {"byte": 0, "type": "integer"},
      "y": {"byte": 1, "type": "integer"},
      "z": {"byte": 2, "type": "integer"}
    }
  }]
}

The Asset Profile corresponding this data type in AllThingsTalk would look like

{
  "type": "object",
  "properties": {
    "x": {"type": "integer"},
    "y": {"type": "integer"},
    "z": {"type": "integer"}
  }
}

Off-the-shelf devices

Below you will find a few examples of off-the-shelf devices and how to use the ABCL to convert their binary data payloads. For each of them, the needed assets in Maker and payload definition are supplied.

Always make sure the assets names you use in Maker match the corresponding asset names in your Payload definition.

SMOVE

SMOVE is a ready to use LoRaWAN accelerometer for remote sensing of motions, vibrations and tilt. The compact enclosure holds all electronics including batteries and embedded antenna. The design has been optimized for Long range, Long battery life, and reliability. The device can be used in harsh indoor and outdoor environments since it has an extended temperature range and furthermore is dust and water proof. Pre-programmed user requested Lora-keys, and sensitivity settings make this device Plug and Play.

Assets

Conversion

{"sense": [
    {"asset": "force", "value": {"byte": 0, "type": "integer"}},
    {"asset": "battery", "value": {"byte": 1, "type": "integer"}}
]}

Example payload

MCS1608

The ITalks MCS 1608 Full LoRa Sensor is designed to start quickly with (Private) Lora applications. This complete Lora modem in an IP67 enclosure, so it can also be used outside, already has numerous sensors, so it can be rapidly deployed to test the desired IoT application. On the basis of the successful testing can after that the appropriate sensor can be chosen for u. The nodes are programmed by us at cost or you can do this yourself if you MCS1608 programming cable ordering it. The device is supplied as standard ABP.

The MCS1608 has a large set of operating states. Below we have implemented the payload definition for the General Sensing setting. The state the device is operating in, is provided by the first byte of the payload, hence the switch/case on byte 0 in our payload definition.

Assets

Conversion

{
  "sense": [{
    "switch": {"byte": 0},
    "on": [
      {"case": 0, "do": [ {"asset": "msgid", "value": "Alive"} ] },
      {"case": 1, "do": [ {"asset": "msgid", "value": "Tracking"} ] },
      {"case": 2, "do": [
        {"asset": "msgid", "value": "GenSens"},
        {"asset": "BaromBar", "value": {"byte": 2, "bytelength": 2, "type": "integer", "calculation": "(val + 100000) / 100"}},
        {"asset": "Temperature", "value": {"byte": 4, "bytelength": 2, "type": "integer", "calculation": "val / 100"}},
        {"asset": "Humidity", "value": {"byte": 6, "type": "integer"}},
        {"asset": "Angle",
          "comment" : "approximate calculation from [-128,127] to [-90,90]",
          "value": {
            "x": {"byte": 7,"type": "integer", "signed": true, "calculation": "val/255*180"},
            "y": {"byte": 8,"type": "integer", "signed": true, "calculation": "val/255*180"},
            "z": {"byte": 9,"type": "integer", "signed": true, "calculation": "val/255*180"}
          }
        },
        {"asset": "Vibration",
          "value": {
            "amplitude": {"byte": 10, "type": "integer"},
            "frequency": {"byte": 11, "type": "integer"}
          }
        }
      ]},
      {"case": 3, "do": [ {"asset": "msgid", "value": "Rot"} ] },
      {"case": 4, "do": [ {"asset": "msgid", "value": "Alarm"} ] },
      {"case": 6, "do": [ {"asset": "msgid", "value": "1WireT"} ] },
      {"case": 7, "do": [ {"asset": "msgid", "value": "Running"} ] },
      {"case": 8, "do": [ {"asset": "msgid", "value": "Vibrate"} ] },
      {"case": 9, "do": [ {"asset": "msgid", "value": "Analog"} ] },
      {"case": 10, "do": [ {"asset": "msgid", "value": "GenSensGravMsg"} ] },
      {"case": 11, "do": [ {"asset": "msgid", "value": "DailyReport"} ] },
      {"case": 14, "do": [ {"asset": "msgid", "value": "Reboot"} ] }
    ]}
  ]
}

Example payload

No timestamp is sent in the message so the _at_ field is always empty. The timestamp will be set in the cloud when the message arrives.

'msgid': {'value': 'GenSens', 'at': None}
'BaromBar': {'value': 1005.67, 'at': None}
'Temperature': {'value': 20.47, 'at': None}
'Humidity': {'value': 43, 'at': None}
'Angle': {'value': {'x': 0, 'y': 0, 'z': -4}, 'at': None}
'Vibration': {'value': {'amplitude': 0, 'frequency': 0}, 'at': None}

Allora Push button

Push is a low-power, long range, wireless push sensor is the ideal product to let customers interact with you. With a simple push on the button you know when the coffee machine needs a refilling, the copier is out paper or support is required at a certain location inside your store.

Assets

Conversion

{
  "sense": [{
    "switch": {"byte": 2},
    "on": [
      {"case": 10, "comment": "case 0A one short push", "do": [
        {"asset": "push", "value": "Short"},
        {"asset": "count", "value": {"byte": 3, "bytelength":2, "type": "integer"}}
      ]},
      {"case": 11, "comment": "case 0B one long push", "do": [
        {"asset": "push", "value": "Long"},
        {"asset": "count", "value": {"byte": 3, "bytelength":2, "type": "integer"}}
      ]}
    ]},
    {"asset": "length", "value": {"byte": 0, "type": "integer"}}
  ]
}

Example payload

AllThingsTalk LPWAN Kits

In partnership with Proximus Enco, we have developed a containerised payload format. A container describes a single sensor type and the data it measures.
It provides an easy way to upload sensor data over the Proximus Public LoRaWAN network.

The containerised payload format is fully implemented in our Arduino LoRAWAN SDK

When manually creating an asset in Maker, make sure you use the Asset name (the numbers in the table below), corresponding with the data type you need.

• Definition - container name in the library code, for example #define BINARY_TILT_SENSOR ((short)2)
• Asset name - asset name in AllThingsTalk Maker
• Data type - the primitive data type in the code
• Asset profile - the corresponding asset profile in AllThingsTalk Maker

Generic containers

Fixed containers

Conversion

{
  "name": "containers",
  "sense": [
    {"switch": {"byte": 4}, "on": [
      {"case": 1, "comment": "Binary sensor", "do": [
        {"asset": "1", "value": {"byte": 5, "type": "boolean"}}]},
      {"case": 2, "comment": "Tilt sensor", "do": [
        {"asset": "2", "value": {"byte": 5, "type": "boolean"}}]},
      {"case": 3, "comment": "Push button", "do": [
        {"asset": "3", "value": {"byte": 5, "type": "boolean"}}]},
      {"case": 4, "comment": "Door sensor", "do": [
        {"asset": "4", "value": {"byte": 5, "type": "boolean"}}]},
      {"case": 5, "comment": "Temp sensor", "do": [
        {"asset": "5", "value": {"byte": 8, "bytelength":4, "type": "number"}}]},
      {"case": 6, "comment": "Light sensor", "do": [
        {"asset": "6", "value": {"byte": 8, "bytelength":4, "type": "number"}}]},
      {"case": 7, "comment": "Temp sensor", "do": [
        {"asset": "7", "value": {"byte": 5, "type": "boolean"}}]},
      {"case": 8, "comment": "Accelerometer", "do": [
        {"asset": "8", "value": {
          "x":  {"byte":  8, "bytelength": 4, "type": "number"},
          "y":  {"byte": 12, "bytelength": 4, "type": "number"},
          "z":  {"byte": 16, "bytelength": 4, "type": "number"}}}]},
      {"case": 9, "comment": "GPS", "do": [
        {"asset": "9", "value": {
          "latitude":  {"byte": 8, "bytelength": 4, "type": "number"},
          "longitude":  {"byte": 12, "bytelength": 4, "type": "number"},
          "altitude":  {"byte": 16, "bytelength": 4, "type": "number"}}}]},
      {"case": 10, "comment": "Pressure sensor", "do": [
        {"asset": "10", "value": {"byte": 8, "bytelength": 4, "type": "number"}}]},
      {"case": 11, "comment": "Humidity sensor", "do": [
        {"asset": "11", "value": {"byte": 8, "bytelength": 4, "type": "number"}}]},
      {"case": 12, "comment": "Loudness sensor", "do": [
        {"asset": "12", "value": {"byte": 8, "bytelength": 4, "type": "number"}}]},
      {"case": 13, "comment": "Air quality sensor", "do": [
        {"asset": "13", "value": {"byte": 7, "bytelength": 2, "type": "integer", "byteorder": "little"}}]},
      {"case": 14, "comment": "Battery sensor", "do": [
        {"asset": "14", "value": {"byte": 7, "bytelength": 2, "type": "integer", "byteorder": "little"}}]},
      {"case": 15, "comment": "Integer sensor", "do": [
        {"asset": "15", "value": {"byte": 7, "bytelength": 2, "type": "integer", "byteorder": "little"}}]},
      {"case": 16, "comment": "Number sensor", "do": [
        {"asset": "16", "value": {"byte": 8, "bytelength": 4, "type": "number"}}]}
    ]}
  ]
}

Helpful tools

A few handy online tools that can help with setting up your own conversion

• Hex to Decimal e.g. 58D54158 to 1490370904
• UNIX epoch to readable DateTime e.g. 1490370904 to Fri, 24 Mar 2017 15:55:04 GMT