Sensor-data-bridge

Project LoraLuftdatenForwarder
LoRaWAN forwarder for particulate matter data
Status	In progress
Contact	bertrik
Last Update	2021-04-12

What is this

This is a companion project of LoraWanDustSensor.

It is a Java application that takes airborne particulate matter measurement data transferred through TheThingsNetwork and forwards it to http://sensor.community (formerly Luftdaten) and http://opensensemap.org .

Features

• Picks up particulate matter measurement data received through TheThingsNetwork, using their "v3" infrastructue
• Forwards measurement data to sensor.community
• Forwards measurement data to opensensemap.org, you can configure the opensense-id by adding a device attribute in TheThingsNetwork console
• Forwards measurement data to cayenne.mydevices.com, you configure the username/password/clientid by adding a device attribute in TheThingsNetwork console
• Supports Cayenne payload format for the data encoding
• Handles particulate matter data (PM10, PM4.0, PM2.5, PM1.0), temperature, humidity, barometric pressure
• Can be run as a systemd service, so it automatically restarts in case the software would crash

Next steps

• add support for direct pushing to cayenne.mydevices.com API description at https://developers.mydevices.com/cayenne/docs/cayenne-mqtt-api/#cayenne-mqtt-api-overview-using-mqtt-with-cayenne-option-3-use-http-to-push-mqtt-data
• add support for NB-IOT modem with t-mobile backend, see my Sim7020 project
• add support for other backends, e.g. feinstaub-app?

Requirements

You need the following:

• a server that is always on and connected to the internet, can be Linux or Windows
• a Java installation (JDK to compile), at least version 8
• some configuration on TheThingsNetwork side
• some configuration of my application (YAML file)

Compilation

To compile the software:

• clone the software from my github archive

 git clone https://github.com/bertrik/LoraLuftdatenForwarder.git

• enter the LoraLuftdatenForwarder/gradle directory

 cd LoraLuftdatenForwarder/gradle

• run the gradle script to build the software:

 ./gradlew assemble

• the application zip & tar is now available in LoraLuftdatenForwarder/LoraLuftdatenForwarder/build/distributions

To update to the latest version:

• Update software from github archive:

 git pull

• perform the last two steps above again

Installation

Unzip the distribution file somewhere on your system. I put it in my home directory, for example

 cd
 tar xvf code/LoraLuftdatenForwarder/LoraLuftdatenForwarder/build/distributions/LoraLuftdatenForwarder.tar

Configuration

Node configuration

The particulate matter measurement device needs to send data in the Cayenne format. I used the following conventions:

• PM10 is encoded as analog value on channel 1
• PM2.5 is encoded as analog value on channel 2
• PM1.0 is encoded as analog value on channel 0 (optional)
• PM4.0 is encoded as analog value on channel 4 (optional)
• Temperature is encoded using standard Cayenne encoding (optional)
• Humidity is encoded using standard Cayenne encoding (optional)
• Barometric pressure is encoded using standard Cayenne encoding (optional)

TheThingsNetwork application/device configuration

TTN API key rights

You need to define an 'application' on TheTheThingsNetwork.

• Go the TTN console: https://console.cloud.thethings.network/ and log in
• You need an 'application', create a new one, or use an existing one
• Within the application you need a 'device', so create a new one, or use an existing one:
  • Use OTAA, LoRaMac version probably 1.0.3
  • Enter the device EUI as displayed on the display
  • Use the application keys as specified in my LoraWanDustSensor page
• You need an API key
  • Create this on the TTN console, grant individual rights as shown in the screenshot
  • NOTE: you have only one chance to copy this key somewhere, so copy/paste it locally to a text file or something

LoraLuftdatenForwarder configuration

To configure the application:

• Start the application without a configuration file, this will create a default template, stop the application again

 cd LoraLuftdatenForwarder
 bin/LoraLuftdatenForwarder
 (ctrl-C)

• Edit the loraluftdatenforwarder.yaml file, example:

---
ttn:
  mqtt_url: "tcp://eu1.cloud.thethings.network"
  identity_server_url: "https://eu1.cloud.thethings.network"
  identity_server_timeout: 20
  apps:
  - name: "particulatematter"
    key: "NNSXS......."
    encoding: "CAYENNE"
luftdaten:
  url: "https://api.sensor.community"
  timeout: 20
opensense:
  url: "https://api.opensensemap.org"
  timeout: 20

So:

• enter the name of your application
• enter the TTN API key you saved earlier
• other defaults are probably OK

Sensor.community

TODO

• Go to https://devices.sensor.community/ and log in
• Register a node with id 'TTN-<device-EUI-as-shown-on-display>' (without the spaces or hyphens, e.g. 'TTN-0000547AF1BF713C')
• Register it with the proper configuration, e.g. SDS011 with BME280

Opensensemap

• Go to opensensemap.org and log in
  • Create an opensense node with the proper configuration
  • Copy the opensensenmap 'box id', a long hexadecimal string
• Go the TTN console: https://console.cloud.thethings.network/ and log in
  • Add an attribute for the device, under 'General settings', name = 'opensense-id', value = boxid that you copied from opensensemap.org
• The mapping from TTN-id to boxid is refreshed by the forwarder once an hour, so within an hour the forwarding to opensensemap.org starts

Work in progress

Forwarding noise data

How it is encoded in the sensor.community firmware:

• Three values are sent in the JSON to sensor.community:
  • "noise_LAeq", value in dB(A), meaning?
  • "noise_LA_min", value in dB(A), some kind of minimum
  • "noise_LA_max", value in dB(A), some kind of maximum

I think these can be encoded in Cayenne as a simple analog value (which has a range of approximately -327..327 with a resolution of 0.01. Just need to assign a channel number to it.

Values are read from the noise sensor as follows:

• call to dnms_calculate_leq()
• call to dnms_read_data_ready(&data_ready) returns 0 if OK and (data_ready != 0)
• call to dnms_read_leq(&dnms_values) returns 0 if OK
• firmware applies a "correction" by adding a fixed offset

Measurement values are encoded as 32-bit units, interpreted as 32-bit floats.

References:

• DNMS is read at https://github.com/opendata-stuttgart/sensors-software/blob/beta/airrohr-firmware/airrohr-firmware.ino#L3392
• DNMS is formatted in the JSON at https://github.com/opendata-stuttgart/sensors-software/blob/beta/airrohr-firmware/airrohr-firmware.ino#L3405
• DNMS I2C code is at https://github.com/opendata-stuttgart/sensors-software/blob/beta/airrohr-firmware/dnms_i2c.cpp