|Project LoRaWAN dust Sensor|
|LoRaWAN airborne particulate matter sensor|
The concept consists of:
- a sensor that measures airborne particulate matter and sends the measurement data using LoRa to TheThingsNetwork.
- a forwarder application that collects the data from TTN and forwards it to luftdaten.info, opensensemap, mycayenne dashboard, etc.
This has been done before by other people, but it appears there is no universal solution. I am publishing all source code on github and will put up documentation on this wiki. This concept uses Cayenne because it is the closest practical thing towards a universal but still reasonably compact format.
A similar thing has been done by:
- TTN Ulm, see https://github.com/verschwoerhaus/ttn-ulm-feinstaub (the sensor code) and https://github.com/verschwoerhaus/ttn-ulm-muecke (the forwarder, in python)
One thing in particular that my concept does better than existing solutions is to use proper OTAA for the LoRa connection to TTN. OTAA means over-the-air-activation and is a mechanism to dynamically negotiate communication/encryption keys instead of having to hard-code each node with individual keys. Once the OTAA is done successfully, the node remembers the network id, device address, session keys, etc for future communication, as per TTN recommendations.
This makes it possible to have a *single* firmware image for all sensor nodes and it simplifies the setup:
- you flash the node with a unified firmware
- the node shows its unique Device EUI on the OLED
- at the TTN console, you register the node with the unique Device EUI
- the sensor node receives encryption keys over the air automatically
(idea: an ESP32 has a wifi connection too, perhaps registering the node can be done fully automatically, over wifi/internet)
- disable screen when idle, to avoid burn-in of the OLED?
- support for other meteo sensors?
- internal web server, for configuration, information?
Useful links for the TTGO LoRa board:
- Example code that joins TTN by OTAA and saves the OTAA parameters
The node is based on Arduino, in particular a TTGO ESP32 board ("LORA v1" I think) with onboard SX1276 LoRa chip. The sensor is an SDS-011, just like in the luftdaten project. For humidity/temperature, I am using a BME280 (superior to the DHT11/22).
|5V||SDS011 5V||pin 3|
|GND||SDS011 GND||pin 5|
|GPIO25||SDS011 RXD||pin 6|
|GPIO35||SDS011 TXD||pin 7|
- NOTE: https://www.thethingsnetwork.org/community/berlin/post/warning-attention-users-of-ttgo21-v16-boards-labeled-t3_v16-on-pcb-battery-exploded-and-got-on-fire
Source code is hosted on github:
- Arduino node, written in C/Arduino, built using platformio. This firmware joins TTN by OTAA and sends the measurement data using Cayenne.
- TTN-to-luftdaten forwarder, written in Java, built using gradle. This picks up the Cayenne encoded data and forwards it to the Luftdaten API.
It's reasonably compact, it's a standard format, you can get a preview of the data in the TTN console. Interacts nicely with other platforms.
Over-the-air payload encoding:
- PM1.0: digital input (type 2), channel id 0, with value in units of 0.01 ug/m3, saturated to 327.67 ug/m3 (optional)
- PM10: digital input (type 2), channel id 1, with value in units of 0.01 ug/m3, saturated to 327.67 ug/m3
- PM2.5: digital input (type 2), channel id 2, with value in units of 0.01 ug/m3, saturated to 327.67 ug/m3
- Temperature: temperature (type 103), with value in units of 0.1 degrees celcius
- Humidity: humidity (type 104), with value in units of 0.5 %
- Pressure: barometer (type 115), with value in units of 0.1 mbar, or 10 Pa (optional)
Dust values higher than 327.67 are encoded as 327.67, this is the maximum that can be represented as analog value in Cayenne. A nice thing about Cayenne is that you can simply leave items out if you don't support them, which results in a shorter yet still valid message.
0x01 0x01 0xXX 0xXX 0x02 0x01 0xYY 0xYY 0x03 0x67 0xTT 0xTT 0x04 0x68 0xHH <== PM10 value ===> <== PM2.5 value ==> <== temperature ==> <= humidity =>
Total payload size is 15 bytes. The LoRaWAN header adds 13 bytes (at least).
Particulate matter values encoded are averaged over the measurement interval.
How other projects encode the data:
- TTN Apeldoorn (?): https://github.com/tijnonlijn/RFM-node/blob/master/dustduino_PPD42NS_example.ino#L327 sends 5 bytes
- 1 byte : 0x04
- 2 bytes: PM25(?) big endian
- 2 bytes: PM10(?) big endian
- TTN Ulm: https://github.com/verschwoerhaus/ttn-ulm-feinstaub/blob/master/ttnulmdust/ttnulmdust.ino#L225 sends 8 bytes:
- 2 bytes: PM10, big endian (unit 0.01 ug/m3)
- 2 bytes: PM2.5, big endian (unit 0.01 ug/m3)
- 2 bytes: humidity (unit of 0.01%)
- 2 bytes: temperature (unit of 0.01 degree Celcius)
- RIVM node, sends 20 bytes
- 1 byte temperature (unit deg Celcius ?)
- 1 byte relative humidity (unit % ?)
- 2 bytes pressure (unit?)
- 2 bytes pm10 (unit?)
- 2 bytes pm25 (unit?)
- 2 bytes op1 (unit?)
- 2 bytes op2 (unit?)
- 4 bytes latitude (unit?)
- 4 bytes longitude (unit?)
- Apeldoorn in data: https://github.com/nijmeijer/TTN_Apeldoorn_in_Data_2018/blob/master/AiD_Dust_2018/AiD_Dust_2018.ino#L184 sends 16 bytes:
- 4 bytes: pm2_5 float big endian (unit?)
- 4 bytes: pm10 float big endian (unit?)
- 4 bytes: humidity float big endian (unit?)
- 4 bytes: temperature float big endian (unit?)
A smaller payload means less time in the air, smaller chance of collision with other LoRaWAN packets and more packets per hour. However, there is always an overhead from the LoRaWAN package (minimum 13 bytes), so using the smallest encoding (5 bytes) compared to the largest (16 bytes), reduces the on-air-time by only 23%.
LoraWan time budget
Consider the following:
- radio regulations generally have a 1% duty cycle requirement for the two bands used by the 8 LoRaWAN (EU) frequencies, so according to the legal limits, there is about 86400x0.01x2 = 1728 seconds per day send time at the best case.
- TheThingsNetwork has a FUP of 30s of data upload per day, actually a huge restriction compared to the send time allowed purely by radio regulations.
- The Luftdaten backend appears to run on a 5 minute interval, or 288 measurements per day.
- The default Luftdaten firmware sends new data every 145s by default
With the TTN FUP of 30s upload per day, we can spend 30s / 288 = 104 ms on each transmission.
Using the LoRaWAN airtime calculator we can determine which modes can be used. At 15 bytes payload, this is only possible at SF7, the highest LoRa speed. Stretching the TTN guideline a bit, say by a factor 2, we can achieve those transfers still only at SF7 and SF8. So you need to be relatively close to a gateway.
With a smaller payload, can we use higher spreading factors? :
- skip T/RH/P completely (8 bytes left): SF7 is possible, at SF8 we spend 102 ms per transmission
- skip P only (15 bytes left): SF7 is possible, at SF8 we spend 123 ms per transmission
So, in conclusion: With the FUP of TTN and use of Cayenne encoding, you can just barely send enough data to transport Luftdaten PM data!
Source code for the particulate matter measurement node can be found on the github page.
The luftdaten backend has a 5 minute "heartbeat", so at least one measurement per 5 minutes should be sent to avoid disappearing from the map. The node firmware (attempts to) send a message every 145 seconds, just like the luftdaten WiFi sensor.
Measurements run in a cycle running through the following states:
- INIT: determine presence of the SDS011, print the SDS011 serial number
- IDLE: wait until the start of the cycle, then turn on the fan
- WARMUP: wait 20 seconds while the sensor "warms up"
- MEASURE: measure 10 seconds, then turn off fan, calculate median/average and send LoRaWAN message
Building with platformio
Platformio is used to compile and upload the code to the node.
To install platformio (example for Debian):
sudo apt install python3-pip sudo pip3 install platformio pio update
To compile and upload:
pio run -t upload
TTN key provisioning
The node needs to be registered at TheThingsNetwork, in order for its messages to be accepted by the TTN.
The following scheme is used to make TTN provisioning as simple as possible:
- Each node can be programmed with the *same* software, no source code modification is required
- The node administrator needs to enter the following properties at the TTN console, for each node:
- The Device EUI is derived from the node-specific ESP32 MAC address, the node shows this on its OLED
- The App EUI has a fixed value and is the same for all nodes
- The App Key has a fixed value and is the same for all nodes
- Use 32-bit frame counter, disable frame counter checks
- The node does over-the-air-activation (OTAA) only once and then stores the OTAA parameters in internal (simulated) EEPROM. Upon reboot, the node resumes the connection with these parameters
- A long press on the PRG button restarts the OTAA procedure
This is implemented by my LoraLuftdatenForwarder.
It currently supports the following:
- subscribe to a TTN MQTT stream and receive incoming message
- decode Cayenne and custom payloads
- forward to luftdaten.info/sensor.community
- forward to opensensemap.org
- forward to feinstaub app (experimental), https://pm.mrgames-server.de/
Example, to receive data using mosquitto, separately from the backend:
mosquitto_sub -h eu.thethings.network -p 1883 -t +/devices/+/up -u particulatematter -P ttn-account-v2.cNaB2zO-nRiXaCUYmSAugzm-BaG_ZSHbEc5KgHNQFsk
Example upstream data:
Example downstream data: