Difference between revisions of "LoraWanDustSensor"

From RevSpace
Jump to navigation Jump to search
m (Binary)
(Pinout)
(42 intermediate revisions by the same user not shown)
Line 7: Line 7:
 
   }}
 
   }}
  
== The idea ==
+
== The concept ==
The idea is to create a system consisting of:
+
The concept consists of:
 
* a sensor that measures airborne particulate matter and sends the measurement data using LoRa to TheThingsNetwork.
 
* a sensor that measures airborne particulate matter and sends the measurement data using LoRa to TheThingsNetwork.
* a forwarder application that collects the data from TTN and forwards it to luftdaten.info
+
* a forwarder application that collects the data from TTN and forwards it to luftdaten.info, opensensemap, mycayenne dashboard, etc.
  
 
This has been done before by other people, but it appears there is no universal solution.
 
This has been done before by other people, but it appears there is no universal solution.
 
I am publishing all source code on github and will put up documentation on this wiki.
 
I am publishing all source code on github and will put up documentation on this wiki.
Everyone invents their own payload format, something more universal like Cayenne LPP would be nice.
+
This concept uses Cayenne because it is the closest practical thing towards a universal but still reasonably compact format.
However I could not find a way to encode particulate matter data using Cayenne, so I'll just invent my own payload format too.
 
  
 
A similar thing has been done by:
 
A similar thing has been done by:
Line 25: Line 24:
 
* (others...)
 
* (others...)
  
One thing in particular I'd like to do better than existing solutions is to use proper OTAA for the LoRa connection to TTN.
+
One thing in particular that my concept does better than existing solutions is to use proper OTAA for the LoRa connection to TTN.
OTAA means over-the-air-activation and is a mechanism to dynamically negotiate communication/encryption keys instead of programmed specifically in each sensor node.
+
OTAA means over-the-air-activation and is a mechanism to dynamically negotiate communication/encryption keys instead of having to hard-code each node with individual keys.
Once the OTAA is done successfully, the node remembers the network id, device address, session keys, etc for future communication.
+
Once the OTAA is done successfully, the node remembers the network id, device address, session keys, etc for future communication, as per TTN recommendations.
  
This makes it possible to have a single firmware image for all sensor nodes and it simplifies the setup:
+
This makes it possible to have a *single* firmware image for all sensor nodes and it simplifies the setup:
 
* you flash the node with a unified firmware
 
* you flash the node with a unified firmware
* the node shows its unique id on the OLED
+
* the node shows its unique Device EUI on the OLED
* at the TTN console, you register a new device with the unique id
+
* at the TTN console, you register the node with the unique Device EUI
 
* the sensor node receives encryption keys over the air automatically
 
* the sensor node receives encryption keys over the air automatically
 
* done!
 
* done!
Line 39: Line 38:
  
 
=== Next steps ===
 
=== Next steps ===
* Are we bound by a 5 minute interval towards Luftdaten (dropping off the maps if sending less frequently)?
+
* update pin description to support other ESP32 LoRa boards, like the heltec lora32 v2
* Implement a kind of schedule: turn the sds011 on, wait some time, take a measurement, turn it off, wait for some time
+
* update build system to support other ESP32 LoRa boards, like the heltec lora32 v2, also support arduino IDE (defines etc.)
** Luftdaten uses default send schedule of 145 seconds
+
* disable screen when idle, to avoid burn-in of the OLED?
** make this match the TTN send schedule? it's useless to do a measurement if we don't have TTN airtime to transmit it
+
* support for other meteo sensors?
** use the built-in on-off schedule of the sds011
+
* internal web server, for configuration, information?
** implement calculating the median of accumulated measurements (half of the measurements is higher, other half is lower)
 
* encode using Cayenne encoding, advantage is that it's more or less standard, and you can more easily interpret the data in the TTN console
 
* display SDS011 serial number and date code on the screen
 
  
 
=== Links ===
 
=== Links ===
Line 57: Line 53:
 
* https://github.com/CivicLabsBelgium/lora_particle_sensor
 
* https://github.com/CivicLabsBelgium/lora_particle_sensor
 
* https://github.com/Cinezaster/ttn2luftdaten_forwarder
 
* https://github.com/Cinezaster/ttn2luftdaten_forwarder
 +
* https://jackgruber.github.io/2020-04-13-ESP32-DeepSleep-and-LoraWAN-OTAA-join/
  
 
== Hardware ==
 
== Hardware ==
The node is based on Arduino, in particular a TTGO ESP32 board with onboard SX1276 LoRa chip.
+
The node is based on Arduino, in particular a TTGO ESP32 board ("ttgo-lora32-v1") with onboard SX1276 LoRa chip.
 
The sensor is an SDS-011, just like in the luftdaten project.
 
The sensor is an SDS-011, just like in the luftdaten project.
For humidity/temperature, I'd like to use a BME280.
+
For humidity/temperature, I am using a BME280 (superior to the DHT11/22).
  
NOTE: https://www.thethingsnetwork.org/community/berlin/post/warning-attention-users-of-ttgo21-v16-boards-labeled-t3_v16-on-pcb-battery-exploded-and-got-on-fire
+
TODO: plaatjes van de hardware
  
[https://primalcortex.wordpress.com/tag/lora32/ Page with correct pinout of the ESP32 LoRa board].
+
=== Pinout ===
 +
{| class="wikitable"
 +
! TTGO LoRa v1 !! Heltec LoRa v2 !! Sensor !! Remark
 +
|-
 +
| 5V || 5V || SDS011 5V (pin 3) || triple-check this, swapping 5V/GND destroys the SDS011
 +
|-
 +
| GND || GND || SDS011 GND (pin 5) || triple-check this, swapping 5V/GND destroys the SDS011
 +
|-
 +
| GPIO25 || - || SDS011 RXD (pin 6) || to be moved to GPIO23
 +
|-
 +
| GPIO35 || - || SDS011 TXD (pin 7) || to be moved to GPIO22
 +
|-
 +
| 3.3V || 3.3V/Vext || BME280 3V || Both Vext and 3.3V can be used
 +
|-
 +
| GPIO4 || GPIO4 || BME280 SDA || data
 +
|-
 +
| GPIO15 || GPIO15 || BME280 SCL || data
 +
|-
 +
| GND || GND || BME280 GND || ground
 +
|}
  
Luftdaten uses a cycle time of 145 seconds for the SDS011.
+
For reference:
 +
* https://randomnerdtutorials.com/esp32-pinout-reference-gpios/
  
Proposed hardware connections:
+
=== TTGO LoRa v1 ===
* SDS011 5V to ESP32 5V
+
Technical documents:
* SDS011 GND to ESP32 GND
+
* [https://primalcortex.wordpress.com/tag/lora32/ Pinout of the ESP32 TTGO  LoRa v1 board].
* SDS011 TXD to ESP32 GPIO35 (maybe I can find two suitable pins close together)
+
* NOTE: https://www.thethingsnetwork.org/community/berlin/post/warning-attention-users-of-ttgo21-v16-boards-labeled-t3_v16-on-pcb-battery-exploded-and-got-on-fire
* SDS011 RXD to ESP32 GPIO25 (maybe I can find two suitable pins close together)
+
 
* BME280 3V todo
+
=== Heltec LoRa v2 ===
* BME280 SDA todo
+
Technical documents:
* BME280 SCL todo
+
* [https://resource.heltec.cn/download/WiFi_LoRa_32/WIFI_LoRa_32_V2.pdf pinout]
* BME280 GND todo
 
  
 
== Software ==
 
== Software ==
 
=== Source code ===
 
=== Source code ===
 
Source code is hosted on github:
 
Source code is hosted on github:
* [https://github.com/bertrik/LoraWanPmSensor Arduino node], written in C/Arduino, built using platformio
+
* [https://github.com/bertrik/LoraWanPmSensor Arduino node], written in C/Arduino, built using platformio. This firmware joins TTN by OTAA and sends the measurement data using Cayenne.
* [https://github.com/bertrik/LoraLuftdatenForwarder TTN-to-luftdaten forwarder], written in Java, built using gradle
+
* [https://github.com/bertrik/LoraLuftdatenForwarder TTN-to-luftdaten forwarder], written in Java, built using gradle. This picks up the Cayenne encoded data and forwards it to the Luftdaten API.
  
=== Common ===
+
=== Payload encoding ===
==== Packet format ====
 
 
 
New idea: encode everything in Cayenne.
 
  
 
===== Cayenne =====
 
===== Cayenne =====
Line 95: Line 108:
 
[https://community.mydevices.com/t/cayenne-lpp-2-0/7510 Specification for Cayenne LPP 2.0]
 
[https://community.mydevices.com/t/cayenne-lpp-2-0/7510 Specification for Cayenne LPP 2.0]
  
HOWEVER:
+
Over-the-air payload encoding:
* A standard analog value has a resolution of 0.01 units and is signed, leaving a maximum of 327.67 ug/m3. This value is possibly exceeded in extreme cases (night of new year's day).
 
* Still not as compact as direct binary encoding
 
 
 
So proposed format:
 
 
* PM1.0: digital input (type 2), channel id 0, with value in units of 0.01 ug/m3, saturated to 327.67 ug/m3 (optional)
 
* PM1.0: digital input (type 2), channel id 0, with value in units of 0.01 ug/m3, saturated to 327.67 ug/m3 (optional)
 
* PM10: digital input (type 2), channel id 1, with value in units of 0.01 ug/m3, saturated to 327.67 ug/m3  
 
* PM10: digital input (type 2), channel id 1, with value in units of 0.01 ug/m3, saturated to 327.67 ug/m3  
 
* PM2.5: digital input (type 2), channel id 2, with value in units of 0.01 ug/m3, saturated to 327.67 ug/m3
 
* PM2.5: digital input (type 2), channel id 2, with value in units of 0.01 ug/m3, saturated to 327.67 ug/m3
* Temperature: temperature (type 103), channel 3, with value in units of 0.1 degrees celcius
+
* Temperature: temperature (type 103), with value in units of 0.1 degrees celcius
* Humidity: humidity (type 104), channel 4, with value in units of 0.5 %
+
* Humidity: humidity (type 104), with value in units of 0.5 %
 +
* Pressure: barometer (type 115), with value in units of 0.1 mbar, or 10 Pa (optional)
 +
 
 +
Dust values higher than 327.67 are encoded as 327.67, this is the maximum that can be represented as analog value in Cayenne.
 +
A nice thing about Cayenne is that you can simply leave items out if you don't support them, which results in a shorter yet still valid message.
  
 
Example:
 
Example:
Line 110: Line 123:
 
   <== PM10 value ===>  <== PM2.5 value ==>  <== temperature ==>  <= humidity =>
 
   <== PM10 value ===>  <== PM2.5 value ==>  <== temperature ==>  <= humidity =>
  
Total payload size is 15 bytes. Over the air, LoRaWAN header adds some more data (13 bytes at least).  
+
<b>Total payload size is 15 bytes.</b> The <b>LoRaWAN header adds 13 bytes</b> (at least).  
  
Values encoded are averaged using the median over the accumulated measurements over the measurement interval.
+
Particulate matter values encoded are averaged over the measurement interval.
  
 
===== Binary =====
 
===== Binary =====
Line 141: Line 154:
 
** 4 bytes: temperature float big endian (unit?)
 
** 4 bytes: temperature float big endian (unit?)
  
A smaller payload means less time in the air, smaller chance of collision with other LoRa packets and more packets per hour.
+
A smaller payload means less time in the air, smaller chance of collision with other LoRaWAN packets and more packets per hour.
 +
However, there is always an overhead from the LoRaWAN package (minimum 13 bytes), so using the smallest encoding (5 bytes) compared to the largest (16 bytes), reduces the on-air-time by only 23%.
  
 
=== LoraWan time budget ===
 
=== LoraWan time budget ===
 
Consider the following:
 
Consider the following:
* radio regulations generally have a 0.1% duty cycle requirement for each of the 8 frequencies, so according to the legal limits, there is about 86400x0.001x8 = 691s send time at the best case.
+
* radio regulations generally have a 1% duty cycle requirement for the two bands used by the 8 LoRaWAN (EU) frequencies, so according to the legal limits, there is about 86400x0.01x2 = 1728 seconds per day send time at the best case.
* TheThingsNetwork has a [https://www.thethingsnetwork.org/forum/t/limitations-data-rate-packet-size-30-seconds-uplink-and-10-messages-downlink-per-day-fair-access-policy-guidelines/1300 FUP of 30s of data upload per day]
+
* TheThingsNetwork has a [https://www.thethingsnetwork.org/forum/t/limitations-data-rate-packet-size-30-seconds-uplink-and-10-messages-downlink-per-day-fair-access-policy-guidelines/1300 FUP of 30s of data upload per day], actually a huge restriction compared to the send time allowed purely by radio regulations.
 
* The Luftdaten backend appears to run on a 5 minute interval, or 288 measurements per day.
 
* The Luftdaten backend appears to run on a 5 minute interval, or 288 measurements per day.
 
* The default Luftdaten firmware sends new data every 145s by default
 
* The default Luftdaten firmware sends new data every 145s by default
Line 153: Line 167:
  
 
Using the [https://www.thethingsnetwork.org/airtime-calculator LoRaWAN airtime calculator] we can determine which modes can be used.
 
Using the [https://www.thethingsnetwork.org/airtime-calculator LoRaWAN airtime calculator] we can determine which modes can be used.
So with 288 measurements, we can spend 30s / 288 = 104 ms per packet.
+
At 15 bytes payload, this is only possible at SF7, the highest LoRa speed.
At 19 bytes payload, this is only possible at SF7, the highest LoRa speed.
+
Stretching the TTN guideline a bit, say by a factor 2, we can achieve those transfers still only at SF7 and SF8. So you need to be relatively close to a gateway.
Stretching the TTN guideline a bit, say by a factor 2, we can achieve 288 transfers of 19 bytes each at SF7 and SF8. So you need to be relatively close to a gateway.
 
  
 
With a smaller payload, can we use higher spreading factors? :
 
With a smaller payload, can we use higher spreading factors? :
Line 161: Line 174:
 
* skip P only (15 bytes left): SF7 is possible, at SF8 we spend 123 ms per transmission
 
* skip P only (15 bytes left): SF7 is possible, at SF8 we spend 123 ms per transmission
  
So, in conclusion: <s>With the FUP of TTN and use of Cayenne encoding, you can just barely send enough data to transport Luftdaten PM data!</s>
+
So, in conclusion: <b>With the FUP of TTN and use of Cayenne encoding, you can just barely send enough data to transport Luftdaten PM data!</b>
  
=== Node ===
+
=== Node design ===
 
Source code for the particulate matter measurement node can be found [https://github.com/bertrik/LoraWanPmSensor on the github page].
 
Source code for the particulate matter measurement node can be found [https://github.com/bertrik/LoraWanPmSensor on the github page].
  
==== Platformio ====
+
The luftdaten backend has a 5 minute "heartbeat", so at least one measurement per 5 minutes should be sent to avoid disappearing from the map.
To compile and upload the code to the node, platformio is used.
+
The node firmware (attempts to) send a message every 145 seconds, just like the luftdaten WiFi sensor.
 +
 
 +
Measurements run in a cycle running through the following states:
 +
* INIT: determine presence of the SDS011, print the SDS011 serial number
 +
* IDLE: wait until the start of the cycle, then turn on the fan
 +
* WARMUP: wait 20 seconds while the sensor "warms up"
 +
* MEASURE: measure 10 seconds, then turn off fan, calculate median/average and send LoRaWAN message
 +
 
 +
==== Building with platformio ====
 +
Platformio is used to compile and upload the code to the node.
  
 
To install platformio (example for Debian):
 
To install platformio (example for Debian):
Line 177: Line 199:
 
   pio run -t upload
 
   pio run -t upload
  
==== Design ====
+
==== TTN key provisioning ====
The function of the node software is to collect data from the SDS011 (particulate matter) and BME280 (temperature/humidity) at regular intervals,
+
The node needs to be registered at TheThingsNetwork, in order for its messages to be accepted by the TTN.
encode this as a data packet and send it over LoRaWAN towards TheThingsNetwork.
 
  
For the LoRaWAN data connection, over-the-air activation (OTAA) is used. I use the following scheme, to keep administration to a minimum:
+
The following scheme is used to make TTN provisioning as simple as possible:
* The Device EUI is derived from the ESP32 MAC address, the node shows this on its OLED
+
* Each node can be programmed with the *same* software, no source code modification is required
* The App EUI is generated in the TTN console, it is the same for all nodes
+
* The node administrator needs to enter the following properties at the TTN console, for each node:
* The App Key is generated in the TTN console, it is the same for all nodes
+
** The Device EUI is derived from the node-specific ESP32 MAC address, the node shows this on its OLED
* The device is registered in the TTN console by the Device EUI (this doesn't happen automatically). Frame counter checks are disabled.
+
** The App EUI has a fixed value and is the same for all nodes
* OTAA is done only once for each node. After that, the OTAA parameters are stored in (simulated) EEPROM.
+
** The App Key has a fixed value and is the same for all nodes
 +
** Use 32-bit frame counter, disable frame counter checks
 +
* The node does over-the-air-activation (OTAA) only once and then stores the OTAA parameters in internal (simulated) EEPROM. Upon reboot, the node resumes the connection with these parameters
 
** A long press on the PRG button restarts the OTAA procedure
 
** A long press on the PRG button restarts the OTAA procedure
* OTAA progress is shown on the OLED
 
* If OTAA has been done successfully, the node restores the session parameters negotiated during OTAA on next bootup, so it can quickly resume sending data.
 
* I'm NOT saving the upload frame counter (this would be preferable), just disable the feature in the TTN console.
 
  
TODO to figure out:
+
=== Backend ===
* What about the channel setup? The node connects using 3 frequencies, but receives a bigger list of frequencies during OTAA JOIN.
+
This is implemented by my [https://github.com/bertrik/LoraLuftdatenForwarder LoraLuftdatenForwarder].
 
 
I've seen the following from the node, receiving an ADR:
 
  40829907: engineUpdate, opmode=0x8
 
  40829935: EV_TXSTART
 
  40829939: engineUpdate, opmode=0x888
 
  40830013: TXMODE, freq=868300000, len=25, SF=11, BW=125, CR=4/5, IH=0
 
  40944876: setupRx1 txrxFlags 0x22 --> 01
 
  start single rx: now-rxtime: 5
 
  40945013: RXMODE_SINGLE, freq=868300000, SF=11, BW=125, CR=4/5, IH=0
 
  rxtimeout: entry: 40951170 rxtime: 40945001 entry-rxtime: 6169 now-entry: 5 rxtime-txend: 63524
 
  41005584: setupRx2 txrxFlags 0x1 --> 02
 
  start single rx: now-rxtime: 4
 
  41005720: RXMODE_SINGLE, freq=869525000, SF=9, BW=125, CR=4/5, IH=0
 
  41017003: process options (olen=0x5)
 
  41017012: LinkAdrReq: p1:11 chmap:00ff chpage:00 uprt:01 ans:86
 
  41017019: ??ack error ack=1 txCnt=0
 
  41017073: decodeFrame txrxFlags 0x2 --> 22
 
  41017312: Received downlink, window=RX2, port=-1, ack=1, txrxFlags=0x22
 
  41017708: EV_TXCOMPLETE (includes waiting for RX windows)
 
  41018027: engineUpdate, opmode=0x800
 
  
=== Backend ===
+
It currently supports the following:
A Java program subscribes to the MQTT stream, decodes the telemetry packets and forwards them to the luftdaten API.
+
* subscribe to a TTN MQTT stream and receive incoming message
There is no storage of measurement data in the Java application.
+
* decode Cayenne and custom payloads
 +
* forward to luftdaten.info/sensor.community
 +
* forward to opensensemap.org
 +
* forward to feinstaub app (experimental), https://pm.mrgames-server.de/
  
To receive data using mosquitto:
+
Example, to receive data using mosquitto, separately from the backend:
 
   mosquitto_sub -h eu.thethings.network -p 1883 -t +/devices/+/up -u particulatematter -P ttn-account-v2.cNaB2zO-nRiXaCUYmSAugzm-BaG_ZSHbEc5KgHNQFsk
 
   mosquitto_sub -h eu.thethings.network -p 1883 -t +/devices/+/up -u particulatematter -P ttn-account-v2.cNaB2zO-nRiXaCUYmSAugzm-BaG_ZSHbEc5KgHNQFsk
  

Revision as of 23:40, 1 July 2020

Project LoRaWAN dust Sensor
LoraWanDustSensor.jpg
LoRaWAN airborne particulate matter sensor
Status In progress
Contact bertrik
Last Update 2020-07-01

The concept

The concept consists of:

  • a sensor that measures airborne particulate matter and sends the measurement data using LoRa to TheThingsNetwork.
  • a forwarder application that collects the data from TTN and forwards it to luftdaten.info, opensensemap, mycayenne dashboard, etc.

This has been done before by other people, but it appears there is no universal solution. I am publishing all source code on github and will put up documentation on this wiki. This concept uses Cayenne because it is the closest practical thing towards a universal but still reasonably compact format.

A similar thing has been done by:

One thing in particular that my concept does better than existing solutions is to use proper OTAA for the LoRa connection to TTN. OTAA means over-the-air-activation and is a mechanism to dynamically negotiate communication/encryption keys instead of having to hard-code each node with individual keys. Once the OTAA is done successfully, the node remembers the network id, device address, session keys, etc for future communication, as per TTN recommendations.

This makes it possible to have a *single* firmware image for all sensor nodes and it simplifies the setup:

  • you flash the node with a unified firmware
  • the node shows its unique Device EUI on the OLED
  • at the TTN console, you register the node with the unique Device EUI
  • the sensor node receives encryption keys over the air automatically
  • done!

(idea: an ESP32 has a wifi connection too, perhaps registering the node can be done fully automatically, over wifi/internet)

Next steps

  • update pin description to support other ESP32 LoRa boards, like the heltec lora32 v2
  • update build system to support other ESP32 LoRa boards, like the heltec lora32 v2, also support arduino IDE (defines etc.)
  • disable screen when idle, to avoid burn-in of the OLED?
  • support for other meteo sensors?
  • internal web server, for configuration, information?

Links

Useful links for the TTGO LoRa board:

Hardware

The node is based on Arduino, in particular a TTGO ESP32 board ("ttgo-lora32-v1") with onboard SX1276 LoRa chip. The sensor is an SDS-011, just like in the luftdaten project. For humidity/temperature, I am using a BME280 (superior to the DHT11/22).

TODO: plaatjes van de hardware

Pinout

TTGO LoRa v1 Heltec LoRa v2 Sensor Remark
5V 5V SDS011 5V (pin 3) triple-check this, swapping 5V/GND destroys the SDS011
GND GND SDS011 GND (pin 5) triple-check this, swapping 5V/GND destroys the SDS011
GPIO25 - SDS011 RXD (pin 6) to be moved to GPIO23
GPIO35 - SDS011 TXD (pin 7) to be moved to GPIO22
3.3V 3.3V/Vext BME280 3V Both Vext and 3.3V can be used
GPIO4 GPIO4 BME280 SDA data
GPIO15 GPIO15 BME280 SCL data
GND GND BME280 GND ground

For reference:

TTGO LoRa v1

Technical documents:

Heltec LoRa v2

Technical documents:

Software

Source code

Source code is hosted on github:

  • Arduino node, written in C/Arduino, built using platformio. This firmware joins TTN by OTAA and sends the measurement data using Cayenne.
  • TTN-to-luftdaten forwarder, written in Java, built using gradle. This picks up the Cayenne encoded data and forwards it to the Luftdaten API.

Payload encoding

Cayenne

It's reasonably compact, it's a standard format, you can get a preview of the data in the TTN console. Interacts nicely with other platforms.

Specification for Cayenne LPP 2.0

Over-the-air payload encoding:

  • PM1.0: digital input (type 2), channel id 0, with value in units of 0.01 ug/m3, saturated to 327.67 ug/m3 (optional)
  • PM10: digital input (type 2), channel id 1, with value in units of 0.01 ug/m3, saturated to 327.67 ug/m3
  • PM2.5: digital input (type 2), channel id 2, with value in units of 0.01 ug/m3, saturated to 327.67 ug/m3
  • Temperature: temperature (type 103), with value in units of 0.1 degrees celcius
  • Humidity: humidity (type 104), with value in units of 0.5 %
  • Pressure: barometer (type 115), with value in units of 0.1 mbar, or 10 Pa (optional)

Dust values higher than 327.67 are encoded as 327.67, this is the maximum that can be represented as analog value in Cayenne. A nice thing about Cayenne is that you can simply leave items out if you don't support them, which results in a shorter yet still valid message.

Example:

 0x01 0x01 0xXX 0xXX  0x02 0x01 0xYY 0xYY  0x03 0x67 0xTT 0xTT  0x04 0x68 0xHH
 <== PM10 value ===>  <== PM2.5 value ==>  <== temperature ==>  <= humidity =>

Total payload size is 15 bytes. The LoRaWAN header adds 13 bytes (at least).

Particulate matter values encoded are averaged over the measurement interval.

Binary

How other projects encode the data:

A smaller payload means less time in the air, smaller chance of collision with other LoRaWAN packets and more packets per hour. However, there is always an overhead from the LoRaWAN package (minimum 13 bytes), so using the smallest encoding (5 bytes) compared to the largest (16 bytes), reduces the on-air-time by only 23%.

LoraWan time budget

Consider the following:

  • radio regulations generally have a 1% duty cycle requirement for the two bands used by the 8 LoRaWAN (EU) frequencies, so according to the legal limits, there is about 86400x0.01x2 = 1728 seconds per day send time at the best case.
  • TheThingsNetwork has a FUP of 30s of data upload per day, actually a huge restriction compared to the send time allowed purely by radio regulations.
  • The Luftdaten backend appears to run on a 5 minute interval, or 288 measurements per day.
  • The default Luftdaten firmware sends new data every 145s by default

With the TTN FUP of 30s upload per day, we can spend 30s / 288 = 104 ms on each transmission.

Using the LoRaWAN airtime calculator we can determine which modes can be used. At 15 bytes payload, this is only possible at SF7, the highest LoRa speed. Stretching the TTN guideline a bit, say by a factor 2, we can achieve those transfers still only at SF7 and SF8. So you need to be relatively close to a gateway.

With a smaller payload, can we use higher spreading factors? :

  • skip T/RH/P completely (8 bytes left): SF7 is possible, at SF8 we spend 102 ms per transmission
  • skip P only (15 bytes left): SF7 is possible, at SF8 we spend 123 ms per transmission

So, in conclusion: With the FUP of TTN and use of Cayenne encoding, you can just barely send enough data to transport Luftdaten PM data!

Node design

Source code for the particulate matter measurement node can be found on the github page.

The luftdaten backend has a 5 minute "heartbeat", so at least one measurement per 5 minutes should be sent to avoid disappearing from the map. The node firmware (attempts to) send a message every 145 seconds, just like the luftdaten WiFi sensor.

Measurements run in a cycle running through the following states:

  • INIT: determine presence of the SDS011, print the SDS011 serial number
  • IDLE: wait until the start of the cycle, then turn on the fan
  • WARMUP: wait 20 seconds while the sensor "warms up"
  • MEASURE: measure 10 seconds, then turn off fan, calculate median/average and send LoRaWAN message

Building with platformio

Platformio is used to compile and upload the code to the node.

To install platformio (example for Debian):

 sudo apt install python3-pip
 sudo pip3 install platformio
 pio update

To compile and upload:

 pio run -t upload

TTN key provisioning

The node needs to be registered at TheThingsNetwork, in order for its messages to be accepted by the TTN.

The following scheme is used to make TTN provisioning as simple as possible:

  • Each node can be programmed with the *same* software, no source code modification is required
  • The node administrator needs to enter the following properties at the TTN console, for each node:
    • The Device EUI is derived from the node-specific ESP32 MAC address, the node shows this on its OLED
    • The App EUI has a fixed value and is the same for all nodes
    • The App Key has a fixed value and is the same for all nodes
    • Use 32-bit frame counter, disable frame counter checks
  • The node does over-the-air-activation (OTAA) only once and then stores the OTAA parameters in internal (simulated) EEPROM. Upon reboot, the node resumes the connection with these parameters
    • A long press on the PRG button restarts the OTAA procedure

Backend

This is implemented by my LoraLuftdatenForwarder.

It currently supports the following:

  • subscribe to a TTN MQTT stream and receive incoming message
  • decode Cayenne and custom payloads
  • forward to luftdaten.info/sensor.community
  • forward to opensensemap.org
  • forward to feinstaub app (experimental), https://pm.mrgames-server.de/

Example, to receive data using mosquitto, separately from the backend:

 mosquitto_sub -h eu.thethings.network -p 1883 -t +/devices/+/up -u particulatematter -P ttn-account-v2.cNaB2zO-nRiXaCUYmSAugzm-BaG_ZSHbEc5KgHNQFsk

Example upstream data:

 particulatematter/devices/ttgo_mac/up {"app_id":"particulatematter","dev_id":"ttgo_mac","hardware_serial":"000084B14CA4AE30","port":1,"counter":16,"payload_raw":"AAEALAAd/////w==","metadata":{"time":"2019-04-13T08:37:45.338427686Z","frequency":868.3,"modulation":"LORA","data_rate":"SF11BW125","airtime":823296000,"coding_rate":"4/5","gateways":[{"gtw_id":"eui-008000000000b8b6","timestamp":2000599916,"time":"2019-04-13T08:37:45.320735Z","channel":1,"rssi":-115,"snr":-3,"rf_chain":1,"latitude":52.0182,"longitude":4.70844,"altitude":27}]}}

Example downstream data:

 particulatematter/devices/ttgo_mac/events/down/sent {"payload":"YPUvASalGgEDEf8AAcqtmOw=","message":{"app_id":"particulatematter","dev_id":"ttgo_mac","port":0},"gateway_id":"eui-008000000000b8b6","config":{"modulation":"LORA","data_rate":"SF9BW125","airtime":164864000,"counter":282,"frequency":869525000,"power":27}}

Gateway API:

 https://account.thethingsnetwork.org/api/v2/gateways/eui-xxxxxxxxxxx