Difference between revisions of "LoraWanDustSensor"

From RevSpace
Jump to navigation Jump to search
(Backend)
(Node)
 
(76 intermediate revisions by the same user not shown)
Line 7: Line 7:
 
   }}
 
   }}
  
 +
== The idea ==
 +
The idea is to create a system consisting of:
 +
* a sensor that measures airborne particulate matter and sends the measurement data using LoRa to TheThingsNetwork.
 +
* a forwarder application that collects the data from TTN and forwards it to luftdaten.info
  
== The plan ==
+
This has been done before by other people, but it appears there is no universal solution.
The plan is to create a system consisting of:
+
I am publishing all source code on github and will put up documentation on this wiki.
* a sensor that measures airborne particulate matter and sends the measurement data using LoRa/TheThingsNetwork to a central location.
+
Everyone appears to invent their own payload format, I'd like something more universal like Cayenne.
* a backend that collects the data from TTN and forwards it to luftdaten.info
+
This page proposes an encoding based on the Cayenne standard.
  
This has been done before by other people, but can't find any really good examples:
+
A similar thing has been done by:
* source code location is obscure, I will publish all source code on github and put up documentation on this wiki
+
* https://github.com/VekotinVerstas/AQLoRaBurk
* payload format is non-standard, I'd like to use something relatively universal and standard, so I think I will try to use the Cayenne LPP format.
+
* https://github.com/alexcorvis84/LoRa_MakersAsturias
 +
* TTN Ulm, see https://github.com/verschwoerhaus/ttn-ulm-feinstaub (the sensor code) and https://github.com/verschwoerhaus/ttn-ulm-muecke (the forwarder, in python)
 +
* https://alexander-schnapper.de/2019/04/02/mobile-feinstaubmessung/
 +
* [https://github.com/nijmeijer/TTN_Apeldoorn_in_Data_2018 Apeldoorn-in-data]
 +
* (others...)
  
This has been done by TTN Ulm, see https://github.com/verschwoerhaus/ttn-ulm-feinstaub (the sensor code) and
+
One thing in particular I'd like to do better than existing solutions is to use proper OTAA for the LoRa connection to TTN.
https://github.com/verschwoerhaus/ttn-ulm-muecke (the forwarder, in python) 
+
OTAA means over-the-air-activation and is a mechanism to dynamically negotiate communication/encryption keys instead of programmed specifically in each sensor node.
 +
Once the OTAA is done successfully, the node remembers the network id, device address, session keys, etc for future communication.
  
Sensors join the network using OTAA (instead of ABP), that way I try to minimize the setup of each individual node.
+
This makes it possible to have a *single* firmware image for all sensor nodes and it simplifies the setup:
 +
* you flash the node with a unified firmware
 +
* the node shows its unique id on the OLED
 +
* at the TTN console, you register a new device with the unique id
 +
* the sensor node receives encryption keys over the air automatically
 +
* done!
  
So I'd like to just re-invent the wheel properly this time.
+
(idea: an ESP32 has a wifi connection too, perhaps registering the node can be done fully automatically, over wifi/internet)
  
 
=== Next steps ===
 
=== Next steps ===
* Make it work with the 'old' LMIC library and ABP
+
* Are we bound by a 5 minute interval towards Luftdaten (dropping off the maps if sending less frequently)? YES PROBABLY
* Try out the 'new' LMIC library at https://github.com/mcci-catena/arduino-lmic
+
* Implement a kind of schedule: turn the sds011 on, wait some time, take a measurement, turn it off, wait for some time
* Experiment with OTAA, saving OTAA parameters, restoring OTAA parameters, see
+
** Luftdaten uses default send schedule of 145 seconds
* Finish the Java software (MQTT listener, payload decoder, luftdaten forwarder)
+
** make this match the TTN send schedule? -> YES, every 5 minutes, this is achievable even within the TTN FUP at SF7.
 +
** use the built-in on-off schedule of the sds011? -> NO, we probably want more control over the on-off times
 +
** implement calculating the median of accumulated measurements (half of the measurements is higher, other half is lower)
  
=== Dump ===
+
=== Links ===
* [https://github.com/akrasnoshchok/LoRa/blob/master/esp32_heltec_v2/weather_station/weather_station.ino code that joins TTN by OTAA and saves the OTAA parameters]
 
  
 
Useful links for the TTGO LoRa board:
 
Useful links for the TTGO LoRa board:
Line 37: Line 52:
 
* https://github.com/fcgdam/TTGO_LoRa32
 
* https://github.com/fcgdam/TTGO_LoRa32
 
* https://ictoblog.nl/2018/01/10/mijn-eerste-chinese-esp32-verbonden-met-the-things-network
 
* https://ictoblog.nl/2018/01/10/mijn-eerste-chinese-esp32-verbonden-met-the-things-network
 +
* [https://github.com/akrasnoshchok/LoRa/blob/master/esp32_heltec_v2/weather_station/weather_station.ino Example code that joins TTN by OTAA and saves the OTAA parameters]
 +
* https://github.com/CivicLabsBelgium/lora_particle_sensor
 +
* https://github.com/Cinezaster/ttn2luftdaten_forwarder
  
 
== Hardware ==
 
== Hardware ==
The node is based on Arduino, in particular a TTGO ESP32 board with onboard LoRa chip.
+
The node is based on Arduino, in particular a TTGO ESP32 board with onboard SX1276 LoRa chip.
 
The sensor is an SDS-011, just like in the luftdaten project.
 
The sensor is an SDS-011, just like in the luftdaten project.
 +
For humidity/temperature, I am using a BME280 (superior to the DHT11/22).
  
 
NOTE: https://www.thethingsnetwork.org/community/berlin/post/warning-attention-users-of-ttgo21-v16-boards-labeled-t3_v16-on-pcb-battery-exploded-and-got-on-fire
 
NOTE: https://www.thethingsnetwork.org/community/berlin/post/warning-attention-users-of-ttgo21-v16-boards-labeled-t3_v16-on-pcb-battery-exploded-and-got-on-fire
Line 51: Line 70:
 
* SDS011 5V to ESP32 5V
 
* SDS011 5V to ESP32 5V
 
* SDS011 GND to ESP32 GND
 
* SDS011 GND to ESP32 GND
* SDS011 TXD to ESP32 GPIO25
+
* SDS011 TXD to ESP32 GPIO35 (maybe I can find two suitable pins close together)
* SDS011 RXD to ESP32 GPIO34
+
* SDS011 RXD to ESP32 GPIO25 (maybe I can find two suitable pins close together)
 +
* BME280 3V todo
 +
* BME280 SDA todo
 +
* BME280 SCL todo
 +
* BME280 GND todo
  
 
== Software ==
 
== Software ==
Source code can be found [https://github.com/bertrik/LoraWanPmSensor on the github page].
+
=== Source code ===
 +
Source code is hosted on github:
 +
* [https://github.com/bertrik/LoraWanPmSensor Arduino node], written in C/Arduino, built using platformio. This firmware joins TTN by OTAA and sends the measurement data using Cayenne.
 +
* [https://github.com/bertrik/LoraLuftdatenForwarder TTN-to-luftdaten forwarder], written in Java, built using gradle. This picks up the Cayenne encoded data and forwards it to the Luftdaten API.
  
 
=== Common ===
 
=== Common ===
==== Packet format ====
 
Packets transferred over LoRa contain:
 
* structure version id: 2 bytes
 
* PM10 value, encoded in units of 0.1 ug/m3: 2 bytes
 
* PM2.5 value, encoded in units of 0.1 ug/m3: 2 bytes
 
* temperature, encoded in units of 0.1 deg C: 2 bytes
 
* relative humidity, encoded in units of 0.1%, 2 bytes
 
Total: 10 bytes
 
  
Not present value is 0xFFFF. Encoding is big endian.
+
===== Cayenne =====
 +
It's reasonably compact, it's a standard format, you can get a preview of the data in the TTN console. Interacts nicely with other platforms.
  
Would be nice to use Cayenne for this, but I don't know if Cayenne has an id for particulate matter.
+
[https://community.mydevices.com/t/cayenne-lpp-2-0/7510 Specification for Cayenne LPP 2.0]
 +
 
 +
HOWEVER:
 +
* A standard analog value has a resolution of 0.01 units and is signed, leaving a maximum of 327.67 ug/m3. This value is possibly exceeded in extreme cases (new year's eve).
 +
* Still not as compact as direct binary encoding
 +
 
 +
So proposed format:
 +
* PM1.0: digital input (type 2), channel id 0, with value in units of 0.01 ug/m3, saturated to 327.67 ug/m3 (optional)
 +
* PM10: digital input (type 2), channel id 1, with value in units of 0.01 ug/m3, saturated to 327.67 ug/m3
 +
* PM2.5: digital input (type 2), channel id 2, with value in units of 0.01 ug/m3, saturated to 327.67 ug/m3
 +
* Temperature: temperature (type 103), channel 3, with value in units of 0.1 degrees celcius
 +
* Humidity: humidity (type 104), channel 4, with value in units of 0.5 %
 +
 
 +
Example:
 +
  0x01 0x01 0xXX 0xXX  0x02 0x01 0xYY 0xYY  0x03 0x67 0xTT 0xTT  0x04 0x68 0xHH
 +
  <== PM10 value ===>  <== PM2.5 value ==>  <== temperature ==>  <= humidity =>
 +
 
 +
<b>Total payload size is 15 bytes.</b> The <b>LoRaWAN header adds 13 bytes</b> (at least).
 +
 
 +
Values encoded are averaged using the median over the accumulated measurements over the measurement interval.
 +
 
 +
===== Binary =====
 +
How other projects encode the data:
 +
* TTN Apeldoorn (?): https://github.com/tijnonlijn/RFM-node/blob/master/dustduino_PPD42NS_example.ino#L327 sends 5 bytes
 +
** 1 byte : 0x04
 +
** 2 bytes: PM25(?) big endian
 +
** 2 bytes: PM10(?) big endian
 +
* TTN Ulm: https://github.com/verschwoerhaus/ttn-ulm-feinstaub/blob/master/ttnulmdust/ttnulmdust.ino#L225 sends 8 bytes:
 +
** 2 bytes: PM10, big endian (unit 0.01 ug/m3)
 +
** 2 bytes: PM2.5, big endian (unit 0.01 ug/m3)
 +
** 2 bytes: humidity (unit of 0.01%)
 +
** 2 bytes: temperature (unit of 0.01 degree Celcius)
 +
* RIVM node, sends 20 bytes
 +
** 1 byte temperature (unit deg Celcius ?)
 +
** 1 byte relative humidity (unit % ?)
 +
** 2 bytes pressure (unit?)
 +
** 2 bytes pm10 (unit?)
 +
** 2 bytes pm25 (unit?)
 +
** 2 bytes op1 (unit?)
 +
** 2 bytes op2 (unit?)
 +
** 4 bytes latitude (unit?)
 +
** 4 bytes longitude (unit?)
 +
* Apeldoorn in data: https://github.com/nijmeijer/TTN_Apeldoorn_in_Data_2018/blob/master/AiD_Dust_2018/AiD_Dust_2018.ino#L184 sends 16 bytes:
 +
** 4 bytes: pm2_5 float big endian (unit?)
 +
** 4 bytes: pm10 float big endian (unit?)
 +
** 4 bytes: humidity float big endian (unit?)
 +
** 4 bytes: temperature float big endian (unit?)
 +
 
 +
A smaller payload means less time in the air, smaller chance of collision with other LoRaWAN packets and more packets per hour.
 +
However, there is always an overhead from the LoRaWAN package (minimum 13 bytes), so using the smallest encoding (5 bytes) compared to the largest (16 bytes), reduces the on-air-time by only 23%.
 +
 
 +
=== LoraWan time budget ===
 +
Consider the following:
 +
* radio regulations generally have a 1% duty cycle requirement for the two band used by the 8 LoRaWAN (EU) frequencies, so according to the legal limits, there is about 86400x0.01x2 = 1728 seconds per day send time at the best case.
 +
* TheThingsNetwork has a [https://www.thethingsnetwork.org/forum/t/limitations-data-rate-packet-size-30-seconds-uplink-and-10-messages-downlink-per-day-fair-access-policy-guidelines/1300 FUP of 30s of data upload per day], actually a huge restriction compared to the send time allowed purely by radio regulations.
 +
* The Luftdaten backend appears to run on a 5 minute interval, or 288 measurements per day.
 +
* The default Luftdaten firmware sends new data every 145s by default
 +
 
 +
With the TTN FUP of 30s upload per day, we can spend 30s / 288 = 104 ms on each transmission.
 +
 
 +
Using the [https://www.thethingsnetwork.org/airtime-calculator LoRaWAN airtime calculator] we can determine which modes can be used.
 +
So with 288 measurements, we can spend 30s / 288 = 104 ms per packet.
 +
At 15 bytes payload, this is only possible at SF7, the highest LoRa speed.
 +
Stretching the TTN guideline a bit, say by a factor 2, we can achieve those transfers still only at SF7 and SF8. So you need to be relatively close to a gateway.
 +
 
 +
With a smaller payload, can we use higher spreading factors? :
 +
* skip T/RH/P completely (8 bytes left): SF7 is possible, at SF8 we spend 102 ms per transmission
 +
* skip P only (15 bytes left): SF7 is possible, at SF8 we spend 123 ms per transmission
 +
 
 +
So, in conclusion: <b>With the FUP of TTN and use of Cayenne encoding, you can just barely send enough data to transport Luftdaten PM data!</b>
  
 
=== Node ===
 
=== Node ===
To compile the code, platformio is used, see the github archive.
+
Source code for the particulate matter measurement node can be found [https://github.com/bertrik/LoraWanPmSensor on the github page].
 +
 
 +
Typical measurement scheme in the Luftdaten firmware, as far as I understand:
 +
* first it waits until (cycle_time - warmup_time - measure_time), typically 145 s, 15 s and 5 s respectively.
 +
* turn on the fan and wait 15 seconds for "warmup"
 +
* measure during 5 seconds
 +
* turn off the fan
 +
 
 +
The luftdaten backend appears to have a 5 minute "heartbeat", so we should at least send one measurement to avoid dropping off the map.
 +
 
 +
Proposed scheme for LoRa sensor:
 +
* turn fan on and wait 15 seconds, indicate this on the screen
 +
* measure for 5 seconds, indicate this on the screen
 +
* turn fan off, show measurement on screen
 +
* send measurement by LoRa
 +
* do nothing until the rest of the 5 minute cycle
  
For OTAA, I use the following scheme, to keep administration to a minimum:
+
==== Platformio ====
 +
To compile and upload the code to the node, platformio is used.
 +
 
 +
To install platformio (example for Debian):
 +
  sudo apt install python3-pip
 +
  sudo pip3 install platformio
 +
  pio update
 +
 
 +
To compile and upload:
 +
  pio run -t upload
 +
 
 +
==== Design ====
 +
The function of the node software is to collect data from the SDS011 (particulate matter) and BME280 (temperature/humidity) at regular intervals,
 +
encode this as a data packet and send it over LoRaWAN towards TheThingsNetwork.
 +
 
 +
For the LoRaWAN data connection, over-the-air activation (OTAA) is used. I use the following scheme, to keep administration to a minimum:
 
* The Device EUI is derived from the ESP32 MAC address, the node shows this on its OLED
 
* The Device EUI is derived from the ESP32 MAC address, the node shows this on its OLED
 
* The App EUI is generated in the TTN console, it is the same for all nodes
 
* The App EUI is generated in the TTN console, it is the same for all nodes
 
* The App Key is generated in the TTN console, it is the same for all nodes
 
* The App Key is generated in the TTN console, it is the same for all nodes
* The device is registered in the TTN console by the Device EUI (if this doesn't happen automatically)
+
* The device is registered in the TTN console by the Device EUI (this doesn't happen automatically). Frame counter checks are disabled.
 
* OTAA is done only once for each node. After that, the OTAA parameters are stored in (simulated) EEPROM.
 
* OTAA is done only once for each node. After that, the OTAA parameters are stored in (simulated) EEPROM.
** Perhaps with a long press on the button, we can reset the OTAA?
+
** A long press on the PRG button restarts the OTAA procedure
 
* OTAA progress is shown on the OLED
 
* OTAA progress is shown on the OLED
* If OTAA has been done successfully, the node restores the session parameters on next bootup
+
* If OTAA has been done successfully, the node restores the session parameters negotiated during OTAA on next bootup, so it can quickly resume sending data.
 
* I'm NOT saving the upload frame counter (this would be preferable), just disable the feature in the TTN console.
 
* I'm NOT saving the upload frame counter (this would be preferable), just disable the feature in the TTN console.
  
For OTAA, the following needs to be saved/restored:
+
TODO to figure out:
* LMIC.nwkKey (16 bytes)
 
* LMIC.artKey (16 bytes)
 
* LMIC.seqnoUp (32-bit number)
 
* LMIC.devaddr (4 bytes)
 
 
 
Logic:
 
* To start an OTAA join from scratch, use LMIC_startJoining();
 
* To continue from previous OTAA
 
** use LMIC_setSession() with parameters retrieved from LMIC_getSessionKeys() just after OTAA join;
 
** <s>LMIC.seqnoUp = savdata.seqnoUp;</s>
 
 
* What about the channel setup? The node connects using 3 frequencies, but receives a bigger list of frequencies during OTAA JOIN.
 
* What about the channel setup? The node connects using 3 frequencies, but receives a bigger list of frequencies during OTAA JOIN.
  
Line 119: Line 227:
  
 
=== Backend ===
 
=== Backend ===
A Java program subscribes to the MQTT stream, decodes the telemetry packets and forwards them to the luftdaten API.
+
This is implemented by my [https://github.com/bertrik/LoraLuftdatenForwarder LoraLuftdatenForwarder].
 +
It subscribes to the MQTT stream, decodes the telemetry packets and forwards them to the luftdaten API.
 
There is no storage of measurement data in the Java application.
 
There is no storage of measurement data in the Java application.
 
I've already developed some Java code that publishes the measurement values towards luftdaten.info.
 
Also I've developed code before to subscribe to the TTN MQTT stream.
 
  
 
To receive data using mosquitto:
 
To receive data using mosquitto:
Line 133: Line 239:
 
Example downstream data:
 
Example downstream data:
 
   particulatematter/devices/ttgo_mac/events/down/sent {"payload":"YPUvASalGgEDEf8AAcqtmOw=","message":{"app_id":"particulatematter","dev_id":"ttgo_mac","port":0},"gateway_id":"eui-008000000000b8b6","config":{"modulation":"LORA","data_rate":"SF9BW125","airtime":164864000,"counter":282,"frequency":869525000,"power":27}}
 
   particulatematter/devices/ttgo_mac/events/down/sent {"payload":"YPUvASalGgEDEf8AAcqtmOw=","message":{"app_id":"particulatematter","dev_id":"ttgo_mac","port":0},"gateway_id":"eui-008000000000b8b6","config":{"modulation":"LORA","data_rate":"SF9BW125","airtime":164864000,"counter":282,"frequency":869525000,"power":27}}
 +
 +
Gateway API:
 +
  https://account.thethingsnetwork.org/api/v2/gateways/eui-xxxxxxxxxxx

Latest revision as of 01:48, 23 May 2020

Project LoRaWAN dust Sensor
LoraWanDustSensor.jpg
LoRaWAN airborne particulate matter sensor
Status In progress
Contact bertrik
Last Update 2020-05-23

The idea

The idea is to create a system consisting of:

  • a sensor that measures airborne particulate matter and sends the measurement data using LoRa to TheThingsNetwork.
  • a forwarder application that collects the data from TTN and forwards it to luftdaten.info

This has been done before by other people, but it appears there is no universal solution. I am publishing all source code on github and will put up documentation on this wiki. Everyone appears to invent their own payload format, I'd like something more universal like Cayenne. This page proposes an encoding based on the Cayenne standard.

A similar thing has been done by:

One thing in particular I'd like to do better than existing solutions is to use proper OTAA for the LoRa connection to TTN. OTAA means over-the-air-activation and is a mechanism to dynamically negotiate communication/encryption keys instead of programmed specifically in each sensor node. Once the OTAA is done successfully, the node remembers the network id, device address, session keys, etc for future communication.

This makes it possible to have a *single* firmware image for all sensor nodes and it simplifies the setup:

  • you flash the node with a unified firmware
  • the node shows its unique id on the OLED
  • at the TTN console, you register a new device with the unique id
  • the sensor node receives encryption keys over the air automatically
  • done!

(idea: an ESP32 has a wifi connection too, perhaps registering the node can be done fully automatically, over wifi/internet)

Next steps

  • Are we bound by a 5 minute interval towards Luftdaten (dropping off the maps if sending less frequently)? YES PROBABLY
  • Implement a kind of schedule: turn the sds011 on, wait some time, take a measurement, turn it off, wait for some time
    • Luftdaten uses default send schedule of 145 seconds
    • make this match the TTN send schedule? -> YES, every 5 minutes, this is achievable even within the TTN FUP at SF7.
    • use the built-in on-off schedule of the sds011? -> NO, we probably want more control over the on-off times
    • implement calculating the median of accumulated measurements (half of the measurements is higher, other half is lower)

Links

Useful links for the TTGO LoRa board:

Hardware

The node is based on Arduino, in particular a TTGO ESP32 board with onboard SX1276 LoRa chip. The sensor is an SDS-011, just like in the luftdaten project. For humidity/temperature, I am using a BME280 (superior to the DHT11/22).

NOTE: https://www.thethingsnetwork.org/community/berlin/post/warning-attention-users-of-ttgo21-v16-boards-labeled-t3_v16-on-pcb-battery-exploded-and-got-on-fire

Page with correct pinout of the ESP32 LoRa board.

Luftdaten uses a cycle time of 145 seconds for the SDS011.

Proposed hardware connections:

  • SDS011 5V to ESP32 5V
  • SDS011 GND to ESP32 GND
  • SDS011 TXD to ESP32 GPIO35 (maybe I can find two suitable pins close together)
  • SDS011 RXD to ESP32 GPIO25 (maybe I can find two suitable pins close together)
  • BME280 3V todo
  • BME280 SDA todo
  • BME280 SCL todo
  • BME280 GND todo

Software

Source code

Source code is hosted on github:

  • Arduino node, written in C/Arduino, built using platformio. This firmware joins TTN by OTAA and sends the measurement data using Cayenne.
  • TTN-to-luftdaten forwarder, written in Java, built using gradle. This picks up the Cayenne encoded data and forwards it to the Luftdaten API.

Common

Cayenne

It's reasonably compact, it's a standard format, you can get a preview of the data in the TTN console. Interacts nicely with other platforms.

Specification for Cayenne LPP 2.0

HOWEVER:

  • A standard analog value has a resolution of 0.01 units and is signed, leaving a maximum of 327.67 ug/m3. This value is possibly exceeded in extreme cases (new year's eve).
  • Still not as compact as direct binary encoding

So proposed format:

  • PM1.0: digital input (type 2), channel id 0, with value in units of 0.01 ug/m3, saturated to 327.67 ug/m3 (optional)
  • PM10: digital input (type 2), channel id 1, with value in units of 0.01 ug/m3, saturated to 327.67 ug/m3
  • PM2.5: digital input (type 2), channel id 2, with value in units of 0.01 ug/m3, saturated to 327.67 ug/m3
  • Temperature: temperature (type 103), channel 3, with value in units of 0.1 degrees celcius
  • Humidity: humidity (type 104), channel 4, with value in units of 0.5 %

Example:

 0x01 0x01 0xXX 0xXX  0x02 0x01 0xYY 0xYY  0x03 0x67 0xTT 0xTT  0x04 0x68 0xHH
 <== PM10 value ===>  <== PM2.5 value ==>  <== temperature ==>  <= humidity =>

Total payload size is 15 bytes. The LoRaWAN header adds 13 bytes (at least).

Values encoded are averaged using the median over the accumulated measurements over the measurement interval.

Binary

How other projects encode the data:

A smaller payload means less time in the air, smaller chance of collision with other LoRaWAN packets and more packets per hour. However, there is always an overhead from the LoRaWAN package (minimum 13 bytes), so using the smallest encoding (5 bytes) compared to the largest (16 bytes), reduces the on-air-time by only 23%.

LoraWan time budget

Consider the following:

  • radio regulations generally have a 1% duty cycle requirement for the two band used by the 8 LoRaWAN (EU) frequencies, so according to the legal limits, there is about 86400x0.01x2 = 1728 seconds per day send time at the best case.
  • TheThingsNetwork has a FUP of 30s of data upload per day, actually a huge restriction compared to the send time allowed purely by radio regulations.
  • The Luftdaten backend appears to run on a 5 minute interval, or 288 measurements per day.
  • The default Luftdaten firmware sends new data every 145s by default

With the TTN FUP of 30s upload per day, we can spend 30s / 288 = 104 ms on each transmission.

Using the LoRaWAN airtime calculator we can determine which modes can be used. So with 288 measurements, we can spend 30s / 288 = 104 ms per packet. At 15 bytes payload, this is only possible at SF7, the highest LoRa speed. Stretching the TTN guideline a bit, say by a factor 2, we can achieve those transfers still only at SF7 and SF8. So you need to be relatively close to a gateway.

With a smaller payload, can we use higher spreading factors? :

  • skip T/RH/P completely (8 bytes left): SF7 is possible, at SF8 we spend 102 ms per transmission
  • skip P only (15 bytes left): SF7 is possible, at SF8 we spend 123 ms per transmission

So, in conclusion: With the FUP of TTN and use of Cayenne encoding, you can just barely send enough data to transport Luftdaten PM data!

Node

Source code for the particulate matter measurement node can be found on the github page.

Typical measurement scheme in the Luftdaten firmware, as far as I understand:

  • first it waits until (cycle_time - warmup_time - measure_time), typically 145 s, 15 s and 5 s respectively.
  • turn on the fan and wait 15 seconds for "warmup"
  • measure during 5 seconds
  • turn off the fan

The luftdaten backend appears to have a 5 minute "heartbeat", so we should at least send one measurement to avoid dropping off the map.

Proposed scheme for LoRa sensor:

  • turn fan on and wait 15 seconds, indicate this on the screen
  • measure for 5 seconds, indicate this on the screen
  • turn fan off, show measurement on screen
  • send measurement by LoRa
  • do nothing until the rest of the 5 minute cycle

Platformio

To compile and upload the code to the node, platformio is used.

To install platformio (example for Debian):

 sudo apt install python3-pip
 sudo pip3 install platformio
 pio update

To compile and upload:

 pio run -t upload

Design

The function of the node software is to collect data from the SDS011 (particulate matter) and BME280 (temperature/humidity) at regular intervals, encode this as a data packet and send it over LoRaWAN towards TheThingsNetwork.

For the LoRaWAN data connection, over-the-air activation (OTAA) is used. I use the following scheme, to keep administration to a minimum:

  • The Device EUI is derived from the ESP32 MAC address, the node shows this on its OLED
  • The App EUI is generated in the TTN console, it is the same for all nodes
  • The App Key is generated in the TTN console, it is the same for all nodes
  • The device is registered in the TTN console by the Device EUI (this doesn't happen automatically). Frame counter checks are disabled.
  • OTAA is done only once for each node. After that, the OTAA parameters are stored in (simulated) EEPROM.
    • A long press on the PRG button restarts the OTAA procedure
  • OTAA progress is shown on the OLED
  • If OTAA has been done successfully, the node restores the session parameters negotiated during OTAA on next bootup, so it can quickly resume sending data.
  • I'm NOT saving the upload frame counter (this would be preferable), just disable the feature in the TTN console.

TODO to figure out:

  • What about the channel setup? The node connects using 3 frequencies, but receives a bigger list of frequencies during OTAA JOIN.

I've seen the following from the node, receiving an ADR:

 40829907: engineUpdate, opmode=0x8
 40829935: EV_TXSTART
 40829939: engineUpdate, opmode=0x888
 40830013: TXMODE, freq=868300000, len=25, SF=11, BW=125, CR=4/5, IH=0
 40944876: setupRx1 txrxFlags 0x22 --> 01
 start single rx: now-rxtime: 5
 40945013: RXMODE_SINGLE, freq=868300000, SF=11, BW=125, CR=4/5, IH=0
 rxtimeout: entry: 40951170 rxtime: 40945001 entry-rxtime: 6169 now-entry: 5 rxtime-txend: 63524
 41005584: setupRx2 txrxFlags 0x1 --> 02
 start single rx: now-rxtime: 4
 41005720: RXMODE_SINGLE, freq=869525000, SF=9, BW=125, CR=4/5, IH=0
 41017003: process options (olen=0x5)
 41017012: LinkAdrReq: p1:11 chmap:00ff chpage:00 uprt:01 ans:86
 41017019: ??ack error ack=1 txCnt=0
 41017073: decodeFrame txrxFlags 0x2 --> 22
 41017312: Received downlink, window=RX2, port=-1, ack=1, txrxFlags=0x22
 41017708: EV_TXCOMPLETE (includes waiting for RX windows)
 41018027: engineUpdate, opmode=0x800

Backend

This is implemented by my LoraLuftdatenForwarder. It subscribes to the MQTT stream, decodes the telemetry packets and forwards them to the luftdaten API. There is no storage of measurement data in the Java application.

To receive data using mosquitto:

 mosquitto_sub -h eu.thethings.network -p 1883 -t +/devices/+/up -u particulatematter -P ttn-account-v2.cNaB2zO-nRiXaCUYmSAugzm-BaG_ZSHbEc5KgHNQFsk

Example upstream data:

 particulatematter/devices/ttgo_mac/up {"app_id":"particulatematter","dev_id":"ttgo_mac","hardware_serial":"000084B14CA4AE30","port":1,"counter":16,"payload_raw":"AAEALAAd/////w==","metadata":{"time":"2019-04-13T08:37:45.338427686Z","frequency":868.3,"modulation":"LORA","data_rate":"SF11BW125","airtime":823296000,"coding_rate":"4/5","gateways":[{"gtw_id":"eui-008000000000b8b6","timestamp":2000599916,"time":"2019-04-13T08:37:45.320735Z","channel":1,"rssi":-115,"snr":-3,"rf_chain":1,"latitude":52.0182,"longitude":4.70844,"altitude":27}]}}

Example downstream data:

 particulatematter/devices/ttgo_mac/events/down/sent {"payload":"YPUvASalGgEDEf8AAcqtmOw=","message":{"app_id":"particulatematter","dev_id":"ttgo_mac","port":0},"gateway_id":"eui-008000000000b8b6","config":{"modulation":"LORA","data_rate":"SF9BW125","airtime":164864000,"counter":282,"frequency":869525000,"power":27}}

Gateway API:

 https://account.thethingsnetwork.org/api/v2/gateways/eui-xxxxxxxxxxx