Difference between revisions of "DustSensor"

From RevSpace
Jump to: navigation, search
 
(33 intermediate revisions by the same user not shown)
Line 3: Line 3:
 
   |Picture=Pms7003.jpg
 
   |Picture=Pms7003.jpg
 
   |Omschrijving=Experiments with a dust sensor
 
   |Omschrijving=Experiments with a dust sensor
   |Status= Initializing
+
   |Status=Completed
 
   |Contact=bertrik
 
   |Contact=bertrik
 
   }}
 
   }}
Line 11: Line 11:
 
In particular, I ordered this one, the [http://www.plantower.com/en/content/?110.html Plantower PMS 7003],
 
In particular, I ordered this one, the [http://www.plantower.com/en/content/?110.html Plantower PMS 7003],
 
[https://nl.aliexpress.com/item/dit_is_m/32639894148.html AliExpress link].
 
[https://nl.aliexpress.com/item/dit_is_m/32639894148.html AliExpress link].
It's being advertised as an advanced generation of dust sensor (7th generation), while still reasonably priced.
+
It's being advertised as an advanced generation of dust sensor (7th generation), while still reasonably priced (E15,-).
It uses lasers to perform the measurement and I think it contains a small fan to move the air around.
+
It uses a laser to perform the measurement and it contains a small fan to move the air around.
 +
 
 +
I recommend to also get the cable converter board. The pitch of the connector on the sensor module is slightly non-standard: 2 x 5 pins with a spacing of 0.05 inch (instead of the common 0.1 inch spacing).
 +
 
 +
[http://graphana.vanheusden.com/dashboard/snapshot/uQwRRfufDQZ8pQefXDvbDNSIns8FqD1V Graph]
 +
 
 +
The data produced by this sensor is sent as JSON to a MQTT server.
 +
From there it is picked up by two protocol converters:
 +
1) the Telegraf plugin to insert it into my own influx db and
 +
2) a [https://github.com/bertrik/samenmetenbridge custom written Java application] to forward it to the RIVM samenmeten website
 +
 
 +
=== Future work ===
 +
 
 +
Next steps:
 +
* join it with a BME280 sensor, to measure temperature, humidity and barometric pressure.
 +
* make it possible to support more than 1 sensor, e.g. also sensors from friends. Like making a distinction between individual sensors using the topic where the data is published.
 +
 
 +
== Results ==
 +
Below is a graph of the dust levels around new years eve 2017/2018, separated by particle size
 +
[[File:dust_2017_2018.png|center]]
 +
 
 +
You can clearly see a peak just after midnight.
 +
 
 +
See also the map of these two experiments:
 +
* [http://samenmeten.rivm.nl/dataportaal/ Samenmeten dataportaal]
 +
* [http://meetnetdata.rivm.nl/vuurwerk/ Vuurwerk-experiment 2017/2018]
  
 
== Hardware ==
 
== Hardware ==
 +
[[File:dustsensor_window.jpg|thumb|right]]
 +
 
Data sheets can be found:
 
Data sheets can be found:
 
* [https://www.pdf-archive.com/2017/04/12/plantower-pms-7003-sensor-data-sheet/plantower-pms-7003-sensor-data-sheet.pdf here (english)] and
 
* [https://www.pdf-archive.com/2017/04/12/plantower-pms-7003-sensor-data-sheet/plantower-pms-7003-sensor-data-sheet.pdf here (english)] and
Line 20: Line 47:
  
 
The module takes 5V to run and communicates using 3.3V levels.
 
The module takes 5V to run and communicates using 3.3V levels.
 +
This is a perfect match with an ESP8266 board like the Wemos D1 mini.
  
It makes an estimate of the number of particles per size category, total 6 categories.
+
The module gives an estimate of the total mass of the particles (microgram/m3) in 3 categories: PM1.0, PM2.5 and PM10, both for "standard particle" (CF-1) and "standard atmosphere".
It also gives an estimate of the total mass of the particles.  
+
It also makes an estimate of the raw number of particles per size category, total 6 categories: 0.3-0.5-1.0-2.5-5.0-10 micrometer.
 +
I don't know how it actually works on the inside and is able to make a distinction between particles of different size.
  
I plan to connect this thing up using an ESP8266.
+
{| class="wikitable"
 +
|+Connections
 +
|-
 +
!Wemos D1 mini
 +
!PMS7003
 +
!Remark
 +
|-
 +
|D1
 +
|TX
 +
|Wemos receive, PMS7003 transmit
 +
|-
 +
|D2
 +
|RX
 +
|Wemos transmit, PMS7003 receive
 +
|-
 +
|D3
 +
|RST
 +
|Pulled-up on Wemos side, not sure if actually needed
 +
|-
 +
|D4
 +
|SET
 +
|Pulled-up on Wemos side, not sure if actually needed
 +
|-
 +
|GND
 +
|GND
 +
|
 +
|-
 +
|5V
 +
|Vcc
 +
|
 +
|-
 +
|}
 +
 
 +
Special thanks to [[User:Crashjuh|Crashjuh]] for helping with the cable, putting dupont connectors on them, making it a lot easier to connect the module to an ESP8266.
  
 
== Software ==
 
== Software ==
The software archive can be found at [https://github.com/bertrik/pms7003_esp github], code has been written but not tested yet.
+
The software archive can be found at [https://github.com/bertrik/pms7003_esp github].
Probably doesn't work, pin assignment on the ESP8266 still has to be decided.
+
Reading measurements works, sending commands to the module does not. I don't know yet whether this is a hardware or software problem.
 +
 
 +
Typing 'make' builds and runs unit tests that verify parsing of measurement data and construction of command data.
 +
The sub-directory 'pms7003_esp' contains the .ino file to be opened in the Arduino IDE.
  
 
Libraries used:
 
Libraries used:
* SoftwareSerial
+
* SoftwareSerial for serial communication with the sensor
* WiFiClient
+
* WiFiClient for WiFi connectivity
* WiFiManager
+
* WiFiManager to present a captive portal and allow selection of an AP to connect to the internet
* PubSubClient
+
* PubSubClient to handle publishing of data over MQTT
  
 
This dust sensor outputs its data as a 32-byte serial data stream at 9600 bps.
 
This dust sensor outputs its data as a 32-byte serial data stream at 9600 bps.
 
I plan to run the sensor in 'passive' mode. This means it's not doing a measurement unless the software explicitly tells it do one measurement.
 
  
 
=== Protocol outgoing data ===
 
=== Protocol outgoing data ===
Line 128: Line 191:
 
Each command frame consists of 7 bytes.
 
Each command frame consists of 7 bytes.
 
It starts with two marker bytes, then a command byte, two data bytes and finally two checksum bytes.  
 
It starts with two marker bytes, then a command byte, two data bytes and finally two checksum bytes.  
 +
 +
{| class="wikitable"
 +
|+Command Protocol
 +
|-
 +
!Value
 +
!Meaning
 +
!Remark
 +
|-
 +
|0x42 0x4D
 +
|Begin marker
 +
|ASCII for characters 'B' and 'M'
 +
|-
 +
|CC
 +
|Command
 +
|0xE1, 0xE2 or 0xE4
 +
|-
 +
|HH LL
 +
|Data
 +
|Depends on command
 +
|-
 +
|C1 C2
 +
|Check code
 +
|basically the sum of all bytes up to the check code
 +
|-
 +
|}
  
 
== References ==
 
== References ==
* a nice list of sensors [https://www.samenmetenaanluchtkwaliteit.nl/sensoren-voor-fijn-stof-pm25pm10 from "samen meten aan luchtkwaliteit"].
+
[http://aqicn.org/sensor/pms5003-7003/ Page on aqicn about the PMS5003/7003]
* another [http://aqicn.org/sensor/ nice overview of dust sensors].
+
 
 +
List of dust sensors:
 +
* list of sensors [https://www.samenmetenaanluchtkwaliteit.nl/sensoren-voor-fijn-stof-pm25pm10 from "samen meten aan luchtkwaliteit"].
 +
* another [http://aqicn.org/sensor/ overview of dust sensors].
 +
 
 +
Citizen science projects for measuring airborne dust:
 +
* [https://www.samenmetenaanluchtkwaliteit.nl/ Dutch RIVM air quality measurement project]
 +
* [http://meetnetdata.rivm.nl/vuurwerk/ Dutch RIVM fireworks smog monitoring during new year's eve]
 +
* [http://ik-adem.be/ Belgian project "ik adem" - fijnstofmetingen]
 +
* [http://luftdaten.info/ German project luftdaten - measure air data by yourself]
 +
 
 +
Dust measurement blog:
 +
* [https://scapeler.wordpress.com/ scapeler]
 +
 
 +
== Making graphs of dust data ==
 +
To create nice graphs, I used the following stack of tools/applications:
 +
* a dust sensor, as described here, that publishes measurement data towards an MQTT server
 +
* an MQTT server to accept the data and forward it to subscribers
 +
* the 'Telegraf' importer that listens on the MQTT stream and converts the data to influx database
 +
* an influx database, to store the measurement data
 +
* grafana, to grab the data from the database and display it
 +
 
 +
=== Sensor ===
 +
The sensor produces JSON, grouping the information from one message from the module together. Example
 +
  bertrik/pms7003/json {"cf1":{"pm1_0":2,"pm2_5":3,"pm10":4},"amb":{"pm1_0":2,"pm2_5":3,"pm10":4}}
 +
 
 +
=== Telegraf ===
 +
Things I did:
 +
* downloaded and installed the Telegraf .deb from [https://github.com/influxdata/telegraf here].
 +
* generated a default configuration using:
 +
  telegraf --input-filter mqtt_consumer --output-filter influxdb config >telegraf.conf
 +
* edited the configuration to set mqtt and influxdb settings
 +
** influxdb output plugin
 +
*** urls = ["http://172.29.0.1:8086"]
 +
** mqtt_consumer plugin
 +
*** topics=[ "bertrik/pms7003/json" ]
 +
*** data_format="json"
 +
*** data_type="integer"
 +
* test-run using 'telegraf <TODO>'
 +
* copied the final telegraf.conf to /etc/telegraf and restarted the telegraf service
 +
  systemctl restart telegraf
 +
 
 +
=== Grafana ===
 +
Add a data source, pointing to the influx DB. Provide credentials and verify by pressing the 'test connection' button.
 +
 
 +
Add a dashboard, add a row to the dashboard, add a graph panel to the row.
 +
Under 'metrics', add a query with the following properties:
 +
* FROM mqtt_consumer WHERE topic = bertrik/pms7003/json
 +
* SELECT field(<b>amb_pm10</b>)
 +
* GROUP BY
 +
* ALIAS BY <b>PM10</b>
 +
* repeat for other particle sizes (amb_pm2_5 and amb_pm_1_0)

Latest revision as of 09:35, 20 January 2018

Project Dust Sensor
Pms7003.jpg
Experiments with a dust sensor
Status Completed
Contact bertrik
Last Update 2018-01-20

Introduction

I ordered a dust sensor module to perform measurements of airborne dust. In particular, I ordered this one, the Plantower PMS 7003, AliExpress link. It's being advertised as an advanced generation of dust sensor (7th generation), while still reasonably priced (E15,-). It uses a laser to perform the measurement and it contains a small fan to move the air around.

I recommend to also get the cable converter board. The pitch of the connector on the sensor module is slightly non-standard: 2 x 5 pins with a spacing of 0.05 inch (instead of the common 0.1 inch spacing).

Graph

The data produced by this sensor is sent as JSON to a MQTT server. From there it is picked up by two protocol converters: 1) the Telegraf plugin to insert it into my own influx db and 2) a custom written Java application to forward it to the RIVM samenmeten website

Future work

Next steps:

  • join it with a BME280 sensor, to measure temperature, humidity and barometric pressure.
  • make it possible to support more than 1 sensor, e.g. also sensors from friends. Like making a distinction between individual sensors using the topic where the data is published.

Results

Below is a graph of the dust levels around new years eve 2017/2018, separated by particle size

Dust 2017 2018.png

You can clearly see a peak just after midnight.

See also the map of these two experiments:

Hardware

Dustsensor window.jpg

Data sheets can be found:

The module takes 5V to run and communicates using 3.3V levels. This is a perfect match with an ESP8266 board like the Wemos D1 mini.

The module gives an estimate of the total mass of the particles (microgram/m3) in 3 categories: PM1.0, PM2.5 and PM10, both for "standard particle" (CF-1) and "standard atmosphere". It also makes an estimate of the raw number of particles per size category, total 6 categories: 0.3-0.5-1.0-2.5-5.0-10 micrometer. I don't know how it actually works on the inside and is able to make a distinction between particles of different size.

Connections
Wemos D1 mini PMS7003 Remark
D1 TX Wemos receive, PMS7003 transmit
D2 RX Wemos transmit, PMS7003 receive
D3 RST Pulled-up on Wemos side, not sure if actually needed
D4 SET Pulled-up on Wemos side, not sure if actually needed
GND GND
5V Vcc

Special thanks to Crashjuh for helping with the cable, putting dupont connectors on them, making it a lot easier to connect the module to an ESP8266.

Software

The software archive can be found at github. Reading measurements works, sending commands to the module does not. I don't know yet whether this is a hardware or software problem.

Typing 'make' builds and runs unit tests that verify parsing of measurement data and construction of command data. The sub-directory 'pms7003_esp' contains the .ino file to be opened in the Arduino IDE.

Libraries used:

  • SoftwareSerial for serial communication with the sensor
  • WiFiClient for WiFi connectivity
  • WiFiManager to present a captive portal and allow selection of an AP to connect to the internet
  • PubSubClient to handle publishing of data over MQTT

This dust sensor outputs its data as a 32-byte serial data stream at 9600 bps.

Protocol outgoing data

The protocol for measurement data from the module is that data is sent in frames. Each frame starts with specific begin marker bytes, then a length byte, then the actual data, and finally a checksum. I think it is a good match to use a simple finite state machine to parse the stream and get synchronized to the frames.

Protocol
Value Meaning Remark
0x42 0x4D Begin marker ASCII for characters 'B' and 'M'
0x00 0x1C Length Length of following data
XX YY PM1.0 concentration (ug/m3) CF=1, standard particles
XX YY PM2.5 concentration (ug/m3) CF=1, standard particles
XX YY PM10 concentration (ug/m3) CF=1, standard particles
XX YY PM1.0 concentration (ug/m3) in atmospheric environment
XX YY PM2.5 concentration (ug/m3) in atmospheric environment
XX YY PM10 concentration (ug/m3) in atmospheric environment
XX YY Number of particles >0.3 um in 0.1 liter air
XX YY Number of particles >0.5 um in 0.1 liter air
XX YY Number of particles >1.0 um in 0.1 liter air
XX YY Number of particles >2.5 um in 0.1 liter air
XX YY Number of particles >5.0 um in 0.1 liter air
XX YY Number of particles >10 um in 0.1 liter air
VV Version number ?
EE Error code ?
C1 C2 Check code basically the sum of all bytes up to the check code

Data is encoded in big-endian format.

Protocol incoming data

This protocol allows commands to be sent to the module, also in frames. Each command frame consists of 7 bytes. It starts with two marker bytes, then a command byte, two data bytes and finally two checksum bytes.

Command Protocol
Value Meaning Remark
0x42 0x4D Begin marker ASCII for characters 'B' and 'M'
CC Command 0xE1, 0xE2 or 0xE4
HH LL Data Depends on command
C1 C2 Check code basically the sum of all bytes up to the check code

References

Page on aqicn about the PMS5003/7003

List of dust sensors:

Citizen science projects for measuring airborne dust:

Dust measurement blog:

Making graphs of dust data

To create nice graphs, I used the following stack of tools/applications:

  • a dust sensor, as described here, that publishes measurement data towards an MQTT server
  • an MQTT server to accept the data and forward it to subscribers
  • the 'Telegraf' importer that listens on the MQTT stream and converts the data to influx database
  • an influx database, to store the measurement data
  • grafana, to grab the data from the database and display it

Sensor

The sensor produces JSON, grouping the information from one message from the module together. Example

  bertrik/pms7003/json {"cf1":{"pm1_0":2,"pm2_5":3,"pm10":4},"amb":{"pm1_0":2,"pm2_5":3,"pm10":4}}

Telegraf

Things I did:

  • downloaded and installed the Telegraf .deb from here.
  • generated a default configuration using:
  telegraf --input-filter mqtt_consumer --output-filter influxdb config >telegraf.conf
  • edited the configuration to set mqtt and influxdb settings
    • influxdb output plugin
    • mqtt_consumer plugin
      • topics=[ "bertrik/pms7003/json" ]
      • data_format="json"
      • data_type="integer"
  • test-run using 'telegraf <TODO>'
  • copied the final telegraf.conf to /etc/telegraf and restarted the telegraf service
  systemctl restart telegraf

Grafana

Add a data source, pointing to the influx DB. Provide credentials and verify by pressing the 'test connection' button.

Add a dashboard, add a row to the dashboard, add a graph panel to the row. Under 'metrics', add a query with the following properties:

  • FROM mqtt_consumer WHERE topic = bertrik/pms7003/json
  • SELECT field(amb_pm10)
  • GROUP BY
  • ALIAS BY PM10
  • repeat for other particle sizes (amb_pm2_5 and amb_pm_1_0)