DustSensor

Project Dust Sensor
Experiments with a dust sensor
Status	Completed
Contact	bertrik
Last Update	2018-04-03

Introduction

To get an idea of the levels of fine dust around my house, I ordered a dust sensor module to play around with. In particular, I ordered this one, the Plantower PMS 7003, AliExpress link. It's being advertised as an advanced generation of dust sensor (7th generation), while still reasonably priced (E15,-). It uses a laser to perform the measurement and it contains a small fan to move the air around.

In parallel with the fine dust level measurement, my device also measures basic meteo data using a BME280 module: temperature, relative humidity and air pressure.

I recommend to also get the cable converter board. The pitch of the connector on the sensor module is slightly non-standard: 2 x 5 pins with a spacing of 0.05 inch (instead of the common 0.1 inch spacing).

Graph

The data produced by this sensor is sent as JSON to a MQTT server. From there it is picked up by two protocol converters: 1) the Telegraf plugin to insert it into my own influx db and 2) a custom written Java application to forward it to the RIVM samenmeten website

Future work

Next steps:

• make temperature, humidity and pressure work with the samenmeten website. This code on github suggests that temperature, pressure, humidity should be submitted as fields "Temp", "Pres", "Hum" respectively (instead of "T", "P" and "RH"!)
• delete old out-of-date data from the samenmeten database, at some point I accidentally submitted some values of 2^31 for some properties (happens when BME280 loses I2C connection): connect to the db using some kind of command-line client
• make it possible to support more than 1 sensor, e.g. also sensors from friends. Like making a distinction between individual sensors using the MQTT topic where the data is published and perhaps keep a table of metadata (with influxdb credentials, sensor location, etc.) at the MQTT listener level
• play with other kinds of sensors, in particular the SDS011, AliExpress link.

Results

Below is a graph of the dust levels around new years eve 2017/2018, separated by particle size

You can clearly see a peak just after midnight.

See also the map of these two experiments:

• Samenmeten dataportaal
• Vuurwerk-experiment 2017/2018

Hardware

Data sheets can be found:

• here (english) and
• here (chinese)

The module takes 5V to run and communicates using 3.3V levels. I connect it using a NodeMCU.

The module gives an estimate of the total mass of the particles (microgram/m3) in 3 categories: PM1.0, PM2.5 and PM10, both for "standard particle" (CF-1) and "standard atmosphere". It also makes an estimate of the raw number of particles per size category, total 6 categories: 0.3-0.5-1.0-2.5-5.0-10 micrometer. I don't know how it actually works on the inside and is able to make a distinction between particles of different size.

Connections

NodeMCU	PMS7003/BME280	Remark
D3	PMS7003-RST	Pulled-up to 3.3V on NodeMCU side, may not be actually needed
D4	PMS7003-SET	Pulled-up to 3.3V on NodeMCU side, may not be actually needed
D5	BME280-SCL	I2C-SCL
D6	BME280-SDA	I2C-SDA,
D7	PMS7003-TX	NodeMCU receive, PMS7003 transmit
D8	PMS7003-RX	NodeMCU transmit, PMS7003 receive
GND	PMS7003-GND / BME280-GND	GND ground reference
VU	PMS7003-VCC	USB voltage (5V)
3.3V	BME280-VCC

Special thanks to Crashjuh for helping with the cable, putting dupont connectors on them, making it a lot easier to connect the module to an ESP8266.

Software

The software archive can be found at github, it consists of the following parts:

• experimental (not yet working) arduino software for reading the sds011 dust sensor
• the arduino software reading the pms7003 and sending it as an MQTT stream; and
• a Java based bridge application that takes the MQTT data produced by the arduino and forwards it into the RIVM influx database.

Reading measurements from the pms7003 works, sending commands to the module does not. I don't know yet whether this is a hardware or software problem.

Typing 'make' builds and runs unit tests that verify parsing of measurement data and construction of command data. The sub-directory 'pms7003_esp' contains the .ino file to be opened in the Arduino IDE.

Libraries used:

• SoftwareSerial for serial communication with the sensor
• WiFiClient for WiFi connectivity
• WiFiManager to present a captive portal and allow selection of an AP to connect to the internet
• PubSubClient to handle publishing of data over MQTT
• BME280I2C and wire for interacting with a BME280 for basic meteo data (temperature, humidity, pressure)

Protocol outgoing data

The protocol for measurement data from the module is that data is sent in frames of 32 bytes at 9600 bps. Each frame starts with specific begin marker bytes, then a length byte, then the actual data, and finally a checksum. I use a simple finite state machine to parse the stream and get synchronized to the frames.

Protocol

Value	Meaning	Remark
0x42 0x4D	Begin marker	ASCII for characters 'B' and 'M'
0x00 0x1C	Length	Length of following data
XX YY	PM1.0 concentration (ug/m3)	CF=1, standard particles
XX YY	PM2.5 concentration (ug/m3)	CF=1, standard particles
XX YY	PM10 concentration (ug/m3)	CF=1, standard particles
XX YY	PM1.0 concentration (ug/m3)	in atmospheric environment
XX YY	PM2.5 concentration (ug/m3)	in atmospheric environment
XX YY	PM10 concentration (ug/m3)	in atmospheric environment
XX YY	Number of particles >0.3 um	in 0.1 liter air
XX YY	Number of particles >0.5 um	in 0.1 liter air
XX YY	Number of particles >1.0 um	in 0.1 liter air
XX YY	Number of particles >2.5 um	in 0.1 liter air
XX YY	Number of particles >5.0 um	in 0.1 liter air
XX YY	Number of particles >10 um	in 0.1 liter air
VV	Version number	?
EE	Error code	?
C1 C2	Check code	basically the sum of all bytes up to the check code

Data is encoded in big-endian format.

Protocol incoming data

This protocol allows commands to be sent to the module, also in frames. Each command frame consists of 7 bytes. It starts with two marker bytes, then a command byte, two data bytes and finally two checksum bytes.

Command Protocol

Value	Meaning	Remark
0x42 0x4D	Begin marker	ASCII for characters 'B' and 'M'
CC	Command	0xE1, 0xE2 or 0xE4
HH LL	Data	Depends on command
C1 C2	Check code	basically the sum of all bytes up to the check code

References

Page on aqicn about the PMS5003/7003

List of dust sensors:

• list of sensors from "samen meten aan luchtkwaliteit".
• another overview of dust sensors.

Citizen science projects for measuring airborne dust:

• Dutch RIVM official air quality map
• Dutch RIVM citizen science air quality map
• Dutch RIVM fireworks smog monitoring during new year's eve
• Belgian project "ik adem" - fijnstofmetingen
• German project luftdaten - measure air data by yourself

Dust measurement blog:

• scapeler

Making graphs of dust data

To create nice graphs, I used the following stack of tools/applications:

• a dust sensor, as described here, that publishes measurement data towards an MQTT server
• an MQTT server to accept the data and forward it to subscribers
• the 'Telegraf' importer that listens on the MQTT stream and converts the data to influx database
• an influx database, to store the measurement data
• grafana, to grab the data from the database and display it

Sensor

The sensor produces JSON, grouping the information from one message from the module together. Example

  bertrik/pms7003/json {"pms7003":{"pm1_0":2,"pm2_5":3,"pm10":4},"bme280":{"t":21.1,"rh":50,"p":1012.4}}

Telegraf

Things I did:

• downloaded and installed the Telegraf .deb from here.
• generated a default configuration using:

  telegraf --input-filter mqtt_consumer --output-filter influxdb config >telegraf.conf

• edited the configuration to set mqtt and influxdb settings
  • influxdb output plugin
    • urls = ["http://172.29.0.1:8086"]
  • mqtt_consumer plugin
    • topics=[ "bertrik/pms7003/json" ]
    • data_format="json"
    • data_type="integer"
• test-run using 'telegraf <TODO>'
• copied the final telegraf.conf to /etc/telegraf and restarted the telegraf service

  systemctl restart telegraf

Grafana

Add a data source, pointing to the influx DB. Provide credentials and verify by pressing the 'test connection' button.

Add a dashboard, add a row to the dashboard, add a graph panel to the row. Under 'metrics', add a query with the following properties:

• FROM mqtt_consumer WHERE topic = bertrik/pms7003/json
• SELECT field(amb_pm10)
• GROUP BY
• ALIAS BY PM10
• repeat for other particle sizes (amb_pm2_5 and amb_pm_1_0)

Regelgeving

From: https://www.rivm.nl/Documenten_en_publicaties/Algemeen_Actueel/Uitgaven/Milieu_Leefomgeving/Dossier_Fijn_stof/Maart_2013/Regelgeving.pdf

 Regelgeving voor fijn stof (PM10)
 De regelgeving voor fijn stof (PM10) kent twee doelstellingen:
 - Een grenswaarde voor het jaargemiddelde: 40 µg/m3 als jaargemiddelde mag niet worden
   overschreden. Aan deze grenswaarde moet sinds 2005 worden voldaan. Behoudens
   ‘derogatie’ zijn geen uitzonderingen mogelijk (->Derogatie). Deze grenswaarde beoogt
   vooral bescherming te bieden tegen de langetermijneffecten van fijn stof.
 - Een grenswaarde voor het daggemiddelde: 50 µg/m3 als daggemiddelde mag op niet meer
   dan 35 dagen per jaar worden overschreden. 5 Aan deze grenswaarde moet sinds 2005
   worden voldaan. Behoudens ‘derogatie’ zijn geen uitzonderingen mogelijk (->Derogatie).
   Deze grenswaarde is vooral bedoeld om bescherming te bieden tegen de
   kortetermijneffecten van fijn stof.

Luftdaten.info

It would be nice to get my data also on the luftdaten.info website. Basically I would use the same kind of mechanism as used for RIVM: write a Java program to capture the MQTT stream and convert it to their API.

Unfortunately I could not find a description of the luftdaten API on their FAQ, so I'll try to reverse engineer it from existing implementations. I could find this but it's not telling me a lot.

The opendata-stuttgart wiki on github appears to have some more information.

reverse engineering

This is a description of the upload process as reverse engineered from here and this code from ttn-ulm-muecke.

• data is sent as a HTTP POST to https://api.luftdaten.info/v1/push-sensor-data/
• the HTTP POST uses headers "X-PIN" and "X-Sensor"
  • header "X-PIN" seems to indicate the type of data
    • 1 = SDS011 or PMS7003
    • 3 = BMP180
    • 5 = PPD42NS
    • 7 = DHT22
    • 11 = BME280
  • header "X-Sensor" seems to indicate the unique sensor id (ESP id?)
  • apparently no "Content-Type" header
• the body of the POST is JSON with the following fields:
  • "software_version": string containing the software version of the sender, for example "python-dusty 0.0.1"
  • "sensordatavalues": array of structures containing measurement data, looking like this:
    • "value_type" (when X-PIN=1): string describing the measurement item type, can be "P1" (PM10 value) or "P2" (PM2.5 value) with dust value in ug/m3 I suppose
    • "value_type" (when X-PIN=7): string describing the measurement item type, can be "temperature", "humidity"
    • "value_type" (when X-PIN=11): string describing the measurement item type, can be "temperature", "pressure", "humidity"
    • "value": numeric type containing the measurement value

As far as I understand, other meta-information like latitude/longitude of the sensor is sent by an e-mail registration process. This is linked to your measurements by means of the unique sensor id.