Meshtastic

Project Meshtastic
Experiments with Meshtastic
Status	In progress
Contact	bertrik
Last Update	2025-04-03

1. Intro

The page describes the research done to figure out if Meshtastic can be used to transfer citizen science measurement data.

The proposition of what I want to achieve:

• a citizen science data sensor sends measurement data into the meshtastic network
• at some point the data reaches a meshtastic node that is connected to the internet, which forwards it over MQTT
• a central listener process picks up the data from the internet and processes it further (e.g. forward it to sensor.community)

Stuff to research:

• how extensive is the network, see meshnet.nl coverage map
• how reliable is the network:
  • Nodes have a wide range of technical configuration, however the majority of nodes appear to use the following:
    • LongFast channel with encryption key '1' is used at 869.525 MHz
    • Internet-connected nodes in the Netherlands are usually connected over MQTT at mqtt.meshnet.nl
    • a fraction of about 1-in-5 nodes appears to be connected to internet
    • hop limit of 3, this is the default
    • most nodes are in CLIENT mode (this is good)

Interesting info:

• My own node: https://db.meshnet.nl/da639b54.html

TODO

• create a particulate matter sensor that sends meshtastic
• write a backend/plugin for my sensor-data-bridge

2. Design

Overall design:

• citizen science nodes send data in meshtastic-compatible format, so packets can be routed accross the meshtastic network
• data is sent from the node typically every 5 minutes
• data is broadcasted inside the network, until it reaches a node with a MQTT backend connection, typically within a maximum of 3 hops.
• data arriving at the MQTT server is picked up by a backend application, which can then process it further

Protocol design:

• Use the common LongFast channel, with the default key, this makes sure that other nodes understand our message and are able to forward it over MQTT
• Follow the the MeshPacket structure, wrap our citizen science playload in a protobuf with specific portnr (PRIVATE_APP = 256)
• Citizen science payload has its own custom encoding, as usual, so it is basically opaque to meshtastic, just a byte array
• A checksum (16-bit/32-bit?) or MAC at the end of the payload allows us to verify that it really is a citizen science data packet after decryption
• Each packet already has a semi-unique packet id. Combined with the node id, we consider this unique, so we can identify duplicates at the backend

Backend:

• The backend application listens on a topic on the de-facto central MQTT server for the netherlands, used by most meshtastic nodes, which is mqtt.meshnet.nl
• Decoding works as follows:
  • Decrypt with the pre-shared-key (this always work but might result in garbage)
  • Attempt to decode according to the the protobuf portnum+payload (meshtastic 'Data')
  • Check the port number
  • Check and remove checksum/MAC in the payload
  • Process the citizen science data payload
• If all of the steps above check out, consider it to be a valid packet
• Check in a local cache if this is a duplicate packet and if so, ignore it
• process citizen science data payload: there is no such thing as TTN attributes, so any data required for further forwarding need to be kept locally (e.g. login credentials for opensense / sensor.community)

2.1. Packet structure

On the LoRa radio level:

[ 16-symbol preamble | lora explicit header | lora payload ]

Within the lora payload:

[ meshtastic 16-byte header | meshtastic private-data protobuf ]

The protobuf is encrypted using key "AQ==" and a nonce derived from data in the 16-byte header.

Within the meshtastic protobuf:

[ portnum | private-data | flag ]

The private-data is not encrypted separately

Within the private-data:

[ citizen-science payload | 4-byte message authentication code ]

With respect to security coding:

• Algorithm is AES-GCM
• The payload is *not* encrypted, but it suffixed by a message authentication code (MAC)
• The MAC is only 32-bit, to keep the length manageable
• The nonce used to calculate the MAC is similar to the earlier used nonce, but with the 'extra nonce' data equal to some application-specific id

Could be something like (python):

cipher = AES.new(key, AES.MODE_GCM, nonce=nonce, mac_len=4)  # 4-byte auth tag
ciphertext, auth_tag = cipher.encrypt_and_digest(message)

or (ESP32):

    mbedtls_gcm_context gcm;
    mbedtls_gcm_init(&gcm);
    mbedtls_gcm_setkey(&gcm, MBEDTLS_CIPHER_ID_AES, key, 128); // 128-bit AES
    mbedtls_gcm_crypt_and_tag(&gcm, MBEDTLS_GCM_ENCRYPT,
                              sizeof(plaintext), nonce, sizeof(nonce),
                              NULL, 0, plaintext, ciphertext,
                              sizeof(auth_tag), auth_tag);

3. Protocol

See https://meshtastic.org/docs/overview/mesh-algo/

Quick links:

• radio parameters, for LongFast
  • Frequency: 869.525 MHz
  • LongFast = SF11BW250, CR 4/5, 16 symbols preamble?, explicit header, CRC on (?)
  • sync word = 0x2B ("to be"), used to be 0x12, source code mentions it to be some hash containing channel name
• data structure:
  • unencrypted header https://meshtastic.org/docs/overview/mesh-algo/#layer-1-unreliable-zero-hop-messaging
  • Service Envelope, wraps a MeshPacket: https://buf.build/meshtastic/protobufs/docs/main:meshtastic#meshtastic.ServiceEnvelope
  • MeshPacket https://buf.build/meshtastic/protobufs/docs/main:meshtastic#meshtastic.MeshPacket loosely based on the message as sent over-the-air
• Port numbers: https://buf.build/meshtastic/protobufs/docs/main:meshtastic#meshtastic.PortNum
• MQTT topic organization: https://meshtastic.org/docs/software/integrations/mqtt/#mqtt-topics
• decryption code used on the liam cottle map: https://github.com/liamcottle/meshtastic-map/blob/master/src/mqtt.js#L632
• Meshtastic encryption https://meshtastic.org/docs/overview/encryption/ This is very incomplete! Details need to be reverse-engineered from the code!

4. Software

I wrote python scripts to interact with the MQTT server and arduino code to run on a LoRa capable ESP32 microcontroller.

See https://github.com/bertrik/mesh-backend

5. Hardware

Nice antenna? https://nl.aliexpress.com/item/1005007301116616.html

6. MQTT

In the netherlands, data is typically sent to the 'meshnet.nl' MQTT server, for example

 mosquitto_sub -h mqtt.meshnet.nl -u boreft -P meshboreft -t "#" -v

Examples of typical data:

 msh/7460-7463/2/stat/!da5857c0 online
 msh/EU_868/NL/2/e/LongFast/!eb66115c �%]�g(=���gx�� H5��Aw=]�gE��H`���������LongFast␦

Topics with data on MQTT have the following structure:

 msh/REGION/2/e/CHANNELNAME/USERID

6.1. Sending messages

Requirements for sending mqtt downlinks:

• the meshtastic node needs to have a channel named "mqtt" (exactly), see https://github.com/meshtastic/firmware/blob/master/src/mqtt/MQTT.cpp#L354
• the meshtastic node has JSON be enabled in its MQTT settings
• -> the meshtastic node listens on topic: "ROOT/2/json/mqtt/+", where ROOT = "msh/gouda" in my case
• -> the mqtt publisher sends to topic: 'msh/gouda/2/json/mqtt/!da639b54' for example
• example payload:

  {"from": 3663960916, "type": "sendtext", "payload": "Test"}'

6.2. Example data

Examples of data as decoded from MQTT using the meshtastic python service wrapper:

packet {
  from: 2732702784
  to: 4294967295
  decoded {
    portnum: POSITION_APP
    payload: "\r\224\234\024\037\025\303\266\233\002\030\n\270\001 "
  }
  id: 663882246
  rx_time: 1741511999
  rx_snr: -18
  hop_limit: 2
  rx_rssi: -128
  hop_start: 3
}
channel_id: "LongFast"
gateway_id: "!da544e50"

Packet with encrypted data:

packet {
  from: 1128181476
  to: 4294967295
  channel: 8
  encrypted: "\007\355{o\340e\352\221\204\3112\365h\304[0\321&\351^{]\264\334\373\320\313>\213\2635\023\345'"
  id: 4272151039
  rx_time: 1741512321
  rx_snr: 5.75
  hop_limit: 4
  rx_rssi: -83
  hop_start: 5
}
channel_id: "LongFast"
gateway_id: "!da5c87d4"