MainsFrequency2.0: Difference between revisions

From RevSpace
Jump to navigation Jump to search
 
(13 intermediate revisions by 2 users not shown)
Line 10: Line 10:
This project is a reboot of this [[MainsFrequency|earlier main frequency counter]], aiming for more accuracy and lower latency.
This project is a reboot of this [[MainsFrequency|earlier main frequency counter]], aiming for more accuracy and lower latency.


It's based on the Arduino platform, using an ESP8266 to do the wifi/network/MQTT stuff.
This has been achieved by using an ESP32c3 board with the 'phasing' algorithm described below.
The frequency measurement principle is to measure the time between zero crossings (in a statistically robust way).
No mains interfacing hardware is required, measurement data can be easily transferred using MQTT over a WiFi network.


=== Concept ===
=== Concept ===
Line 27: Line 27:
* https://www.electroschematics.com/voltage-sensor/
* https://www.electroschematics.com/voltage-sensor/


=== Zero cross algoritm ===
=== Phasing algorithm ===
Algorithm for getting accurate instantaneous frequency:
The algorithm is as follows:
* Sample the mains frequency waveform continuously at 5000 Hz
* Correlate each sample with a perfect 50 Hz reference waveform, by keeping a sum of the waveform multiplied by a cosine (I) and a sum of the waveform multiplied by a sine (Q)
* For each second, calculate the phase using the accumulated I and Q sums with atan2(Q, I).
* The change in phase during one second is directly proportional to the frequency deviation from the 50 Hz reference frequency
 
This results in:
* A continuous process, with a new frequency value every second
* The algorithm uses *all* of the data contained in one second (there is no single threshold value), thus producing a robust value
* Any DC offset in the mains frequency waveform is automatically balanced out, no need to keep track of the median or quartile values
 
=== Zero cross algoritm (old idea) ===
This is my older idea for an algorithm to get accurate instantaneous frequency:
* During approximately 100 ms, sample the mains frequency waveform and store it in a buffer.
* During approximately 100 ms, sample the mains frequency waveform and store it in a buffer.
* Calculate the median, lower and upper quartiles of the waveform amplitude data
* Calculate the median, lower and upper quartiles of the waveform amplitude data
Line 35: Line 47:
* Keep track of the first and the 50th interpolated zero-crossing time, then calculate the frequency from the time difference
* Keep track of the first and the 50th interpolated zero-crossing time, then calculate the frequency from the time difference
-> this should give about 1 millihertz frequency resolution in one second
-> this should give about 1 millihertz frequency resolution in one second
=== Phasing algorithm ===
Alternative algorithm:
* Sample the mains frequency waveform at about 5000 Hz
* Correlate during 1 second with a perfect 50 Hz reference waveform, by keeping a sum of the waveform multiplied by a cosine (I) and a sum of the waveform multiplied by a sine (Q)
* After one second, calculate the phase using, atan2(Q, I). Changes in the phase compared to the previous second represent the deviation / time shift from the perfect 50 Hz frequency.
This results in:
* continuous process, with a new frequency value every second
* we use *all* of the data contained in one second, a condition for producing a robust value
* any DC offset in the mains frequency waveform is automatically balanced out, no need to keep track of the median or quartile values
=== Ideas ===
* While playing around with an STM32 blue pill, it turned out you might not need a mains power interface at all. A piece of wire picks up ambient 50 Hz already.
** Typical accuracy of the STM32 clock crystal appears a bit lower than that of the ESP8266, but for visualisation this might not be so important.
** You cannot do this with the Wemos D1 mini board (containing an ESP8266), because the A/D input has voltage-level matching resistors attached that pull it towards ground
* A raspberry pi pico appears to satisfy many requirements:
** Able to sample mains frequency directly from ambient signals on its sensitive A/D input, without the need for a transformer circuit
** Able to sample a signal with good timing accuracy, it has a crystal better than 30 ppm (0.0015 Hz at 50 Hz)
** Might allow control of a LED ring using FastLED for visualisation (not possible with STM32), to be researched
** Has WiFi support, although rather dodgy on the Arduino platform, for publishing directly to mqtt
** Relatively inexpensive
* We can probably run the stats-collecting and frequency-calculating processes in parallel, continuously.
** The frequency-calculating process can copy a snapshot of the quartile values when needed, to get a recent set, but keep decision-levels stable while running.


== Visualisation ==
== Visualisation ==
Line 64: Line 52:
* publish frequency as a number over WiFi / MQTT for visualization as a graph-over-time on our grafana server
* publish frequency as a number over WiFi / MQTT for visualization as a graph-over-time on our grafana server
* idea: directly on a LED ring. The ring shows an integer number (e.g.) of 50 Hz cycles, with the color of the pixel indicating the analog value
* idea: directly on a LED ring. The ring shows an integer number (e.g.) of 50 Hz cycles, with the color of the pixel indicating the analog value
* just show the relative phase angle on the LED ring, this could be a single bright pixel for the latest phase measurement and progressively dimmer pixels for older phase measurements.


=== Mains waveform ring ===
=== Mains waveform ring ===
Line 81: Line 70:


== Hardware ==
== Hardware ==
[[File:luatos_esp32c3_pinout.png|right|thumb|ESP32c3 luatos board pinout]]
=== Luatos ESP32C3 ===
This module (I recommend the one with serial chip) seems to work quite well so far:
* Senses mains waveform from surroundings with just a wire into its ADC input, using the 'phase' software algorithm
* Clock crystal appears to be accurate, ESP32c3 datasheet claims it requires a 10 ppm crystal (or 0.5 mHz at 50Hz). Measured value corresponds to https://mainsfrequency.com
* Has WiFi, works with WifiManager library, so can easily push measurements to MQTT over WiFi for example
* Inexpensive, only 3 euros or so
The ADC1/UART1 RX pin is the input for 50 Hz. You can just leave that unconnected and it will pick up 50 Hz, or perhaps connect a short piece (20 cm or so) of wire to it.
=== Using ZMPT101B (old idea) ===
For measurement with an ESP8266, like a Wemos D1 mini or nodemcu, you need to put a 180k ohm resistor in line with the output from the ZMPT101B to the A0 input.
For measurement with an ESP8266, like a Wemos D1 mini or nodemcu, you need to put a 180k ohm resistor in line with the output from the ZMPT101B to the A0 input.
The A0 input already has a 220k/100k resistive divider, effectively becoming a 400k/100k resistive divider with the series resistor, scaling down the 0-5V range to the 0-1V range required for the ADC on the ESP8266.
The A0 input already has a 220k/100k resistive divider, effectively becoming a 400k/100k resistive divider with the series resistor, scaling down the 0-5V range to the 0-1V range required for the ADC on the ESP8266.
Line 106: Line 107:
|Analog mains waveform, 0..5V, 180 kohm resistor in series
|Analog mains waveform, 0..5V, 180 kohm resistor in series
|}
|}
=== Luatos ESP32C3 ===
[[File:luatos_esp32c3_pinout.png|right|thumb|ESP32c3 luatos board pinout]]
This module (with serial chip) seems to work quite well so far:
* Senses mains waveform under some circumstances even without a wire attached, using the 'phase' software algorithm
* Clock crystal appears to be accurate, ESP32c3 datasheet claims it requires a 10 ppm crystal (or 0.5 mHz at 50Hz). Measured value corresponds to https://mainsfrequency.com
* Has WiFi, works with WifiManager library, so can easily push measurements to MQTT over WiFi for example
* Inexpensive, only 3 euros or so
The ADC1/UART1 RX pin is the input for 50 Hz. You can just leave that unconnected and it will pick up 50 Hz, or perhaps connect a short piece (20 cm or so) of wire to it.


== Software ==
== Software ==
Line 133: Line 123:
* connect with serial terminal, to watch frequency measurements
* connect with serial terminal, to watch frequency measurements
   pio device monitor
   pio device monitor
== Integration into revspace sensor infrastructure ==
There is a measurement device present at revspace (above the green tool board). It sends frequency data to the revspace MQTT server once every 5 seconds.
Processing of this data into something that our grafana can display is unfortunately broken (is that documented somewhere anyway!?), see:
https://revspace.nl/grafiekjes/d/3deykmVmz/power?orgId=1&refresh=1m
You can watch the data (using mosquitto for example):
  mosquitto_sub -h revspace.nl -t revspace/sensors/ac/frequency -v
Example data:
  topic = "revspace/sensors/ac/frequency", payload = "50.025 Hz" (retained)
In the #revspace IRC channel, you can issue a command to show the most recently known value:
  !netfrequentie
(where is the script that handles this?)
Example data:
  23:00 <@bertrik> !netfrequentie
  23:00 <@bar> bertrik: 49.898 Hz

Latest revision as of 14:46, 5 October 2023

Project MainsFrequency2.0
ZMPT101B.png
A simple mains frequency counter
Status In progress
Contact bertrik
Last Update 2023-10-05

Introduction

This project is a reboot of this earlier main frequency counter, aiming for more accuracy and lower latency.

This has been achieved by using an ESP32c3 board with the 'phasing' algorithm described below. No mains interfacing hardware is required, measurement data can be easily transferred using MQTT over a WiFi network.

Concept

Instead of just counting pulses from zero-crossings, we sample the actual 50 Hz waveform and try to estimate the zero-crossing as accurately as possible.

Desired end result:

  • get more accurate frequency measurement, aiming for 1 milli-Hertz accuracy
  • get more responsive frequency measurement, i.e. instantaneous value (1 second), not a running average over 50 seconds.

A suitable module for relatively safely sampling the mains voltage is this ZMPT101B module. It contains a transformer and an op-amp circuit.

More information about this module:

Phasing algorithm

The algorithm is as follows:

  • Sample the mains frequency waveform continuously at 5000 Hz
  • Correlate each sample with a perfect 50 Hz reference waveform, by keeping a sum of the waveform multiplied by a cosine (I) and a sum of the waveform multiplied by a sine (Q)
  • For each second, calculate the phase using the accumulated I and Q sums with atan2(Q, I).
  • The change in phase during one second is directly proportional to the frequency deviation from the 50 Hz reference frequency

This results in:

  • A continuous process, with a new frequency value every second
  • The algorithm uses *all* of the data contained in one second (there is no single threshold value), thus producing a robust value
  • Any DC offset in the mains frequency waveform is automatically balanced out, no need to keep track of the median or quartile values

Zero cross algoritm (old idea)

This is my older idea for an algorithm to get accurate instantaneous frequency:

  • During approximately 100 ms, sample the mains frequency waveform and store it in a buffer.
  • Calculate the median, lower and upper quartiles of the waveform amplitude data
  • During approximately 1000 ms, sample the waveform and apply a linear regression algorithm on the waveform value (shifted by the median value)
  • The linear regression algorithm is active in between the lower and higher quartile values and calculates an interpolated zero crossing of the waveform (with sub-sample resolution)
  • Keep track of the first and the 50th interpolated zero-crossing time, then calculate the frequency from the time difference

-> this should give about 1 millihertz frequency resolution in one second

Visualisation

There are (at least) the following two ways we can output the data:

  • publish frequency as a number over WiFi / MQTT for visualization as a graph-over-time on our grafana server
  • idea: directly on a LED ring. The ring shows an integer number (e.g.) of 50 Hz cycles, with the color of the pixel indicating the analog value
  • just show the relative phase angle on the LED ring, this could be a single bright pixel for the latest phase measurement and progressively dimmer pixels for older phase measurements.

Mains waveform ring

Concepts:

  • The LED ring shows the raw waveform over time. Position along the ring is time, intensity/color is based on the instantaneous value of the mains voltage. So (for example) three 50 Hz cycles show up as 3 dark spots and 3 light spots around the ring, approximately 120 degrees apart. Basically it shows the phase compared to a reference 50 Hz frequency.
  • The LED ring is drawn based on a reference time (derived from the crystal oscillator), assumed to be exactly 50 Hz. A slightly fast mains waveform results in a clockwise rotation of the waveform pattern, a slightly slow mains waveform results in a counter-clockwise rotation of the waveform pattern.
  • Use a colourful gradient, not just intensity. Example: https://github.com/FastLED/FastLED/wiki/Gradient-color-palettes

Calculation:

  • Mains frequency is nominally 50 Hz, so period is 20 ms (0.02 sec)
  • With three waveforms around the ring, the ring represents 60 ms of mains signal
  • Intended LED ring has 24 RGB LEDs, so 60ms / 24 LEDs = 2.5 ms per LED. So we could sample the waveform at 400 Hz, and put 1 sample on each LED.
  • Example: 16 LED ring -> sample frequency 266.66.. Hz, or sample at 800 Hz and average 3 samples/LED
  • Example: 40 LED ring -> sample frequency 666.66.. Hz, or sample at 2000 Hz and average 3 samples/LED
  • Example: 45 LED ring -> sample frequency 750 Hz
  • Example: 60 LED ring -> sample frequency 1000 Hz

Hardware

ESP32c3 luatos board pinout

Luatos ESP32C3

This module (I recommend the one with serial chip) seems to work quite well so far:

  • Senses mains waveform from surroundings with just a wire into its ADC input, using the 'phase' software algorithm
  • Clock crystal appears to be accurate, ESP32c3 datasheet claims it requires a 10 ppm crystal (or 0.5 mHz at 50Hz). Measured value corresponds to https://mainsfrequency.com
  • Has WiFi, works with WifiManager library, so can easily push measurements to MQTT over WiFi for example
  • Inexpensive, only 3 euros or so

The ADC1/UART1 RX pin is the input for 50 Hz. You can just leave that unconnected and it will pick up 50 Hz, or perhaps connect a short piece (20 cm or so) of wire to it.

Using ZMPT101B (old idea)

For measurement with an ESP8266, like a Wemos D1 mini or nodemcu, you need to put a 180k ohm resistor in line with the output from the ZMPT101B to the A0 input. The A0 input already has a 220k/100k resistive divider, effectively becoming a 400k/100k resistive divider with the series resistor, scaling down the 0-5V range to the 0-1V range required for the ADC on the ESP8266.

The "blue pill" seems to have too low accuracy of the built-in crystal, about 100 ppm, while we need about 20 ppm to get 1 mHz resolution. Notes about blue pill crystal accuracy: https://sparklogic.ru/arduino-for-stm32/accurate-blue-pill-clock-frequency-adjustment.html

Connections
ZMPT101B Wemos D1 mini Remark
GND GND Ground
VCC 5V Powers the ZMPT101B from the wemos D1 mini
OUT A0 Analog mains waveform, 0..5V, 180 kohm resistor in series

Software

Github project: https://github.com/bertrik/MainsFrequency

To flash the esp32c3 board:

  • install platformio
  • check out the code
 git clone https://github.com/bertrik/MainsFrequency
  • enter the esp32c3phase directory
 cd MainsFrequency
 cd esp32c3phase
  • connect the board, compile and upload
 pio run -t upload
  • connect with serial terminal, to watch frequency measurements
 pio device monitor

Integration into revspace sensor infrastructure

There is a measurement device present at revspace (above the green tool board). It sends frequency data to the revspace MQTT server once every 5 seconds.

Processing of this data into something that our grafana can display is unfortunately broken (is that documented somewhere anyway!?), see: https://revspace.nl/grafiekjes/d/3deykmVmz/power?orgId=1&refresh=1m

You can watch the data (using mosquitto for example):

 mosquitto_sub -h revspace.nl -t revspace/sensors/ac/frequency -v

Example data:

 topic = "revspace/sensors/ac/frequency", payload = "50.025 Hz" (retained)

In the #revspace IRC channel, you can issue a command to show the most recently known value:

 !netfrequentie

(where is the script that handles this?) Example data:

 23:00 <@bertrik> !netfrequentie
 23:00 <@bar> bertrik: 49.898 Hz