Difference between revisions of "MainsFrequency2.0"

From RevSpace
Jump to navigation Jump to search
(Hardware)
m (Zero cross algoritm)
 
(51 intermediate revisions by 2 users not shown)
Line 1: Line 1:
 
   {{Project
 
   {{Project
 
   |Name=MainsFrequency2.0
 
   |Name=MainsFrequency2.0
   |Picture=netfrequentiemeter.jpg
+
   |Picture=ZMPT101B.png
 
   |Omschrijving=A simple mains frequency counter
 
   |Omschrijving=A simple mains frequency counter
 
   |Status=In progress
 
   |Status=In progress
Line 10: Line 10:
 
This project is a reboot of this [[MainsFrequency|earlier main frequency counter]], aiming for more accuracy and lower latency.
 
This project is a reboot of this [[MainsFrequency|earlier main frequency counter]], aiming for more accuracy and lower latency.
  
It's based on the Arduino platform, using an ESP8266 to do the wifi/network/MQTT stuff.
+
This has been achieved by using an ESP32c3 board with the 'phasing' algorithm described below.
The frequency measurement principle is to measure the time between zero crossings (in a statistically robust way).
+
No mains interfacing hardware is required, measurement data can be easily transferred using MQTT over a WiFi network.
  
 
=== Concept ===
 
=== Concept ===
Line 27: Line 27:
 
* https://www.electroschematics.com/voltage-sensor/
 
* https://www.electroschematics.com/voltage-sensor/
  
Algorithm for getting accurate instantaneous frequency out of it:
+
=== Phasing algorithm ===
* Sample the mains frequency waveform during approximately one second (say 5 - 10 kHz)
+
The algorithm is as follows:
* Determine the median, lower and upper quartiles of the waveform, and shift data so its values are symmetrically around zero
+
* Sample the mains frequency waveform continuously at 5000 Hz
* Calculate zero crossings by doing linear regression to find the zero-crossing of the wave in the region around the zero crossing (in between the quartile values), this gives sub-sample time resolution
+
* Correlate each sample with a perfect 50 Hz reference waveform, by keeping a sum of the waveform multiplied by a cosine (I) and a sum of the waveform multiplied by a sine (Q)
* The linear regression runs a kind of state machine, just keeping track of sum(x), sum(y), sum(x*x), sum(x*y) is enough to perform a linear regression at any time
+
* For each second, calculate the phase using the accumulated I and Q sums with atan2(Q, I).
* Do this over the approximately 50 cycles contained in the one second data and determine average frequency
+
* The change in phase during one second is directly proportional to the frequency deviation from the 50 Hz reference frequency
 +
 
 +
This results in:
 +
* A continuous process, with a new frequency value every second
 +
* The algorithm uses *all* of the data contained in one second (there is no single threshold value), thus producing a robust value
 +
* Any DC offset in the mains frequency waveform is automatically balanced out, no need to keep track of the median or quartile values
 +
 
 +
=== Zero cross algoritm (old idea) ===
 +
This is my older idea for an algorithm to get accurate instantaneous frequency:
 +
* During approximately 100 ms, sample the mains frequency waveform and store it in a buffer.
 +
* Calculate the median, lower and upper quartiles of the waveform amplitude data
 +
* During approximately 1000 ms, sample the waveform and apply a linear regression algorithm on the waveform value (shifted by the median value)
 +
* The linear regression algorithm is active in between the lower and higher quartile values and calculates an interpolated zero crossing of the waveform (with sub-sample resolution)
 +
* Keep track of the first and the 50th interpolated zero-crossing time, then calculate the frequency from the time difference
 
-> this should give about 1 millihertz frequency resolution in one second
 
-> this should give about 1 millihertz frequency resolution in one second
 +
 +
== Visualisation ==
 +
There are (at least) the following two ways we can output the data:
 +
* publish frequency as a number over WiFi / MQTT for visualization as a graph-over-time on our grafana server
 +
* idea: directly on a LED ring. The ring shows an integer number (e.g.) of 50 Hz cycles, with the color of the pixel indicating the analog value
 +
* just show the relative phase angle on the LED ring, this could be a single bright pixel for the latest phase measurement and progressively dimmer pixels for older phase measurements.
 +
 +
=== Mains waveform ring ===
 +
Concepts:
 +
* The LED ring shows the raw waveform over time. Position along the ring is time, intensity/color is based on the instantaneous value of the mains voltage. So (for example) three 50 Hz cycles show up as 3 dark spots and 3 light spots around the ring, approximately 120 degrees apart. Basically it shows the phase compared to a reference 50 Hz frequency.
 +
* The LED ring is drawn based on a reference time (derived from the crystal oscillator), assumed to be exactly 50 Hz. A slightly fast mains waveform results in a clockwise rotation of the waveform pattern, a slightly slow mains waveform results in a counter-clockwise rotation of the waveform pattern.
 +
* Use a colourful gradient, not just intensity. Example: https://github.com/FastLED/FastLED/wiki/Gradient-color-palettes
 +
 +
Calculation:
 +
* Mains frequency is nominally 50 Hz, so period is 20 ms (0.02 sec)
 +
* With three waveforms around the ring, the ring represents 60 ms of mains signal
 +
* Intended LED ring has 24 RGB LEDs, so 60ms / 24 LEDs = 2.5 ms per LED. So we could sample the waveform at 400 Hz, and put 1 sample on each LED.
 +
* Example: 16 LED ring -> sample frequency 266.66.. Hz, or sample at 800 Hz and average 3 samples/LED
 +
* Example: 40 LED ring -> sample frequency 666.66.. Hz, or sample at 2000 Hz and average 3 samples/LED
 +
* Example: 45 LED ring -> sample frequency 750 Hz
 +
* Example: 60 LED ring -> sample frequency 1000 Hz
  
 
== Hardware ==
 
== Hardware ==
 +
[[File:luatos_esp32c3_pinout.png|right|thumb|ESP32c3 luatos board pinout]]
 +
 +
=== Luatos ESP32C3 ===
 +
This module (I recommend the one with serial chip) seems to work quite well so far:
 +
* Senses mains waveform from surroundings with just a wire into its ADC input, using the 'phase' software algorithm
 +
* Clock crystal appears to be accurate, ESP32c3 datasheet claims it requires a 10 ppm crystal (or 0.5 mHz at 50Hz). Measured value corresponds to https://mainsfrequency.com
 +
* Has WiFi, works with WifiManager library, so can easily push measurements to MQTT over WiFi for example
 +
* Inexpensive, only 3 euros or so
 +
 +
The ADC1/UART1 RX pin is the input for 50 Hz. You can just leave that unconnected and it will pick up 50 Hz, or perhaps connect a short piece (20 cm or so) of wire to it.
 +
 +
=== Using ZMPT101B (old idea) ===
 
For measurement with an ESP8266, like a Wemos D1 mini or nodemcu, you need to put a 180k ohm resistor in line with the output from the ZMPT101B to the A0 input.
 
For measurement with an ESP8266, like a Wemos D1 mini or nodemcu, you need to put a 180k ohm resistor in line with the output from the ZMPT101B to the A0 input.
 
The A0 input already has a 220k/100k resistive divider, effectively becoming a 400k/100k resistive divider with the series resistor, scaling down the 0-5V range to the 0-1V range required for the ADC on the ESP8266.
 
The A0 input already has a 220k/100k resistive divider, effectively becoming a 400k/100k resistive divider with the series resistor, scaling down the 0-5V range to the 0-1V range required for the ADC on the ESP8266.
Line 55: Line 101:
 
|VCC
 
|VCC
 
|5V
 
|5V
|Power
+
|Powers the ZMPT101B from the wemos D1 mini
 
|-
 
|-
 
|OUT
 
|OUT
Line 65: Line 111:
 
Github project:
 
Github project:
 
https://github.com/bertrik/MainsFrequency
 
https://github.com/bertrik/MainsFrequency
 +
 +
To flash the esp32c3 board:
 +
* install platformio
 +
* check out the code
 +
  git clone https://github.com/bertrik/MainsFrequency
 +
* enter the esp32c3phase directory
 +
  cd MainsFrequency
 +
  cd esp32c3phase
 +
* connect the board, compile and upload
 +
  pio run -t upload
 +
* connect with serial terminal, to watch frequency measurements
 +
  pio device monitor
 +
 +
== Integration into revspace sensor infrastructure ==
 +
There is a measurement device present at revspace (above the green tool board). It sends frequency data to the revspace MQTT server once every 5 seconds.
 +
 +
Processing of this data into something that our grafana can display is unfortunately broken (is that documented somewhere anyway!?), see:
 +
https://revspace.nl/grafiekjes/d/3deykmVmz/power?orgId=1&refresh=1m
 +
 +
You can watch the data (using mosquitto for example):
 +
  mosquitto_sub -h revspace.nl -t revspace/sensors/ac/frequency -v
 +
Example data:
 +
  topic = "revspace/sensors/ac/frequency", payload = "50.025 Hz" (retained)
 +
 +
In the #revspace IRC channel, you can issue a command to show the most recently known value:
 +
  !netfrequentie
 +
(where is the script that handles this?)
 +
Example data:
 +
  23:00 <@bertrik> !netfrequentie
 +
  23:00 <@bar> bertrik: 49.898 Hz

Latest revision as of 15:46, 5 October 2023

Project MainsFrequency2.0
ZMPT101B.png
A simple mains frequency counter
Status In progress
Contact bertrik
Last Update 2023-10-05

Introduction

This project is a reboot of this earlier main frequency counter, aiming for more accuracy and lower latency.

This has been achieved by using an ESP32c3 board with the 'phasing' algorithm described below. No mains interfacing hardware is required, measurement data can be easily transferred using MQTT over a WiFi network.

Concept

Instead of just counting pulses from zero-crossings, we sample the actual 50 Hz waveform and try to estimate the zero-crossing as accurately as possible.

Desired end result:

  • get more accurate frequency measurement, aiming for 1 milli-Hertz accuracy
  • get more responsive frequency measurement, i.e. instantaneous value (1 second), not a running average over 50 seconds.

A suitable module for relatively safely sampling the mains voltage is this ZMPT101B module. It contains a transformer and an op-amp circuit.

More information about this module:

Phasing algorithm

The algorithm is as follows:

  • Sample the mains frequency waveform continuously at 5000 Hz
  • Correlate each sample with a perfect 50 Hz reference waveform, by keeping a sum of the waveform multiplied by a cosine (I) and a sum of the waveform multiplied by a sine (Q)
  • For each second, calculate the phase using the accumulated I and Q sums with atan2(Q, I).
  • The change in phase during one second is directly proportional to the frequency deviation from the 50 Hz reference frequency

This results in:

  • A continuous process, with a new frequency value every second
  • The algorithm uses *all* of the data contained in one second (there is no single threshold value), thus producing a robust value
  • Any DC offset in the mains frequency waveform is automatically balanced out, no need to keep track of the median or quartile values

Zero cross algoritm (old idea)

This is my older idea for an algorithm to get accurate instantaneous frequency:

  • During approximately 100 ms, sample the mains frequency waveform and store it in a buffer.
  • Calculate the median, lower and upper quartiles of the waveform amplitude data
  • During approximately 1000 ms, sample the waveform and apply a linear regression algorithm on the waveform value (shifted by the median value)
  • The linear regression algorithm is active in between the lower and higher quartile values and calculates an interpolated zero crossing of the waveform (with sub-sample resolution)
  • Keep track of the first and the 50th interpolated zero-crossing time, then calculate the frequency from the time difference

-> this should give about 1 millihertz frequency resolution in one second

Visualisation

There are (at least) the following two ways we can output the data:

  • publish frequency as a number over WiFi / MQTT for visualization as a graph-over-time on our grafana server
  • idea: directly on a LED ring. The ring shows an integer number (e.g.) of 50 Hz cycles, with the color of the pixel indicating the analog value
  • just show the relative phase angle on the LED ring, this could be a single bright pixel for the latest phase measurement and progressively dimmer pixels for older phase measurements.

Mains waveform ring

Concepts:

  • The LED ring shows the raw waveform over time. Position along the ring is time, intensity/color is based on the instantaneous value of the mains voltage. So (for example) three 50 Hz cycles show up as 3 dark spots and 3 light spots around the ring, approximately 120 degrees apart. Basically it shows the phase compared to a reference 50 Hz frequency.
  • The LED ring is drawn based on a reference time (derived from the crystal oscillator), assumed to be exactly 50 Hz. A slightly fast mains waveform results in a clockwise rotation of the waveform pattern, a slightly slow mains waveform results in a counter-clockwise rotation of the waveform pattern.
  • Use a colourful gradient, not just intensity. Example: https://github.com/FastLED/FastLED/wiki/Gradient-color-palettes

Calculation:

  • Mains frequency is nominally 50 Hz, so period is 20 ms (0.02 sec)
  • With three waveforms around the ring, the ring represents 60 ms of mains signal
  • Intended LED ring has 24 RGB LEDs, so 60ms / 24 LEDs = 2.5 ms per LED. So we could sample the waveform at 400 Hz, and put 1 sample on each LED.
  • Example: 16 LED ring -> sample frequency 266.66.. Hz, or sample at 800 Hz and average 3 samples/LED
  • Example: 40 LED ring -> sample frequency 666.66.. Hz, or sample at 2000 Hz and average 3 samples/LED
  • Example: 45 LED ring -> sample frequency 750 Hz
  • Example: 60 LED ring -> sample frequency 1000 Hz

Hardware

ESP32c3 luatos board pinout

Luatos ESP32C3

This module (I recommend the one with serial chip) seems to work quite well so far:

  • Senses mains waveform from surroundings with just a wire into its ADC input, using the 'phase' software algorithm
  • Clock crystal appears to be accurate, ESP32c3 datasheet claims it requires a 10 ppm crystal (or 0.5 mHz at 50Hz). Measured value corresponds to https://mainsfrequency.com
  • Has WiFi, works with WifiManager library, so can easily push measurements to MQTT over WiFi for example
  • Inexpensive, only 3 euros or so

The ADC1/UART1 RX pin is the input for 50 Hz. You can just leave that unconnected and it will pick up 50 Hz, or perhaps connect a short piece (20 cm or so) of wire to it.

Using ZMPT101B (old idea)

For measurement with an ESP8266, like a Wemos D1 mini or nodemcu, you need to put a 180k ohm resistor in line with the output from the ZMPT101B to the A0 input. The A0 input already has a 220k/100k resistive divider, effectively becoming a 400k/100k resistive divider with the series resistor, scaling down the 0-5V range to the 0-1V range required for the ADC on the ESP8266.

The "blue pill" seems to have too low accuracy of the built-in crystal, about 100 ppm, while we need about 20 ppm to get 1 mHz resolution. Notes about blue pill crystal accuracy: https://sparklogic.ru/arduino-for-stm32/accurate-blue-pill-clock-frequency-adjustment.html

Connections
ZMPT101B Wemos D1 mini Remark
GND GND Ground
VCC 5V Powers the ZMPT101B from the wemos D1 mini
OUT A0 Analog mains waveform, 0..5V, 180 kohm resistor in series

Software

Github project: https://github.com/bertrik/MainsFrequency

To flash the esp32c3 board:

  • install platformio
  • check out the code
 git clone https://github.com/bertrik/MainsFrequency
  • enter the esp32c3phase directory
 cd MainsFrequency
 cd esp32c3phase
  • connect the board, compile and upload
 pio run -t upload
  • connect with serial terminal, to watch frequency measurements
 pio device monitor

Integration into revspace sensor infrastructure

There is a measurement device present at revspace (above the green tool board). It sends frequency data to the revspace MQTT server once every 5 seconds.

Processing of this data into something that our grafana can display is unfortunately broken (is that documented somewhere anyway!?), see: https://revspace.nl/grafiekjes/d/3deykmVmz/power?orgId=1&refresh=1m

You can watch the data (using mosquitto for example):

 mosquitto_sub -h revspace.nl -t revspace/sensors/ac/frequency -v

Example data:

 topic = "revspace/sensors/ac/frequency", payload = "50.025 Hz" (retained)

In the #revspace IRC channel, you can issue a command to show the most recently known value:

 !netfrequentie

(where is the script that handles this?) Example data:

 23:00 <@bertrik> !netfrequentie
 23:00 <@bar> bertrik: 49.898 Hz