What is the best practice for dealing with spurious readings from censors?
I’ve been experimenting with Pi and ESP8266 using AM2302, TSL2561, BMP180, and DS18B20.
The DS18B20 (1-wire) is near perfect (less than 1 bad reading on 5760 15 second samples per day), but AM2302 (modified 1-wire) and BMP (i2c) seem to produce dozens of bad reading per day (out of 5760).
So I have tried to the
read censor twice, and check that they are within 1%
keep a moving average of last 10 reading and drop any reading outside of 10%, but include it in moving average, in case it is actually correct
So is there another technique or best practice?
I notice the cayenne software that reads the device automatically does not contain bad readings, so they must to located and dropped, or averaged away to keep a smooth curve.
Best is to find out why you hare having spurious issues and address that. May be as simple as slowing down your serial bus speed, or adding a capacitor. Most of the time you can at least determine what the issue is, but if you still need a filtering algorithm, I recommend a median filter.
A median filter will sort all the measurements from smallest to largest, then pick the one in the middle. If you have some odd measurements, either high or low, they won’t skew your results like an average will, and you won’t have to mess with your sampling times as this can cause issues elsewhere with other processes like say the wifi stack or the Cayenne business.
Here’s a post where I’ve used one in the past. Feel free to use this code:
+1 for this. Best to find the root cause. In addition to what @kreggly said, try a couple different software libraries to rule out software, then connect to another device to rule out hardware.
Thanks for the info. I’ve googled the i2c speed idea, so I’ll try that. I can’t immediately find anything on the capacitor idea, so if you could send a link.
My bmp - pi experiment only has about 8 inches of wire, with 2 i2c devices, partially soldered, but goes through a breadboard and gives me this (air pressure)
with my moving average algorithm tightened-up part way through, to drop the far out errors, but little errors coming through.
What error-rate would normally be expected on i2c or 1-wire?
I assumed zero would be the expectation, and crc’s would dispose of the bad data, but I haven’t dug through the protocol and the libraries to see if that exists on the i2c or 1-wire systems.
I have tested BME/BMP280 and noisy line and unstable enviroment can affect him.
As for hardware setup, it needs to be away from psu and esp antenna if any.
It needs to enclosed so no direct impact in measuring enviroment can affect him (wind blows etc.)
As for software part proper mode of operation for specific situation is recomended.
BMP sensors give middle value of data that was read during the loop like @kreggly said.
Example if you have 1 read at 1000 and one at 500 he will give you 750 even the actual value is lets say 1000
So either resamplin rates or reading mode is not ok.
Most of the library available support this but you need to define it in your code.
All of this suits both I2C or SPI modes.
Download BME/BMP datasheet and see the examples for usage in different situations.
0 is ideal, but not practical. Less than 1% would be realistic. As @Agroelektronik mentioned if you have electrical noise in the area that will cause errors. You can try shielding the wires to see if that helps at all.
