Bad data in the chart


#1

Hello!

Arduino Uno R3 with Yun Shield, dashboard - web.

  1. I’m getting a wrong data in the charts. The wrong values were not sent by my device. I have a log in Arduino and see all data sent to Cayenne. These wrong values always appear at a round hours like 10:00 AM or 03:00 PM, etc. Look at the screenshots:



    This is a very bad thing, because I may have triggers to do something on such changes, but it is wrong change!

2, What type of smoothing do you use for data? Is it possible to download original data sent to Cayenne? Not the smoothed data you show in the chart.
Do you plan to add settings for chart? Smoothing type, chart type, etc.?

Best regards,
Dmitry


#2

So you said that these values are generated by the platform, not by your device? It sounds very strange!


#3

Yes. I have all data sent logged by Arduino IDE and there are not such spikes.
Also, note, that these spikes at the round hours only!


#4

I have the same Issue with DHT11 sensor, but I don’t keep the log file in the raspberry, so I was thinking that it is from the sensor peaks.


#5

exactly the same problem here, adruino with ds18b20


#6

I have both DS18B20 and DHT22. The same problems.


#7

Another illustration of the bug:


I’ve wrote data sent to Cayenne to Arduino log and then use Excel to draw chart.
This is the same time range. As usual there is a unexpected spike in the Cayenne and there is no such data in actual chart.
The worst thing is that Cayenne developers does not answer to this bug report and does nothing to fix it. I wrote about this bug a couple of months ago, but noting has changed.


#8

@bestes @rsiegel - Something about that? I also have similar bug with DHT11. I was thinking that is from the sensor, but I purchase original one month ago. Still has some peask :frowning:


#9

Hey @dsitnikov @a.marinos.83 @ognqn.chikov,

Sorry for late response here…I’ve had my head down working on a few things :slight_smile: hmmm. Yes if multiple people are reporting this then likely is something that is happening on our end.

@rsiegel do you have some time to look into this so we can get into backlog? It seems was brought up before here too by @svoskar


#10

Sure, apologies that this did not get a quicker response guys.

If you have a moment, could each of you grab the .csv file from the ‘Download Chart Data’ button on a chart where you see these spikes, and attach it to an an email to support@mydevices.com or Private Message to me along with a screenshot of the Cayenne line chart that doesn’t match the data? This will help a ton towards getting our developers some data to investigate the issue.

Tagging @svoskar to see if they can provide this as well?


#11

Hi,

here attached, there is my csv file and chart picture. Each time at 2,00 am, data doesn’t match

rsiegel <http://community.mydevices.com/u/rsiegel> Technical Product Manager 

April 27
Sure, apologies that this did not get a quicker response guys.

If you have a moment, could each of you grab the .csv file from the ‘Download Chart Data’ button on a chart where you see these spikes, and attach it to an an email to support@mydevices.com mailto:support@mydevices.com or Private Message to me along with a screenshot of the Cayenne line chart that doesn’t match the data? This will help a ton towards getting our developers some data to investigate the issue.

Tagging @svoskar http://community.mydevices.com/u/svoskar to see if they can provide this as well?

Visit Topic http://community.mydevices.com/t/bad-data-in-the-chart/2826/10 or reply to this email to respond.

In Reply To

bestes <http://community.mydevices.com/u/bestes> Product Owner 

April 27
Hey @dsitnikov http://community.mydevices.com/u/dsitnikov @a.marinos.83 http://community.mydevices.com/u/a.marinos.83 @ognqn.chikov http://community.mydevices.com/u/ognqn.chikov, Sorry for late response here…I’ve had my head down working on a few things :slight_smile: hmmm. Yes if multiple people are reporting this then likely is something that is happening on our end. @rsiegel http://community.mydevices.com/u/rsiegel do you have some time to look into this so w…
Visit Topic http://community.mydevices.com/t/bad-data-in-the-chart/2826/10 or reply to this email to respond.

To unsubscribe from these emails, click here http://community.mydevices.com/email/unsubscribe/29723b96728f61ef7fe2cb86ac86a5fb78103f577a86ad0fd4d1cc0124a0ce11.

History_V03-3DOVFJWK-FJWK-A8DF-6EE7-E39C-9A4F-F8A1_c696e201df24e15f44991fa3fb124b1411ea4512_hour_2017-04-01-00_00_00_2017-04-05 23_59_00.csv (40.8 KB)


#12

Hi @svoskar, thank you for the data, but I don’t think that CSV matches this chart?

The chart range looks to be roughly April 11 to April 17, and the CSV shows April 1 through April 5.

I do see the drops at 2:00 AM every day in your CSV data however, and it looks like the voltage recorded matches that?


#13

Hi,
here are data again, csv:

and related chart:

thanks for checking that

History_V03-3DOVFJWK-FJWK-A8DF-6EE7-E39C-9A4F-F8A1_c696e201df24e15f44991fa3fb124b1411ea4512_hour_2017-04-21-00_00_00_2017-04-28 23_59_00.csv (63.3 KB)


#14

Thanks, I see the matching chart and CSV data now, and I do see the drops at 2:00 each day.

Would it be possible for you to capture the Arduino IDE serial monitor output while one of these spikes are occurring? With #define CAYENNE_DEBUG uncommented, so we can see the full log? This way we can see if the low values are coming from the Sensor/Arduino to Cayenne (and we are just logging what we get), or if we’re getting different values and then corrupting them on our end somehow.


#15

Hi, here is just example - serial monitor capture using CoolTerm (I do not know how to capture Arduino IDE serial monitor into text file). If this is sufficient, I will try to run it over night, to see critical time around 2 a.m. One thing I had to do was adding code to Serial.print value of photoresistor, as it was not in the original code, because I added photoresistor directly at widget dashboard by specifying analogpin. (so reading nor serial.print was not in original code)

here is serial monitor file:
CoolTerm.txt (94.0 KB)

and here is my code:

CayenneESPNodeMCU_Vcely_Ul_Test_debug - kópia.txt (3.1 KB)


#16

I don’t see the attachments. You might need to log into the forum at http://community.mydevices.com/ rather than replying via email.


#17

my last post edited - files uploaded. Can be checked…


#18

FYI I’m seeing the same issue. Right at the top of the hour I sometimes get erroneous data. This problem was happening a couple weeks ago and I thought it was just my sensor, then it was good for like 2 weeks. Now in the last couple days it started back up again. I’ll get my print logs for ya.


#19

Here are my raw data logged over night. All in all 9 hours, data each one second:
serial log up to 3am.txt (2.5 MB)
Serial log 3am to 6am.txt (2.6 MB)
Serial log 6am to 8 40am.txt (2.3 MB)

and chart from that time:


#20

Question: “finest” time period of downloaded data is one hour. How Cayenne agregates data into reported hour? Is for example value 968 for reported 3 am average from 2:00:00 up to 2:59:59? What is the formula for time data agregation?
Thanx for your help