Internet / Database Clog?


Howdy, I have been trying to figure out why my data is getting delayed on the way to the cayenne database.

The picture below shows the data arriving from an 8266 device at about 9 AM MST. On the left in the green zone, the data arrives at normal 15 sec intervals, but then the blue zone, shows a minute or two between samples, then in the red zone, there is a burst of updates 2-5 seconds apart to catch up with all the delayed updates, and then the green zone gets back to normal with the 15 second updates.

Here is another example last night about 9:20 PM MST, where is the data is just a counter that increments with every 15 second update. In this example, you can see the long delays between updates, but there is also the problem that the 150836 value is stored in the database several times before the flood of delayed updates arrive, and then towards the end of the flood, several samples are lost or discarded. See the spreadsheet below for count 150842 and 43 which have disappeared. The spreadsheet also shows the gap between updates diverges from the normal 15-16 seconds. The yellow line just shows where the data should be. As far as I can see the updates are never out-of-order.

So my question is: where is the problem. I know my 8266 code does not re-transmit or delay transmissions. So it could be the basic internet software from the arduino http object, or the code within the cayenne arduino code which is delaying or retransmiting updates after it has lost connection or something. Or it could be general internet problems where my data is roaming the internet trying to find a path to the cayenne computer server. Or perhaps the server has a flood of data and queued up the updates for storage later??

Any ideas?

Another bit of anecdotal evidence is that my homemade android app does seem to timeout on requests for a single point update at about the same time as these blips in the data history.


Spreadsheet of data from first graph of 9 AM temperature data: