I found an issue with data retention and it took me quite a while to figure out when exactly it happens.
Using MQTT my data used to be stored just fine until an update earlier this year (not sure when exactly, probably between march and june 2017, maybe earlier).
Before the update, all data would be retained automatically. Now the ‘retain’ flag in the MQTT header has to be set. So far so good. It works when I test it using MQTT.fx to manually submit messages.
BUT: only if the channel in the topic is an integer. If I use any other name for the channel number it gets published and displayed fine but the data is not retained. This used to work fine prior to the upgrade.
Example:
topic: “v1/username/things/clientID/data/2” data: “temp,c=12” → works fine (if retain flag is set)
topic: “v1/username/things/clientID/data/Sensor1” data: “temp,c=12” → does not work (channel shows in dashboard but data is not retained)
The only workaroud is to only use integer channel names. Can this be fixed?
I was actually under the impression that only integers were allowed for MQTT channels. @rsiegel was there a change recently that would change the behavior here?
Hi, sorry to jump on this, but I’m new to cayenne, and to mqtt, and to hacking about with IoT stuff in general.
I’m experiencing a similar (the same?) problem - using mqtt for python from an RPi, and naming my widgets, I have no persistence, which for my project is pretty much the whole point…but I don’t see any change when renaming to integers…
Should I be raising a new topic, or can we aim for two birds with one stone here?
IF with the latest version only integers are allowed then this should be stated in the documentation. Or did I miss it?
It certainly did work before, in fact I have screenshots of it working back in january on my blog: [ESP8266 & Cayenne – Bäschteler of Science Blog]
In the code there I am sending a string as the channel and in the screenshots you can see how it works as a graph, i.e. data is retained.
I’ll say that I’ve never considered using anything other than integers here (to be clear, for the ‘Channel’ field – you can name the widgets whatever you want in the ‘Widget Name’ field in the Cayenne UI of course)
That said, I don’t know if its a technical limitation or just something I hadn’t considered trying. I know that MQTT as a protocol is OK with strings, but maybe there is something in our implementation that limits us to integers here. I’m tagging @eptak and @jburhenn to see if they know more.
Just to clarify, when you mention retained data are you referring to the historical data that is shown in the charts? If so, that should actually be unrelated to the retain flag, which is just an MQTT feature to retain the last MQTT message sent for a topic, so you can receive it the next time you subscribe to the topic.
If you are referring to the historical data not being saved then that could be a backend issue, or potentially as-designed since we’ve only really used integer channels before. Perhaps it only happened to work with string channels before by luck. I’m not too familiar with the backend code so I’m not sure the answer. Adding @asanchezdelc1, in case he would know.
Might be worth updating the docs for Bring Your Own Thing API. It doesn’t really specify that you should be using an integer, but all the examples use integers which is probably why I assumed that’s all you could use.
Thanks for looking into this.
By retain I actually mean the storing of the data and the graph display. When I tested that using MQTT.fx I thought it only worked with the ‘retain’ flag set in the MQTT message but I may be wrong.
Using alphanumeric channel names it does work actually i.e. the widget is created and displayed just none of the values are stored. When the widget page is reloaded, all values are blank until the next update arrives.