Device Offline Again

Hello,

Again my Pi3 is appearing offline from both the Web Console and the Android App. I have rebooted the pi several times and checked the services by ssh into the Pi. All seems ok. I am not running any MQTT just basic triggers. This is very frustrating as alerts were not going through and the offline status kept my pool heater running to ridiculous temperatures.

Is there any stability insight for Cayenne?

Thanks,

Adam

Hi @walters,

I ran into someone yesterday who mentioned that when they saw this condition, running:

service myDevices stop
then
service myDevices start

brought the device back online. Iā€™m not suggesting this is an acceptable fix, just that it would narrow down the connectivity issue to the Pi agent locking up or failing in some manner, which is then resolved by restarting it. Iā€™d be curious if you see the same behavior.

Iā€™ll mention much the same as I did in that thread that weā€™ve almost got a new agent out the door (in the next week or two) It might have this issue or not, though we havenā€™t seen it in testing. For that reason its unlikely weā€™re going to bugfix the one you currently have installed, but Iā€™d be happy to give you early access to the one weā€™re testing now to see if it behaves a little better in this regard.

Hello @rsiegel

The services to not seem to bring it back online. They do seem to communicate for a second however. I can tell this because the temperature gauges in the dashboard update with the current values.

Yes, i can test out the new client here. I have a couple of Pi3 on the bench that i can play around with. Also note that both of my devices currently show offline so Iā€™m not sure if its something server side?

Do you have any change logs or documentation on the new client?

Thanks,

Adam

Iā€™ll have better specific bullet points/bugfixes for the new client when we officially release it, and I make a post in the Announcements category.

That said, big goals of this update were to:

  • Provide support for the 4.9 Linux kernel (and 4.4 too for those running older machines)
  • Integrate the webiopi component of the client that used to be its own standalone software into the client, so there is only one piece to install and uninstall.
  • The current pi agent runs as root, this new one will run as a ā€˜cayenneā€™ user which has more locked down permissions, improving the security risk any exploitable bug in the software could cause.

One of the nice side benefits once you have this installed is that you donā€™t need to use the command line to uninstall anymore, just removing the device on the web dashboard should remove the software without leaving bits behind, including logs. On the topic of logs, there is now only one and it lives at /var/log/myDevices/cayenne.log.

It could be that if there was a performance issue locking up your agent, it still exists in this new build, or it has gone away with the refactoring. But better to test against this new agent, since there will be active interest in the bug from our development team should it persist in this new agent.

Lastly, just a fair warning that this is pre-release software. Weā€™re getting awfully close to releasing it so I donā€™t think itā€™s going to melt your Pi or anything, but donā€™t run it on anything super mission critical. :slight_smile: The one gotcha that Iā€™m aware exists right now is that it doesnā€™t play nice with our current Android build, so donā€™t install it if you care about interacting with that Pi from Android. Weā€™ll be updating Android first to prepare it for this agent, thatā€™s actually the biggest holdup from releasing this as an automatic update at the moment.

With those caveats, to try it out:

First get the test agent install script:
wget http://updates.mydevices.com/raspberry/rpi_testing.sh

If updating an existing installation, run:
sudo bash rpi_testing.sh -update

If this is a new install get the invite code from adding a new Raspberry Pi via the Cayenne Dashboard. The invite code is part of the install file name listed in the Terminal/SSH instructions: rpi_[invitecode].sh. Run the rpi_testing.sh script with this invite code:

sudo bash rpi_testing.sh -code [invite code]

Got same issue today, just restarted service right now and it went online again.

This problem didnā€™t happen some time ago, and stability is very important to this kind of IoT platform, we need to trust it to use it. Please make some (more) effort on a definitive fix.

Thank you.

If the service restart on your Pi is bringing you back online, this strikes me as an issue with the agent software rather than on our server side. You may want to consider the test agent I posted above (if youā€™re comfortable with the caveats) or wait a few more weeks for the official release.

Point taken that the platform needs to be stable, regardless of where the failure is happening, so weā€™ll continue to work hard to ensure that is the case.

I will install the new agent in next day or so also but was wondering if Cayenne recommends a particular version of Jessie OS or if we should just run upgrade and install commands. Or perhaps Cayenne could provide a specific image for download to keep testing consistent ( my main aim is IOT so donā€™t require any other software )

I updated the software - within 24 hrs the standard dash went offline but the MQTTR dash is still updating

We are experiencing the same problem over here. The device (Pi 3) is simply offline. We tried the service stop and start as well as rebooting the Pi. Is there anything else we could try?

@adriane.elinski, was this with the new client I posted in this thread, or the original one?

@shovelheadman ā€“ sorry to hear that the new client didnā€™t resolve this for you. If youā€™re still in that offline state with the pi, could you please share the output of service myDevices status and email me at support@mydevices.com with a copy of /var/log/myDevices/cayenne.log from that pi?

My raspberry Pi is also going offline quite frequently for several hours. Same goes for 2 of my friends Pis.
Any ideas?

Presuming youā€™ve tried that new agent, my current suspicion is that this is on our server side, and I do apologize for that, the platform needs to be stable. I donā€™t think its anything agent or Pi related since weā€™ve had some people in other threads mentioning similar behavior from Arduino / ESP8266 devices.

We have our dev/IT guys investigating it and Iā€™ll update as soon as I have more information.

1 Like

@rsiegel I have sent the log file as requested

Thanks, this is much appreciated.

@rsiegel Hi - anything that perhaps jumped out at you from the log?

(He is out today and should be back tomorrow :slight_smile: )

Apologies that I didnā€™t update a few days ago. I donā€™t see anything strange in the log, and at this point we suspect the issue is server side and not anything wrong with your specific Pi. Iā€™ll update as soon as our developers have had some time to investigate the general issues with offline devices that users have been seeing.

Replying to myself to say that this old test agent is no longer necessary as weā€™ve now updated our Pi agent to support the 4.9 kernel (it still supports 4.4 as well). The update was automatic in the background so there is nothing you need to do on your end. Feel free to use rpi-update to move forward to the 4.9 kernel at this point if youā€™d like!

Hi,

Iā€™m having the same issues as above with the device showing offline on the dashboard. I had previously used the same Pi for prolonged periods of time with no issues. Even though it is showing offline on the Cayenne dashboard, i can use the internet on the actual device itself. Could you please offer some advice?

Matt

I think this might be a browser state issue, Iā€™ve seen occasional ā€œofflineā€ banners incorrectly that were resolved after a force refresh or two. The next time you see this, Iā€™d be curious if you can open your account in another browser, and if it shows offline there too, or not.