Device offline in cayenne

I also assume some downtime yesterday, but now is working fine. I see that it takes like a minute to the dashboard to show that the devices are active as well as the last data package sent and etc.

@rsiegel Was working great for a couple days but the issues seems to reoccurred this morning. The device shows offline sporadically.

Please let me know if there is anything i can do to help? Curious is it a server load problem with a growing Cayenne user base? Where are your servers hosted, what type of infrastructure does Cayenne use?

Thanks,

Adam

As mentioned in some other threads, our server cloud.mydevices.com was throwing 400 and 500 errors this morning and this is likely the explanation for the behavior seen by yourself and @aicraga this weekend as well. Weā€™ve already make a fix today that we believe should put an end to this issue. Our servers are hosted on AWS ā€“ I donā€™t believe it is a load issue rather a configuration issue with the system in question. Again thank you for your reports and patience as we stabilize this system.

Yet on the same condition:

Last known status: 2 days ago (Jul 9, 2017, 2:40 am)

Rpi was rebooted more than once and I just logged Cayenne off and on again.

Could you share your output from:

service myDevices status?

ā— myDevices.service - myDevices service
Loaded: loaded (/etc/systemd/system/myDevices.service; enabled)
Active: failed (Result: start-limit) since Seg 2017-07-10 22:59:04 BRT; 39s ago
Process: 26118 ExecStart=/usr/bin/python3 -m myDevices -P /var/run/myDevices.pid (code=killed, signal=ABRT)
Main PID: 26118 (code=killed, signal=ABRT)

2017-07-10 19:33 GMT-03:00 Rsiegel <mydevices@discoursemail.commailto:mydevices@discoursemail.com>:
[https://sea1.discourse-cdn.com/business5/user_avatar/community.mydevices.com/rsiegel/45/1305_1.png] rsiegelhttp://community.mydevices.com/u/rsiegel Technical Product Manager
July 10

Could you share your output from:

service myDevices status?

From this, I donā€™t think you ran into any issue on our server side as happened further up this thread, it looks instead like the myDevices service has failed to start.

You can try to restart it with sudo service myDevices start. When it is running properly, service myDevices status should show (running) similar to this:

I suspect it may fail again, potentially even immediately, if itā€™s failing even after reboots, however. If thatā€™s the case, let me know the output from systemctl status myDevices to see if it yields any more information about what is happening here.

Hi !

As you suspect, even after the ā€œstartā€ command the result was ā€œfailedā€ again, so that is the ā€œsystemctlā€ output:

systemctl status myDevices
ā— myDevices.service - myDevices service
Loaded: loaded (/etc/systemd/system/myDevices.service; enabled)
Active: failed (Result: start-limit) since Ter 2017-07-11 21:13:25 BRT; 35s ago
Process: 3984 ExecStart=/usr/bin/python3 -m myDevices -P /var/run/myDevices.pid (code=killed, signal=ABRT)
Main PID: 3984 (code=killed, signal=ABRT)

2017-07-11 12:41 GMT-03:00 Rsiegel <mydevices@discoursemail.commailto:mydevices@discoursemail.com>:
[https://sea1.discourse-cdn.com/business5/user_avatar/community.mydevices.com/rsiegel/45/1305_1.png] rsiegelhttp://community.mydevices.com/u/rsiegel Technical Product Manager
July 11

From this, I donā€™t think you ran into any issue on our server side as happened further up this thread, it looks instead like the myDevices service has failed to start.

You can try to restart it with sudo service myDevices start. When it is running properly, service myDevices status should show (running) similar to this:

[https://sea1.discourse-cdn.com/business5/uploads/mydevices/optimized/2X/4/44a3c276d4d299d6042d087b54dd8db4b1de087b_1_690x125.png]https://community.mydevices.com/uploads/short-url/9NdjVlATg793JdgkLwjpWWUjyOL.png

I suspect it may fail again, potentially even immediately, if itā€™s failing even after reboots, however. If thatā€™s the case, let me know the output from systemctl status myDevices to see if it yields any more information about what is happening here.

Did you restore some files from the server this last weekend?

Iā€™m asking because some time ago Iā€™ve added a new widget for my raspberry and deleted the old one because that was not working, so I think I could be using a Cayenne key distinct that Iā€™d be using (if some old key was restored).

Is that a good theory?

I donā€™t think we did anything like that.

To address the output you posted yesterday however, this looks to me like the myDevices service is just failing at startup on that pi (for whatever reason). The start-limit part tells me that systemd tried to start it several times with the same result before giving up. This meshes with the fact that your manual start didnā€™t work either.

Do you mind uninstalling/re-installing Cayenne on that Pi to ensure a good installation before further troubleshooting? Iā€™ve not really seen a case before where this service fails to run, and suspect itā€™s either a bad install, or a bad state of the PI (something weā€™re expecting to be there that isnā€™t). It would be easiest to rule out the former before focusing on the latter.

To uninstall Cayenne, you can run these two commands on the Pi:

sudo /etc/myDevices/uninstall/./uninstall.sh
then:
sudo /etc/webiopi/uninstall/./uninstall.sh

Then you can remove the device from your dashboard and re-install from Add New > Device/Widget > Single Board Computers > Raspberry Pi

If you have a lot of customization on that Pis dashboard already, let me know and I can show you steps to re-install with the same invite code so you donā€™t have to create a new entry.

Tried to reinstall using a fake account and got a Python ref. error:

(ā€œQua Jul 12 19:09:53 BRT 2017 Installing software components
Qua Jul 12 19:09:54 BRT 2017 Cannot install for Python 3.4.2 : missing development headers\n
Qua Jul 12 19:09:54 BRT 2017 Finished agent code
Qua Jul 12 19:09:54 BRT 2017 ERROR: myDevices cannot be installed - please check errors aboveā€)

After reinstalling Python Cayenne install was ok, but as I could have more software problems, decided to restore my last backup. Iā€™ve lost my last changes on some scripts but solved this problem by now.

Thank you a lot for your attention!

2017-07-12 18:40 GMT-03:00 Rsiegel <mydevices@discoursemail.commailto:mydevices@discoursemail.com>:
[https://sea1.discourse-cdn.com/business5/user_avatar/community.mydevices.com/rsiegel/45/1305_1.png] rsiegelhttp://community.mydevices.com/u/rsiegel Technical Product Manager
July 12

I donā€™t think we did anything like that.

To address the output you posted yesterday however, this looks to me like the myDevices service is just failing at startup on that pi (for whatever reason). The start-limit part tells me that systemd tried to start it several times with the same result before giving up. This meshes with the fact that your manual start didnā€™t work either.

Do you mind uninstalling/re-installing Cayenne on that Pi to ensure a good installation before further troubleshooting? Iā€™ve not really seen a case before where this service fails to run, and suspect itā€™s either a bad install, or a bad state of the PI (something weā€™re expecting to be there that isnā€™t). It would be easiest to rule out the former before focusing on the latter.

To uninstall Cayenne, you can run these two commands on the Pi:

sudo /etc/myDevices/uninstall/./uninstall.sh
then:
sudo /etc/webiopi/uninstall/./uninstall.sh

Then you can remove the device from your dashboard and re-install from Add New > Device/Widget > Single Board Computers > Raspberry Pi

If you have a lot of customization on that Pis dashboard already, let me know and I can show you steps to re-install with the same invite code so you donā€™t have to create a new entry.

1 Like

Thanks for the update! Glad to hear you were able to get up and running with your restore from backup.

We donā€™t really need to troubleshoot it in that case, but that Python error likely indicated why you had the service which keeps you online failing at startup. If it ever starts happening again let me know ā€“ I have a few things you can try.

@mechbuster2

Just wanted to bump this thread to note that weā€™ve now updated our Pi agent to support the 4.9 kernel (it still supports 4.4 as well). The update was automatic in the background so there is nothing you need to do on your end. Feel free to use rpi-update to move forward to the 4.9 kernel at this point if youā€™d like!

Hi rsiegel,

I am having this same problem with my Raspberry Pi3-B going Offline after Cayenne is installed on my Android phone. Since I installed Raspbian a few weeks ago and had the same problem it looks like this problem was not fixed.

Setup: Pi3 B, OS=Stretch v4.9, 32GB Micro-SD

Can you tell me if there is an update for Cayenne that fixes this problem? If so can you include the link in your reply.

Rick R

Hi Rick,

Iā€™m not aware of any more recent update for this issue (and in fact, updates to our Cayenne Pi agent software would happen automatically provided you already have internet connectivity).

I donā€™t think installing the Android app shouldnā€™t make any impact on this one way or another, since if you already have the Pi agent software installed from the web, the Android app is basically just viewing/interacting with the install already present on your Pi. The next time your Pi is in this offline state in Cayenne, could you verify that it otherwise has internet connectivity, and let me know the output from these two commands?

service myDevices status
service webiopi status

This may be a case of 2 Cayenne accounts and the device is switching between the accounts. Can you verify you are using the same account when you log in using your browser and your android app?

Hi everyone,

My raspberry pi zero went offline (it was online for appr. a year without any problem). After several restarts, I tried to uninstall and reinstall without success, still offline. IP connection is ok, but I get the following output if I check myDevices service status:

ā— myDevices.service - myDevices service
Loaded: loaded (/etc/systemd/system/myDevices.service; enabled)
Active: active (running) since Sun 2018-06-17 21:44:01 CEST; 30s ago
Main PID: 1208 (python3)
CGroup: /system.slice/myDevices.service
ā€£ 1208 /usr/bin/python3 -m myDevices -P /var/run/myDevices.pid

Jun 17 21:44:01 raspberrypizero systemd[1]: Started myDevices service.
Jun 17 21:44:01 raspberrypizero systemd[1208]: pam_exec(login:session): conversation failed
Jun 17 21:44:01 raspberrypizero systemd[1208]: pam_unix(login:session): session opened for user root by (uid=0)

In Network.ini file:
2018-06-17 17:26:25 - myDevices - INFO - Daemon failure handling8
2018-06-17 17:26:25 - myDevices - ERROR - Start failed: cloud.mydevices.com:8181 Error:EOF occurred in violation of protocol (_ssl.c:600)

Can anyone help on this issue?

@jburhenn tagging for this.

@tamas.dekany76 It looks like you are running the older Pi agent. You will need to update to the latest Pi agent. You can do this by adding a new Rasberry Pi via the web dashboard. If you log in to the dashboard for your current agent it should show a message with some instructions for updating.

2 Likes

Hi! Thanks for the advice. I have updated the agent and it is working now. I thought the old version remains in parallel until the most important features are developed in the new version. My pi is primary an irrigation controller, so I cannot use it until a newer sw is released. As a workaround I solved the scheduling via crontab. Waiting for the new features :). Cayenne is a great tool, thanks for the team!

1 Like