Bugs with multiple devices in single dash board on same internet connection


#1

This is more of a query to see if others have seen similar behaviour

I have a number of Arduino based devices with ethernet connected at test sites and sharing a connection to the internet

One site has a device with no built in MAC and uses the cayenne generated one - The other site has proper unique MAC’s so i dont think thats the issue. All units have unique ID’s of course

The units on a site all have the same functions attached to them and use a pretty use standard cayenne - i have some additional code handing a few virtual pins in and out

If i only run ONE of the devices on a given dashboard them seem to behave perfectly - the more devices attached to a dashboard the flakier the whole thing becomes to the point where it isnt stable enough to trust

Both sites exhibit the same/similar behaviour where the devices go “offline” but the “uptime” value i have in the dash board keeps incrementing which means they ARE online but stop responding to commands i send to them

Also one site the slide keeps resetting to ZERO randomly which is annoying…

ideas?


#2

Extra info to go with this

4 identical dashboards - with identical triggers

Different results/ trigger reliability on the SAME hardware device depending on the cayenne token used in the device

Looks to me like a dashboard/backend issue


#3

More issues - devices showing ONLINE that are OFFLINE and vice versa

I’m seeing this across multiple arduino platforms/dashboards

Something really seems to be screwy at the moment!

@bennyestes11? any ideas?


#4

Still having this issue - device goes offline but is still connected at the device end and still sending data which shows up in the dashboard - commands back to device wont work as its "offline"
Restart of device doesnt bring it back online even though serial monitor output of device says its got an ip and is connected and is sending data

Anyone?!?!?!?!’

About to chuck Cayenne as a platform due to inability to get it working stable


#5

@bestes???


#6

Sorry for the delay and frustrations. Can you share your code? Maybe we can try to reproduce what’s going on.


#7

Its pretty much just the std code plus a few bits - the device is still sending data and shows as CONNECTED but the dashboard says offline - completely random! Can go for minutes hours or days just fine. A device power cycle MAY OR MAY NOT bring it back online in the dashboard!

//#define CAYENNE_DEBUG         // Uncomment to show debug messages
#define CAYENNE_PRINT Serial  // Comment this out to disable prints and save space
#include <CayenneEthernet.h>
#include "SPI.h"
#include <SimpleTimer.h>
#include <avr/wdt.h>
#include <avr/pgmspace.h>
#include <SoftReset.h>
#include <EEPROMAnything.h>
#include <PinChangeInt.h>


#define W5100_ETHERNET_SHIELD 

// Cayenne authentication token. This should be obtained from the Cayenne Dashboard.
char token[] = "xxxxxxxxxx";

#define SONAR_PWR   9

int     i = 0;
int     Sresult = 0;
int     lastLevel = 0;

double  tankHeight = 240;

byte KTA_mac[] = {0xff, 0xff, 0xff, 0xff, 0xff, 0xff};

boolean firstConnect = true;
boolean stringComplete = false;
boolean rainRESET = false;
boolean sonarPresent = false;

volatile uint16_t rainPulse=0;

SimpleTimer timer;

void setup(){
  pinMode(SONAR_PWR,OUTPUT);
  pinMode(10,OUTPUT);
  pinMode(A5, OUTPUT);
  delay(2000);
  Serial.begin(9600);

  delay(5000);

  Serial.print(F("Token = "));Serial.println(token);
  
  EEPROM_readAnything(1000,tankHeight);
  Serial.print(F("Tank height setting = "));Serial.println(tankHeight,0);
  Serial.println();
  delay(2000);
  //SONAR DETECTION ROUTINE
  digitalWrite(SONAR_PWR,HIGH);
  digitalWrite(A5, LOW);
  delay(1000);
  if (Serial.find("MaxBotix")){
    sonarPresent = true;
    flushBuffer();
  }
  else {
    sonarPresent = false;
  }
  digitalWrite(A5, HIGH);
  digitalWrite(SONAR_PWR,LOW);
  Serial.println();
  delay(500);
  if (sonarPresent == true){
     Serial.println(F("Sonar Connected"));
  }
  else{
    Serial.println(F("Sonar Not Present!"));
  }
  //
  
  // Raingauge interrupt pin setup
  attachPinChangeInterrupt(A4, rainCount, FALLING);
  //

  // Timer configuration
  timer.setInterval(60000, sendUptime);                         // send uptime every minute
  if (sonarPresent == true){                                    // IF sonar is connected
    timer.setInterval(300000, sendLevel);                       // send level every 5 minutes
  }
  //
  //
  
  // read out MAC address  
  SPI.begin();
  for (i=0; i<6; i++) {
    KTA_mac[i] = MAC_read(0xfa + i);
  }
  Serial.println();  
  Serial.print("KTA323 MAC address is: ");
  Serial.print(KTA_mac[0], HEX);Serial.print("-");
  Serial.print(KTA_mac[1], HEX);Serial.print("-");
  Serial.print(KTA_mac[2], HEX);Serial.print("-");
  Serial.print(KTA_mac[3], HEX);Serial.print("-");
  Serial.print(KTA_mac[4], HEX);Serial.print("-");
  Serial.println(KTA_mac[5], HEX); Serial.println();  
  //

  // Start CAYENNE  
  Cayenne.begin(token,KTA_mac);
  //
}

void loop(){
  wdt_reset();
  Cayenne.run();                                                // Runs main loop
  timer.run();                                                  // Initiates SimpleTimer
  if (millis() >= 259200000){                                  // reboot monthly (add a zero - current setting 3 days)
    Serial.println(F("Periodic Reboot..."));
    Serial.println();
    delay(1000);
    soft_restart();                                             // 
  }                                                             //  
}                                                               // END main loop    

CAYENNE_CONNECTED(){
  CAYENNE_LOG("Connection established");
  Serial.println(F("=> Connection OK"));
}

CAYENNE_DISCONNECTED(){
  CAYENNE_LOG("Connection Lost");
  Serial.println(F("=> Connection Lost"));
  //Relays OFF
  Serial.println(F("ALL OUTPUTS OFF"));
  digitalWrite(2,LOW);
  digitalWrite(3,LOW);
  digitalWrite(4,LOW);
  digitalWrite(5,LOW);
  digitalWrite(6,LOW);
  digitalWrite(7,LOW);
  digitalWrite(8,LOW);

}

CAYENNE_IN(V25){
  CAYENNE_LOG("V25 Value Received");
  tankHeight = getValue.asDouble();
  Serial.print(F("=> Tank height set to "));Serial.println(tankHeight,0);
  EEPROM_writeAnything(1000,tankHeight);
}

CAYENNE_IN(V24){
  CAYENNE_LOG("V24 Value Received");
  rainRESET = getValue.asInt();
  if (rainRESET == true){
    rainPulse = 0;
    Cayenne.virtualWrite(V4,(rainPulse*0.2));
    Serial.println(F("=> Rain Gauge RESET"));
    rainRESET = false;
  }
}


void sendUptime(){
  Serial.println(F("=> Send Uptime"));
  Cayenne.virtualWrite(V5, millis() / 60000);
  float battVolts = (analogRead(0) * 0.0207);
  Cayenne.virtualWrite(V0,sonarPresent);
  Cayenne.virtualWrite(V1,battVolts);
  Cayenne.virtualWrite(V4,(rainPulse*0.2));
}

int MAC_read(byte addr) {                                     //Reads out bytes from the 25AA02E48 (MAC EEPROM)
  digitalWrite(10, HIGH);
  SPI.transfer(0x03);
  SPI.transfer(addr);
  byte data = SPI.transfer(0);
  digitalWrite(10, LOW);
  return data;
}  

void sendLevel(){
   Serial.println(F("=> Get Level"));
   digitalWrite(SONAR_PWR,HIGH);
   delay(1000);
   flushBuffer();
      lastLevel = Sresult;
      int range = EZread();
        if(stringComplete){
          stringComplete = false;                                //reset stringComplete ready for next reading
          range = range/10;
          range = range - 30;
          int tanklevel = tankHeight - range;
          int rangePercentage = map(tanklevel,0,tankHeight,0,100);
          Cayenne.virtualWrite(V8,rangePercentage);
          digitalWrite(A5,HIGH);
        }
        digitalWrite(SONAR_PWR,LOW);
        Serial.println();
}


int EZread(){
  Serial.println(F("=> Read Sonar"));
  digitalWrite(A5,LOW);
  char SonarData[5];                                          //char array to read data into
  int Sindex = 0;
  flushBuffer();
  delay(100);
  while (stringComplete == false){
      if (Serial.available()){
        char rByte = Serial.read();                     //read serial input for "R" to mark start of data
        if(rByte == 'R'){
          while (Sindex < 4){                                  //read next three character for range from sensor
            if (Serial.available()){
              //Serial.println(freeRam());
              SonarData[Sindex] = Serial.read();
              Sindex++;                                       // Increment where to write next
            } 
          }
        SonarData[Sindex] = 0x00;                              //add a padding byte at end for atoi() function
        }
      rByte = 0;                                           //reset the rByte ready for next reading
      Sindex = 0;                                           // Reset index ready for next reading
      stringComplete = true;                               // Set completion of read to true
      //Serial.println(SonarData);
      Sresult = atoi(SonarData);                               // Changes string data into an integer for use
      }
  }

  //Serial.print(F("Result = "));Serial.println(Sresult);
  
  if ((Sresult == 0) || (Sresult > 4999)){
    Serial.println(F("=> Level error!"));
    Sresult = lastLevel;
    return Sresult;
  }
  else return Sresult;
}

//=====================================
// Rain Gauge Interupt Rotuine
//=====================================
void rainCount(){
  rainPulse++;
}



void flushBuffer(){                                          // flushes serial buffer to avoid buffer overflow
  if (Serial.available()){
      while (Serial.available()){
       Serial.read();
      }
  }
}

#8

Enabling CAYENNE DEBUG isnt showing anything helpful - the output looks identical when its showing in the dash board and when the dash board says its OFFLINE


#9

Well the good (or bad) news is that I don’t see any of the usual suspects for getting an offline message. Generally sleeping or using delays of more than 2 seconds without calling cayenne.run will cause the issue. Can you post all the hardware you are using? I’ll try to match as close as possible and let it run at my house to see if I have any issues. @bestes @rsiegel Maybe someone at Cayenne can do the same?


#10

This is the device I’m using:

https://oceancontrols.com.au/KTA-323.html


#11

Yikes @tim2, so sorry for not getting back to you sooner. Are you using a single Arduino w/ multiple auth tokens? Or you are saying you are using multiple Arduinos, each w/ their own auth tokens?

If you have the time can we jump on a Skype call or Google hangouts so I can see the issue in real time and see how your dashboard is configured? You can pick any time on my calendar here :slight_smile: https://calendly.com/bestes/

~Benny


#12

Multiple devices with individual auth tokens


#13

I was having weird issues like this until some days ago. I was restarting my RPi every night and notice that was common to have problems after boots, so I stopped the restart procedure and Cayenne just started to work smoothly. Can be only a coincidence, I’m only at third test day by now.


#14

Sorry, that’s a bit over my budget for the “let’s just pick that up because it looks cool” column. I can still try to connect up a few Arduino’s to see if I have issues.


#15

Please keep us updated w/ how next days go :slight_smile:

~Benny


#16

Hi @bestes - not sure how we are going to coordinate you looking at this as its random! All I can tell you is that even when the dashboard says a device is offline the output of cayenne debug looks correct with the flow of messages continuing and the device indicating its online. SO frustrating!


#17

I’m curious if the next time you see a false ‘Offline’ state, if you could load the account in a different browser and let me know if that browser also sees the incorrect Offline state.


#18

Generally all browsers show the same offline devices at the same time – im waiting for the device that is connected to the debug computer atm the moment to go offline


#19

Ok – the device on debug has gone “offline” – 4 different browsers, 3 different computers, 2 different internet connections

The device on debug is still communicating an the cayenne debug out says its still connected – and while im typing this its come back online!

Another that was offline came back by itself and one that was online has gone offline! Frustration!