Rate Limits Question


#1

Finally have most things working as they should, now looking to steam line my message rates etc, however I don’t really understand this -

Clients may send no more than 60 messages per IP address per minute.
Clients may attempt no more than 50 connections every 10 minutes per IP address.

is this LAN IP or WAN IP?
is 1 message 1 virtual write?
What is the difference between messages and connections?

Sorry if this all sounds dumb, certain I am well over these, looking to improve things.

I have already read a similar thread but still not 100%, if i have 5 devices connected, is that rate divided by devices or per device or does this depend on WAN IP?

Thanks


#2

Hi @paulminize,

Here’s a bit more details on the rate limits. The IP address we refer is to the IP provided by your Internet Service Provider (ISP). One message is considered one virtual write per device. The difference between connections and messages. A device will maintain one connection to our platform regardless of the amount of messages it will send. However, we limit how many connections one particular device attempts within a time period, because it could be considered an abuse, or something at the device level is not right (hardware or software).

If all your devices uses the same IP, then the connections limit will apply to all of them at once. Although, message limits is apply per device basis.

Hope that helps

  • Adrian

#3

Hi Adrian, Thanks for the reply, I thought I understood clearer, but now I think I am even more confused…

So is the reality that although it says per IP for the messages that it is per device - so If I have 1 devices and 10 channels I can virtualwrite each channel every 10 seconds for total of 60 messages?

Onto the connections Can you give me a worked example - lets say I have 5 devices with 10 channels each.

Sorry, I want to get it right as I don’t want to be abusing the service and I think for many months now I am way over some of these limits - in the earliest days for example I had virtualwrites in the loop without a timer :astonished:

Thanks again for your time


#4

Hi @paulminize,

Yes, if you have 1 device with 10 virtualWrites you can call the virtualWrites every 10 seconds.

If you have 5 devices with 10 virtualWrites on one user account you can also call them every 10 seconds. When it refers to “50 connections” it is referring to disconnecting/reconnecting, so if a device gets into some error loop where it is constantly disconnecting/reconnecting it will eventually be blocked for that 10 minute period. But 5 devices making a single connection each should not be a problem.

For more info about making sure your code is sending messages under the rate limit you can check this topic: Sending MQTT messages within rate limits

And I agree that help file entry referencing IPs is a little confusing. We may need to update that to be a bit more clear.


#5

Hey that makes perfect sense now, thanks.

I will streamline the send rates and see where I can get to.

I had 58 sensors over 5 devices sending every 10 secs :astonished:

Massive lumps of those were sent too often - I expect (hope) they never will change state, but when they do the 10 second timer would delay things (Which could be bad)

It might be worth encouraging a more proactive approach to this issue with some worked examples
maybe mentioning in the documentation some common sense approaches to ‘appropriate’ intervals - for example all the documentation is in relation to putting the virtualwrites inside a timer, in the real world virtualwrites could be tied to state changes, and thus only sent when absolutely needed, a state change could be far less infrequent in comparison to sending unchanged data every 10 seconds.

A fire sensor needs to be immediate and thus state activated, but temperature for example could be sent regularly on a timer - no need to send it 60 times a minute though, every 5 minutes or even even 6 hours could be enough for many applications.

I think with the examples all being timers it could lead newbies and non tech users down a inefficient route, not to mention 6 months ago I didn’t even understand how to use a timer

Sorry to ramble, but as a non tech user I think I may have some useful points to add - basically the parts I struggled with over the past 6 months.

Cheers for you help :+1:

Just found this and thought it was applicable -

Clearest explanations I have found so far


#6

Good point about only sending data when it changes. I think I’ll add that to that rate limit message FAQ.


#7

I wonder how that would affect graphs?


#8

Yeah, it could end up making graphs look a little weird. I’ve added a note about that in that FAQ.


#9

This works for me: Digital sensor almost instant reaction with writes only on change, temperature at 1 min intervals. As usual, code bits borrowed from this site.
Feel free to correct any errors.
John

unsigned long lastMillis = 0;
int previousState = -1;
int currentState = -1;

void setup() {
  Serial.begin(9600);
  delay(5000); 
  Cayenne.begin(username, password, clientID, ssid, wifiPassword);
  
  pinMode(12, OUTPUT);
  dht1.begin();
  sensor_t sensor;
}

void loop() {
  Cayenne.loop();
  
  // Publish data every 60 seconds (60000 milliseconds). Change this value to publish at a different interval.
    if (millis() - lastMillis > 60000) { 
    lastMillis = millis();  
  
    sensors_event_t event; 
     
    Cayenne.virtualWrite(1, dht1.readTemperature(true),"temp","f");
    Cayenne.virtualWrite(2, dht1.readHumidity(),"rel_hum","p"); 
  }
  // Check the sensor state and send data when it changes.
  {  currentState = digitalRead(16);
    if (currentState != previousState)
  
  { Cayenne.virtualWrite(16, digitalRead(16), "digital_sensor", "d");   // status
    previousState = currentState;
  }
 }
   
}

// This function is called when data is sent from Cayenne.
CAYENNE_IN(VIRTUAL_CHANNEL)
{
  int value = getValue.asInt();
   // Write the value received to the digital pin.
  digitalWrite(12, value);
}

#10

I was thinking more for fire / flood alerts, I guess the odd looking graph is the least of worries in this situation - :open_mouth:

Obviously for other data I can imagine the graph would / could be odd looking

In relation to optimizing the send rates, I have just completed this and the numbers look like this -

Before: 58 Channels wrote every 10 secs = 348 Per minute / 20880 Per Hour / 501120 Per Day - (Half a million) :astonished:

After: 58 Channels around 55 writes Per minute / 3521 Per Hour / 84520 Per Day (Still Astonished!)

A huge improvement I think and absolutely worthwhile, the other benefit being much quicker response to those channels that are now state based and not behind a timer.


#11

Yes I am now using same system, I don’t think any individual approach is ‘correct’ it is about considering the appropriate needs for the project, and more than likely combining a combination as you have shown.