Using Node-RED as a Local Fallback Server

Required items:

Raspberry Pi with the newest image of Raspbian Jessie (Node-RED is pre-installed)
A client device (Arduino, Pi, or any other microcontroller that has an MQTT library)
Arduino IDE

Overview:

Cayenne will be a display for our data and easy to use interface.
The Raspberry Pi will be a local server for our client device to send data to. Through Node-RED we can create functions that are currently missing from Cayenne such as time delay buttons, GPIO interrupts, and add sensors that are currently not supported for Raspberry Pi.

Setup:
Cayenne:
Log in to your dashboard and click Add new…>Device/Widget
Click the Bring Your Own Thing button - keep this window open for later
That’s it for now!

Raspberry Pi server:
Install a local MQTT server using apt-get:

sudo apt-get install mosquitto

Set up Node-RED to start on boot using the command below:

sudo systemctl enable nodered.service

Now you can either start Node-RED by rebooting or using “sudo service nodered start”

The Node-RED web interface can be accessed by going to your Pi’s IP address and appending :1880 to the end (ex. http://192.168.0.10:1880)

For now we will create a flow to receive an MQTT command from Cayenne and another to send a value to Cayenne using MQTT. The very basic flow below can be imported in to Node-RED by clicking Settings (three lines in the top right)>Import>Clipboard then click where you want to place it on the flow tab.

[{"id":"bb4f8d87.5978a8","type":"mqtt in","z":"841dbd34.248ee8","name":"Cayenne Input","topic":"See Post","qos":"2","broker":"94fad99f.8e2728","x":120,"y":540,"wires":[["d8185bc2.e76a78","3369d01a.b20798"]]},{"id":"d8185bc2.e76a78","type":"debug","z":"841dbd34.248ee8","name":"","active":false,"console":"false","complete":"false","x":350,"y":500,"wires":[]},{"id":"56762e2e.625f48","type":"inject","z":"841dbd34.248ee8","name":"Send Temp Value of 20c","topic":"","payload":"temp,c=20","payloadType":"str","repeat":"","crontab":"","once":false,"x":150,"y":600,"wires":[["5d89c061.6d73"]]},{"id":"5d89c061.6d73","type":"mqtt out","z":"841dbd34.248ee8","name":"Cayenne Output Channel 2","topic":"See Post","qos":"0","retain":"true","broker":"94fad99f.8e2728","x":400,"y":600,"wires":[]},{"id":"6648df8b.44f7f8","type":"mqtt out","z":"841dbd34.248ee8","name":"Cayenne Output Channel 1","topic":"See Post","qos":"0","retain":"true","broker":"94fad99f.8e2728","x":780,"y":520,"wires":[]},{"id":"3369d01a.b20798","type":"function","z":"841dbd34.248ee8","name":"Send Back Received Value","func":"msg.payload = msg.payload.slice(-1);\n\nreturn msg;","outputs":1,"noerr":0,"x":400,"y":540,"wires":[["6648df8b.44f7f8","1656dfff.8cb4e8","30b1895d.a1218e"]]},{"id":"1656dfff.8cb4e8","type":"debug","z":"841dbd34.248ee8","name":"","active":true,"console":"false","complete":"false","x":730,"y":480,"wires":[]},{"id":"c42d2749.18218","type":"mqtt in","z":"841dbd34.248ee8","name":"Arduno Input","topic":"arduino_1_data","qos":"2","broker":"58cb32ce.d41f54","x":110,"y":660,"wires":[["919819a4.c4f0e8"]]},{"id":"919819a4.c4f0e8","type":"function","z":"841dbd34.248ee8","name":"Send Data Received From Client To Cayenne","func":"var MQTTusername = global.get(\"MQTTusername\");\nvar MQTTclientID = global.get(\"MQTTclientID\");\nvar msgarray = msg.payload.split(',');\nvar channel;\n\nfor (i = 0; i < msgarray.length; i++){\n    if (msgarray[i].match(/channel/gi)){\n        channel = msgarray[i].split('=')[1];\n        msgarray.splice(i, 1);\n    }\n}\n\nmsg.payload = msgarray.join(\",\");\nmsg.topic = \"v1/\" + MQTTusername + \"/things/\" + MQTTclientID + \"/data/\" + channel;\n\nreturn msg;","outputs":1,"noerr":0,"x":460,"y":660,"wires":[["e17e25f8.3d3508"]]},{"id":"e17e25f8.3d3508","type":"mqtt out","z":"841dbd34.248ee8","name":"Cayenne Output Arduino Input","topic":"","qos":"0","retain":"true","broker":"94fad99f.8e2728","x":790,"y":660,"wires":[]},{"id":"30b1895d.a1218e","type":"mqtt out","z":"841dbd34.248ee8","name":"Client Output","topic":"CayenneInput","qos":"0","retain":"true","broker":"58cb32ce.d41f54","x":730,"y":580,"wires":[]},{"id":"736cbb2b.50afec","type":"inject","z":"841dbd34.248ee8","name":"Setup","topic":"","payload":"","payloadType":"str","repeat":"","crontab":"","once":true,"x":90,"y":420,"wires":[["f1692282.cd432"]]},{"id":"f1692282.cd432","type":"function","z":"841dbd34.248ee8","name":"Set Global Variables","func":"global.set(\"MQTTusername\",\"See Post\");\nglobal.set(\"MQTTclientID\",\"See Post\");\n\nreturn msg;","outputs":1,"noerr":0,"x":380,"y":420,"wires":[["131e7e5a.1370ba"]]},{"id":"131e7e5a.1370ba","type":"debug","z":"841dbd34.248ee8","name":"","active":false,"console":"false","complete":"false","x":730,"y":420,"wires":[]},{"id":"94fad99f.8e2728","type":"mqtt-broker","z":"","broker":"mqtt.mydevices.com","port":"1883","clientid":"See Post","usetls":false,"compatmode":true,"keepalive":"60","cleansession":true,"willTopic":"","willQos":"0","willPayload":"","birthTopic":"","birthQos":"0","birthPayload":""},{"id":"58cb32ce.d41f54","type":"mqtt-broker","z":"","broker":"localhost","port":"1883","clientid":"","usetls":false,"compatmode":true,"keepalive":"60","cleansession":true,"willTopic":"","willQos":"0","willPayload":"","birthTopic":"","birthQos":"0","birthPayload":""}]

After placing the flow edit the Set Global Variables node with your MQTT Username/Password listed on your Cayenne dashboard for the device we added earlier.

Edit the Cayenne Input node by double clicking it. Change the topic to fit your settings, the channel should be 1. Below is the breakdown of what the topic means:

v1/username/things/clientID/cmd/channel
v1 - This is the API version which for now will always be v1
username - Change this to the MQTT username listed on your Cayenne dashboard for the device we added earlier
things - This can stay things - not exactly sure what this is
clientID - Change this to the Client ID listed on your Cayenne dashboard for the device we added earlier
cmd - This is the type of message.  cmd is a command message from the Cayenne server
channel -  This is where the data ends up on the dashboard.  Each widget must have a unique channel

In the end your Topic string should look like this:

v1/XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX/things/XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX/cmd/X

After the topic is set click on the pencil next to the server. On this page paste in your Client ID in the Client ID field then click the Security tab and paste in your username and password from the Cayenne dashboard.

Next edit the Cayenne Output Channel 1 node by double clicking it. Change the topic to fit your settings. You can use basically the same topic from the input node, but this time we want to change the message type to digital to tell the Cayenne server we received the command so it updates the button on the dashboard. The channel should be 1. Your Topic string should look like this:

v1/XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX/things/XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX/data/X

Finally, edit the Cayenne Output Channel 2 node by double clicking it. Make the same changes as the Cayenne Output Channel 1 node but notice the second to last parameter is data instead of digital this time. Sending the type data tells the server we are sending it a value to update a widget. The channel should be set to 2.

v1/XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX/things/XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX/data/X

After the 2 MQTT nodes are configured you can click Deploy to save the changes. At this point the MQTT settings page should go away and take you to a blank dashboard. You can test that communication to the server is working by clicking the blue button to the left of the Send Temp Value of 20c inject node. You should see a temporary widget appear on the blank dashboard. From here if you want to make it permanent just click the + in the upper right corner.

To test that values are being received from the server, click Add new…>Device/Widget>Custom Widgets>Button. Enter the following settings:

Name - Enter a name.  Button is a good choice
Device - Select your MQTT device
Channel - Enter a 1
Icon - Whatever your preference.  Light is usually the one I select

Click Add Widget then click on the button that shows up on the dashboard. Go back to Node-RED and click on the Debug tab in the top right. You should see two debug messages:

XXXXXXXXXXXXXXX,1 which means your message was received. 1 means the button was turned on and 0 means it was turned off. The first part of the message is a random sequence.
1 - 1 means the button was turned on and 0 means it was turned off. This is the value sent back to the server to verify the command was successful and update the button on the dashboard.

Client Device:
Arduino:
For now I’ll just go over configuring an ESP8266 device to send data to our local server (Raspberry Pi coming soon?)
Copy this sketch to your Arduino (I’m using an ESP8266 - you may have to include more libraries for your device). This sketch simply sends some hard coded values to your dashboard to verify everything is working. Note - you will have to change SSID to your wireless network name, PASSWORD to your wireless password, and SERVER_IP to the IP address of your Pi.

#include <PubSubClient.h>
#include <ESP8266WiFi.h>

const char* ssid = "SSID";
const char* password = "PASSWORD";
const char* mqtt_server = "SERVER_IP";

WiFiClient espClient;
PubSubClient client(espClient);
long lastMsg = 0;
char msg[50];
int value = 0;

void setup() {
  int timeoutcounter = 0;
  
  Serial.begin(9600);
  WiFi.begin(ssid, password);

  Serial.println();
  Serial.println();
  Serial.print("Connecting to ");
  Serial.println(ssid);
  
  while (WiFi.status() != WL_CONNECTED) {
    timeoutcounter++;
    delay(500);
    Serial.print(".");
    if (timeoutcounter >= 30){
      Serial.println("Failed to connect to wireless - sleeping for 5 minutes");
      delay(100);
      ESP.deepSleep(300000000, WAKE_RF_DEFAULT);
      delay(100);
    }
  }
  
  Serial.println("");
  Serial.println("WiFi connected");

  client.setServer(mqtt_server, 1883);
  client.setCallback(callback);
}

void reconnect() {
  // Loop until we're reconnected
  while (!client.connected()) {
    Serial.print("Attempting MQTT connection...");
    // Attempt to connect
    if (client.connect("ESP8266Client")) {
      Serial.println("connected");
      client.subscribe("CayenneInput");
    } else {
      Serial.print("failed, rc=");
      Serial.print(client.state());
      Serial.println(" try again in 5 seconds");
      // Sleep for 5 minutes before retrying
      Serial.println("Failed to connect to MQTT server - sleeping for 5 minutes");
      delay(100);
      ESP.deepSleep(300000000, WAKE_RF_DEFAULT);
      delay(100);
    }
  }
}

void callback(char* topic, byte* payload, unsigned int length) {
  Serial.print("Message arrived [");
  Serial.print(topic);
  Serial.print("] ");
  for (int i=0;i<length;i++) {
    Serial.print((char)payload[i]);
  }
  Serial.println();
}

void loop() {
  if (!client.connected()) {
    reconnect();
  }
  client.loop();

  float t = 20.2;
  float h = 30.3;
  float hif = 40.4;
  
  String mqtt_message = "temp,f=" + String(t) + ",channel=10";
  mqtt_message.toCharArray(msg, 50);

  Serial.print("Publish message: ");
  Serial.println(msg);
  client.publish("arduino_1_data", msg);

  mqtt_message = "rel_hum,rel_hum=" + String(h) + ",channel=11";
  mqtt_message.toCharArray(msg, 50);

  Serial.print("Publish message: ");
  Serial.println(msg);
  client.publish("arduino_1_data", msg);

  mqtt_message = "temp,f=" + String(hif) + ",channel=12";
  mqtt_message.toCharArray(msg, 50);

  Serial.print("Publish message: ");
  Serial.println(msg);
  client.publish("arduino_1_data", msg);
  delay(5000);
}

After you upload the code to your ESP you should see 3 new widgets on your MQTT device dashboard. Here you can click the + on the widget to add it permanently, or just use this as a validation that everything is working. From here you can modify the sketch to send any data you want to the dashboard. There are a couple important parts to sending data to the dashboard. If you are using my Node-RED flow, the string you send to the Node-RED server must have 3 comma separated values: message type, value, and channel (ex. “temp,f=20.2,channel=10”) For the value you must include the unit value from the list of data types in the Cayenne docs. Here is a list of the valid data types and units: http://www.cayenne-mydevices.com/docs/#bring-your-own-thing-api-supported-data-types

To receive data from Cayenne click the button we created earlier. This will send a 0 or a 1 to your ESP through the CayenneInput MQTT Topic. To act on the received data from Cayenne you will have to modify the callback function. In here you can do anything, put the device to sleep, toggle a GPIO pin, flash an LED, etc. The possibilities are endless. The one downfall you will notice is that the received data is only process once per loop. There are some ways around this, but I won’t get in to that now. I will most likely update this post to include these changes.

I’ll add to this post over time. Right now I have planned to add a section on using the Raspberry Pi with this setup and some other cool stuff like adding in your own features such as the time delay button. If there is anything you would like to see here just post a request.

9 Likes

@adam Fantastic informative post, thank you Adam! I want to use Cayenne as part of my dissertation but was worried that the criticism for the system would be that should the internet link fail the system wouldn’t function. This largely solves that issue, just a shame that Node Red seems to need quite a bit of learning compared to the easiness of setting things up in Cayenne!

Yes, Node-RED has quite the learning curve. I’ve been using it for about 2 years now and still learning about new ways to do things. If you need help with anything Node-RED let me know.

1 Like

Thanks @mike306dt for spotting the missing Set Global Variables part of the tutorial which cases the data to not make it back to the Cayenne server. I updated the project to include it.

Hi Adam,

Thank you very much for investing efforts into community education.

Couple of days I am struggling to resolve problems related with launching of nodered service. Mainly, problems are related to screwed nodejs environment on Raspbian Jessie v8. Googled got solution and tried tens of them. Nothing helps. Nodered seems impossible option on my R-Pi.

Is there a chance to connect Mosquitto broker with Cayenne without nodered?

As far as the node-red issues there are a few way to fix that. The simplest is to just get a fresh image of Raspbian on an SD card since the new version has Node-RED installed by default. You can also use the following commands to uninstall and reinstall NodeJS and Node-RED:

node-red-stop
sudo apt-get remove nodered
sudo apt-get remove nodejs nodejs-legacy
sudo apt-get remove npm   # if you installed npm
curl -sL https://deb.nodesource.com/setup_4.x | sudo bash -
sudo apt-get install -y build-essential python-rpi.gpio nodejs
sudo npm cache clean
sudo npm install -g --unsafe-perm  node-red

You can also send MQTT messages without Node-RED but the process to do that depends on what you plan to send them with.

@adam Excellent tutorial on this complex subject.

I’ve made it as far as producing the temp widget but not getting anything back from the Light Button …

The example topic for this is …

Note it states /data/X … and not digital.

What should it be?

Thanks.

it should be data you can find more info here Cayenne Docs

As Shramik said, it should be data as far as I know. These instructions could use a refresh though as they are from 3 years ago (time flies when you’re having fun!). In the next couple days I will walk through them and update anything that is incorrect. It might be easier if you want to connect on Slack.

2 Likes

Yip … Data it be.

All working aok, tho, have to refresh myD web page to get Light button widget to show actual status … any thoughts?

Thought perhaps my Brave browser shields up was blocking it … but nope. Tried ms Edge and it’s the same, … tho’ … have to admit it seems to work better on Edge. :grimacing:

Will now explore how to connect openhab to myD via mqtt via nr.

Wish me luck!

What is Slack?

http://slack.mydevices.com/

1 Like

@adam … yes time flies even when you’re not having fun :grimacing:

Good news is that I’ve got live data from my openhab server feeding into the nr flow … as follows …

It’s a setpoint value (degC) SP1a … … the question is … how do I ‘inject’ that value into the ‘Cayenne Output Channel 2’ node?

I assume it will have to be in the same format as the static inject node ‘Send Temp Value of 20c’ … ie …

Payload - temp,c=20 … where I want to replace the static 20 value with my SP1a live data.

How to do?

Pint of beer for the first correct answer. :slightly_smiling_face:

Sorry, I’ve been dealing with some problems with my 3d printer and haven’t had time to look at it. Can you connect on slack and send me an export of your flow? We can chat there about what to change and then post an update here when it’s fixed.

I have been trying to feed NR back from an MQTT Out button to a Cayenne data/ch input (even from a simple button on NR) but no luck so far. Is there a Simple demo anywhere ? The above examples are overly complex for all I need. I have tried the above but the latest NR version and buttons are slightly different. The closest I get is the connected buttons blinking on and off and cayenne dashboard going on and off line.

Can someone ‘export’ a working example of Inject > MQTT Output please ?
~ A

v1/******/things/+++++/response/2

spero ti sia utile, se poi riescia spiegarmi meglio ti aiuto come posso

Thanks, would this pass a data value or should the format be something like:
v1/XXXXXXXX/things/YYYYYYYY/data/X

Thank you for the help. Here is an example of a test I made up


The topic format is:
v1/username/things/clientID/data/1
To get it to work I had a spare BYOD project that I deleted all the channels from and when it started up it created the channel 1 data and type automatically.
The Inject msg. payload node was set to string and contained temp,c=20 and I set this to repeat every 5 seconds for testing.
There are quite a few other settings like qos and retain but these are blank at the moment.
I hope to use Node red to generate a processed data stream that will be sent back to Cayenne. The project in mind is to detect temperature fluctuations = penguin nesting in it’s burrow :slight_smile:
So far so good
~ A

1 Like

I can see now that adding the msg into the end of the topic is a bit messy. It half worked but is probably not recommended. I have found as above Manually adding A custom widget works best. The choice of custom widget gives more or less features. I recommend tinkering with a few different types of widget.
I can manually demo, test ideas like inject a complete message string e.g. “temp,c=12.3” etc to auto generate the widget and data in Cayenne. It would be nice to learn how to generating this multi part txt + Live data numeric string in the correct way using JSON. I would be interested in any suggestions.
If anyone is interested I can post the code so far and function node for the function node (penguin Max - Min temp = activity) detector.
Thanks
~ Andrew

Here’s the docs on sending as JSON: