[TUTORIAL] SigFox to Cayenne : how to?

cayenne
node-red
dashboard
sigfox
multi-sensor

#1

SigFox : what is it ?

Sigfox, has developed a low-bandwidth, energy-efficient cellular network for long-range communications.
A subscription of one to ten dollars / year can transmit up to 140 messages / days. (1 message = 12 Bytes max)
No need for wifi, or 4G, the Sigfox certified card with its antenna can emit up to 30 or 50km from a SigFox antenna.
They can be powered by battery and thanks to their low consumption, the autonomy, with two 18650 batteries, is between 2 and 3 years.
Data are sent to a global backend whatever the network provider you are passing through and you have contract with.This backend will take you back to your data in the future.

Advantages are :

  • it activates the online card on its SIgfox account and it can transmit the data of its object in 5 minutes.
  • cheap subscription: a few dollars / year.
  • very good range

The inconvenients:

  • can only send 140 messages / days = 1 message / 10minutes (1 message = 12 Bytes max)
  • can only receive 4 messages / day (1 message = 8 Bytes)
  • Some countries are not yet covered

This is ideal for a solar powered project. A water meter, electricity in a cellar where the wifi or 4G do not pass.
Supervise the presence sensors in a secondary house without electricity, a cottage, water leaks etc …
And when you want a plug and play system with a very good network hook.

A SigFox board idea: https://www.disk91.com/2017/news/technologies/snoc-sfm10r1-the-easiest-way-to-access-sigfox-for-makers/

what is this tutorial for? ?

In this tutorial, I will not describe where to find or how to connect a sigFox card to an arduino or other microcontroller, but rather how to recover the data transmitted on its SigFox backend for display in Cayenne.
Before that, I had to try @tad.dvor’s post here Sigfox to Cayenne but it was unsuccessful. I decided to take things from the beginning thanks to the help of @Adam s.

what do you need?

A sigfox account with a SigFox card connected to a microcontroller (arduino etc …). You must know the programming of your microcontroller to modify the data emitted.

Node-Red https://nodered.org/ is an open source project supported by IBM. It is an ideal graphical programming language to start programming and understand the Internet of Things. (And many other things).

A Cayenne account, it’s obvious! :wink:

Let’s go !

As you have seen, the SigFox subscription only sends a 12 bytes message every 10 minutes. He will be able to save emissions on the side of the SigFox card:

  • it will send data only if they change
  • On 12 Bytes, we can not transmit a lot of sensors: we must make a choice on the accuracy of the value, assign a location to each sensor in the 12 Bytes. In short we are limited!

The answer : establish a protocol ! To put it simply: we will send a key representing the sensor followed by its value (of this sensor).
The decoupage of the 12 Bytes are thus:
6 bits for the key + 10 bits for the value | 6 bits for the key + 10 bits for the value …

6 + 10 | 6 + 10 | 6 + 10 | 6 + 10 | 6 + 10 | 6 + 10 = 96 bits = 12Bytes

To summarize:

  • 6bits of key = 0-> 63 different types of sensors (it’s more than enough)
  • 10bits of value = 0 -> 1023 (not bad, can do better, we’ll see after :wink:)
  • 1 SigFox message can contain 6 sensors every 10 minutes (not bad)

A quick example of what the microcontroller should send to SigFox:
The microcontroller measures the atmospheric pressure (by a BME 820 for example): 955 hpa
In our protocol we will say that the pressure sensor is the N ° 1.
The No. 1 sensor measured 955 : key : 1 = 000001 , value : 955 = 1110111011
we stick the 2 : 0000011110111011 = 07BB Hexa
That’s all : you must send to SigFox: 0x07BB which meen "pressure is: 955hpa"
This is an example for 1 sensor. We can go up to 63 sensors. Up to 6 sensors per message (every 10 minutes). :stuck_out_tongue_winking_eye:

From theory to practice : recover the APIs on the site SigFox:

click on DEVICE tab and note your DEVICE ID you will need in NodeRed

go to USER and click on your name to the right

the GROUP tab is displayed. click on the API ACCESS menu. Add a NEW at the top right:

give a name + your Time Zone + click on DEVICES_MESSAGES and LIMITED_ADMIN to appear in the table on the right. click OK

note LOGIN and PASSWORD these are the keys you will need in NodeRed

On the side of Node-Red

Open Node-Red and Create a new Flow

Copy this Flow

 [{"id":"97760f2c.f3d46","type":"inject","z":"26a3b1c2.b1d25e","name":"30s delay","topic":"","payload":"1","payloadType":"str","repeat":"30","crontab":"","once":false,"onceDelay":"","x":100.01952743530273,"y":1193.0039463043213,"wires":[[]]},{"id":"7420147f.139fac","type":"function","z":"26a3b1c2.b1d25e","name":"Get SigFoxData","func":"if(msg.payload.data[0] !== undefined){\nmsg.payload = msg.payload.data[0].data;\nreturn msg;\n}","outputs":1,"noerr":0,"x":394.6306381225586,"y":1129.6704206466675,"wires":[["e2fd0d7e.fcef3"]]},{"id":"e2fd0d7e.fcef3","type":"function","z":"26a3b1c2.b1d25e","name":"extract Code/Val + calcul","func":"/*\nIn this node you configur the protocole of each keyCode and Value of each sensor\n*/\nfunction hex2bin(hex){\n    return (\"0000000000000000\" + (parseInt(hex, 16)).toString(2)).substr(-16);\n}\n\nfunction pad(n, width, z) { //for join 2x10bits\n  z = z || '0';\n  n = n + '';\n  return n.length >= width ? n : new Array(width - n.length + 1).join(z) + n;\n}\n\nvar hexSplitPayload = msg.payload.match(/.{1,4}/g);\nvar binarySplitPayload = [];\nvar code = [];\nvar value = [];\nvar hundreds =-1000;\nvar thousands=-1000;\n\n    for (var i=0; i < hexSplitPayload.length; i++){\n    binarySplitPayload[i] = hex2bin(hexSplitPayload[i]);\n    code[i+1] = parseInt(binarySplitPayload[i].slice(0,6),2);\n    value[i+1] = parseInt(binarySplitPayload[i].slice(6,16),2);\n    \n    switch (code[i+1]) {        //data received : 0-> 1023 (Decimal)\n        case 0:                 //0 key code is not used\n            break;\n         case 1:                        //pressure sensor\n            msg.pressure = value[i+1];  //receive 0->1023 => 0->1023 hpa\n            break;\n        case 2:                        //humidity sensor\n            msg.hum = value[i+1]/10;       //receive 0->1023 => 0->102.3 %\n            break;\n        case 3:                        //voltage  sensor\n            msg.vBatt = value[i+1] /100;  //receive 0->1023 =>  0->10.23 v\n            break;\n        case 4:\n            if(value[i+1]>=512) {                        //temperature sensor\n                msg.temp=-(value[i+1]-512)/10     //receive 0->1023 => +/-51.1 °c\n            }\n            else{\n                msg.temp = value[i+1]/10\n            }\n            break;\n        case 5:\n            if(value[i+1]>=512) {                        //amperage sensor\n                msg.mAmpBatt = -(value[i+1] - 512);      //receive 0->1023 => +/-511 mA  \n            } else {\n                msg.mAmpBatt=value[i+1]\n            }\n            break;\n        case 6:                             //if 1023 values is not enouth : 0->1023999\n            hundreds = pad(value[i+1], 3);  //Here first part (hundreds) : xxxx999 h\n            break;\n        case 7:\n            thousands = value[i+1];         //Here secund part (thousand) : 1023xxxx h\n            break;                          \n        case 8:                            //to display text message with through a code\n            msg.state = value[i+1];     //Here State messages\n            break;                        \n//\n//......... ADD other sensor .......\n//\n/*\n        case 63:                            //LAST sensor: no more 63 ! \n            msg.lastSensor = value[i+1];\n            break;\n*/            \n           }// END of switch\n          \n           \n}// END of FOr \n\nmsg.code = code;\nmsg.value = value;\n\nif (hundreds != -1000 && thousands != -1000){\nmsg.hours = parseInt(thousands.toString() + hundreds.toString());//89\n}\n\nreturn msg;","outputs":1,"noerr":0,"x":559.6306381225586,"y":1189.33704662323,"wires":[["359a8ff8.94b7d","30aac088.d9174"]]},{"id":"359a8ff8.94b7d","type":"debug","z":"26a3b1c2.b1d25e","name":"","active":false,"tosidebar":true,"console":false,"tostatus":false,"complete":"code","x":726.6306533813477,"y":1139.670386314392,"wires":[]},{"id":"30aac088.d9174","type":"debug","z":"26a3b1c2.b1d25e","name":"","active":false,"tosidebar":true,"console":false,"tostatus":false,"complete":"value","x":732.6306648254395,"y":1230.6703901290894,"wires":[]}]

import a new Flow & paste the Flow in it & clic import

Don’t forget to deploy after each change

Put a HTTP request Node in the Flow, wire it,double clic on it and complete it with the https adress :
https://LOGIN:PASSWORD@backend.sigfox.com/api/devices/DEVICE_ID/messages?limit=1

Let’s test if it works

put a inject node & wiring it & put 7bb in the payload in String Type

if you see blue bubbles it’s because you forgot to deploy !

clic on inject button & see result on Debug Console


We see msg.code =1 & msg.value=955 great it’s working : we have simulated a message from SigFox of sensor No. 1 which indicates 955 (hpa)

Finally: we send to Cayenne!!

add new… deviceWidget %20deviceWidget

bring your own thing bring%20your%20own%20thing

Note your Cayenne credential


Cayenne waiting for a connection …

In Node-Red : Add a Cayenne Connection

copy this flow

[{"id":"ef62ed57.28992","type":"function","z":"26a3b1c2.b1d25e","name":"fct","func":"var counter = 0;\nmsg.payload = [];\nif (msg.pressure !== undefined){    //pressure 1st\n    msg.payload[counter] = {\n        channel: 1,              //for Cayenne : channel of widget\n        type: \"pm\",              //for Cayenne : type of widget\n        unit: \" \",               //for Cayenne : unit of widget\n        value: msg.pressure,\n        name: \"pressure1\"        //for Cayenne : name of widget\n    };\n        node.send([{payload:msg.pressure},null,null,null,null,null,null,null]); //position 1/8\n    counter++;\n}\nif (msg.hum !== undefined){     //humidity 2nd\n    msg.payload[counter] = {\n        channel: 2,\n        type: \"soil_moist\",\n        unit: \"p\",\n        value: msg.hum,\n        name: \"hum2\"\n    };\n       node.send([null,{payload:msg.hum},null,null,null,null,null,null]); // position 2/8\n     counter++;\n}\nif (msg.vBatt !== undefined){ //voltage 3th\n    msg.payload[counter] = {\n        channel: 3,\n        type: \"voltage\",\n        unit: \"v\",\n        value: msg.vBatt,\n        name: \"vBatt3\"\n    };\n         node.send([null,null,{payload:msg.vBatt},null,null,null,null,null]); //position 3/8\n    counter++;\n}\n    \nif (msg.temp !== undefined){     //temperature 4th\n    msg.payload[counter] = {\n        channel: 4,\n        type: \"temp\",\n        unit: \"c\",\n        value: msg.temp,\n        name: \"temp4\"\n    };\n     node.send([null,null,null,{payload:msg.temp},null,null,null,null]); //position 4/8\n    counter++;\n}\nif (msg.mAmpBatt !== undefined){    //mAmp Battery 5th\n    msg.payload[counter] = {\n        channel: 5,\n        type: \"curent\",\n        unit: \"a\",\n        value: msg.mAmpBatt,\n        name: \"mAmpBatt5\"\n    };\n        node.send([null,null,null,null,{payload:msg.mAmpBatt},null,null,null]); //position 5/8\n    counter++;\n}\nif (msg.hours !== undefined){   //hours 6th \n    msg.payload[counter] = {\n        channel: 67,\n        type: \"counter\",\n        unit: \"null\",\n        value: msg.hours,\n        name: \"hours67\"\n    };\n    node.send([null,null,null,null,null,{payload:msg.hours},null,null]); //position 6/8\n    counter++;\n}\n\nif (msg.state !== undefined){   //hours 7th \n    msg.payload[counter] = {\n        channel: 8,\n        type: \"pm\",\n        unit: \" \",\n        value: msg.state,\n        name: \"state8\"\n    };\n    node.send([null,null,null,null,null,null,{payload:msg.state},null]); //position 7/8\n    counter++;\n}\n\n\nreturn [null,null,null,null,null,null,null,msg]; //position 8/8 : message to Cayenne","outputs":8,"noerr":0,"x":535.0244140625,"y":1316.5673828125,"wires":[[],[],[],[],[],[],[],["4ddccce5.6039a4"]]}]

import a new Flow (look above how to do) & paste the Flow in it & clic import & wire it

Put a Node MQTT output and register: v1 / MQTT_USERNAME / things / CLIENT_ID / data / json
In Server, click on the small arrow and select “Add new mqtt-Broker …” call it as you want

clic on the pen on the right and register your CLIENT_ID & in the security tab the MQTT_USER and MQTT PASSWORD


The Server is : mqtt.mydevices.com:1883 Port: 1883

press again on the inject button & see Cayenne Dashboard

Nice !

Other things will happen … Wait And see ! :wink:


#2

Nice tutorial @SuperNinja . Thanks for sharing with the community.


#3

I think this is just the beginning :wink:


#4

So the payload that gets sent to the mqtt send node is a string, looking something like this ???

{‘channel’:‘3’,‘value’:333,‘unit’:‘c’,‘device_type’:‘analog_sensor’}

I find the cayenne broker connects, but when I send the data, it disconnects, and reconnects, but no data sent. :frowning_face:


#5

In a blanc field, in Node-Red, put a inject Node + MQTT Node ( with Your Cayenne credential).
Copy and paste this in the inject node “payload” ( in String type “a z”)
[{"channel":3,"type":"temp","unit":"c","value":333,"name":"temperature3"}]

Don’t forget to DEPLOY ! And clic on the Inject button :

It’s working for me :
image


#6

so there is no topic in the inject node, and the topic in the mqtt node is

v1/971da7b0-fd21-11e6-ac86-a9a56e70acce/things/ddee1cc0-ea90-11e8-809d-0f8fe4c30267/data/json

with my username and clientid, qos=1, retain=true

and the server is mqtt.mydevices.com: 1883, with my username as above, and my password from the byot config page and no SSL/TSL, clean session, and Legacy 3.1 selected.

I managed to get subscriptions working fine, but the writing mqtt has not succeeded.


#7


in the inject node : copy and past this in the payload field near the “az”: [{"channel":3,"type":"temp","unit":"c","value":333,"name":"temperature3"}]

  1. is there a green fixed bubble on the mqtt node ?

  2. if you add a debug node did you see that in the debug console ? :

    ?

  3. share your screen shot


#8

Thanks for your help … here are the screenshots

new byot:

node-red code - just the stuff at top - the other stuff subscribes to data and it all work - the SendMqtt starts as connected

After I trigger the Inject - it goes disconnected, but gets the debug data (the second debug message is a clock, not related)

11/17/2018, 4:40:23 PMnode: 3495a14f.3364fe
msg.payload : string[74]
“[{“channel”:3,“type”:“temp”,“unit”:“c”,“value”:333,“name”:“temperature3”}]”

On the node-red log, we see disconnection - brokers Cayenne with client jam works and receives data. The broker Cay2 client jamcay is the one that fails

the inject


[{“channel”:3,“type”:“temp”,“unit”:“c”,“value”:333,“name”:“temperature3”}]

the mqtt node
image

v1/971da7b0-fd21-11e6-ac86-a9a56e70acce/things/ddee1cc0-ea90-11e8-809d-0f8fe4c30267/data/json

The broker
image

and broker security
image

Generally one discovers the error, but preparing the question … but I still cannot find it.


#9

ok, I found it.

Clientid for a receive-mqtt can be anything, but a send-mqtt must be the clientid from the Cayenne byot config.

So in above example “jamcay” code is replaced with the “ddee1…” number.

Thanks … setting off to find new confusion in node-red! :grinning: