Multiple MQTT Clients on 8266


#1

Here is a little demo to answer the question about how to move data between 8266s.

My first solution was for the 8266 receiver to query the Cayenne server using REST API, which worked until the REST API outage problem, so I finally read up MQTT and used it.

I have several 8266 reporting temperatures, pressures, etc. that all report to Cayenne every 15 sec or 5 minutes, and all these can be put together on a single web page or graph at the cayenne website, and previously fetched down to my Android phone using some programs on another post, but I wanted a continuous display I could put on the wall to see everything without getting a computer or phone. So this program has one 8266 subscribing to updates from all the other 8266s, and cayenne (the mqtt broker) echo’es the mqtt back from all my devices back to the device with the display.

I was unable to get the cayenne mqtt library to do this (details to follow), so I looked for another mqtt library that is used in the display.

This example has 2 8266s belonging to 2 users that send data to cayenne in the regular way, and then a 3rd 8266 belonging to one of those users that sends a little more data to cayenne, and subscribes to the data from the previous 2 devices. You need usernames, passwords, client-ids for all the devices of course.

//
//  multiMQTT
//
//  Demo of using Cayenne mydevices.com on ESP 8266 with light weight mqtt library https://github.com/256dpi/arduino-mqtt
//
//  This demo sends data to cayenne, and receives data from cayenne for this device, plus 2 other 8266 running the standard cayenne mqtt library.
//  
//  IMPORTANT NOTE - Due to the extra long client-id used by cayenne, you have to edit the file MQTTClient.h in 
//                    .../Arduino/libraries/MQTT/src/MQTTClient.h on line 142 as shown to increase the buffer size 
//
//   explicit MQTTClient(int bufSize = 192) {              // jz 2018-12-01 change from 128 to 192
//
//   James Zahary 2018-12-11
//


#define CAYENNE_DEBUG
#define CAYENNE_PRINT Serial

//#include <CayenneMQTTESP8266.h>  - standard cayenne mqtt library must be removed as there are conflicts with structure names

#include <ESP8266WiFi.h>
#include <MQTT.h>       // this is the 256pi library from github

// WiFi network info.
char ssid[] = "TELUS0602";
char wifiPassword[] = "peanut";

// ~~~~~~~~~~~~
// Client A belonging to user A, sending data with regular cayenne virtualwrite() - we will receive mqtt from this device
// You could receive any data belonging to user A by duplicating this string "desk" below, and putting in different channel 
// numbers - the "2" - and different client numbers for different 8266 or pi's belong to User A - the "9e5 ... 267" number.

WiFiClient netA;
MQTTClient clientA;

char usernameA[] = "8bc435a0-3367-11e8-aeac-8375e928efd4";
char passwordA[] = "1e4peanutpeanutpeanutpeanutpeanutpeanut4";
char clientIDA[] = "displayA";

char desk[] = "v1/8bc435a0-3367-11e8-aeac-8375e928efd4/things/9e506830-f390-11e8-809d-0f8fe4c30267/data/2";

// ~~~~~~~~~~~~
// Client J belonging to user J, sending with regular cayenne virtualwrite() - we will receive mqtt from this device

WiFiClient netJ;
MQTTClient clientJ;

char usernameJ[] = "971da7b0-fd21-11e6-ac86-a9a56e70acce";
char passwordJ[] = "257peanutpeanutpeanutpeanutpeanutpeanut4";
char clientIDJ[] = "displayJ";  

char green[] = "v1/971da7b0-fd21-11e6-ac86-a9a56e70acce/things/ef7d67f0-339f-11e8-b6d4-35a0ad51f849/data/1";

// ~~~~~~~~~~~~
// This device is client E - we will send data to cayenne, and also receive data from cayenne for this device
// The cleint ID below is provided by cayenne when you create the device.
// Client E is also owned by User J, but we need two different mqtt channels or clients to access this device, versus all of User J's other devices 
// The other cleint IDs above are just unique random names - they must NOT be the client ID that cayenne provides to the 8266 that updates cayenne
// We are sending using the json style which is slightly difference syntax than the virtualwrite() uses.
// And the echo back from cayenne is also in this json style.

WiFiClient netE;
MQTTClient clientE;

char usernameE[] = "971da7b0-fd21-11e6-ac86-a9a56e70acce";
char passwordE[] = "257peanutpeanutpeanutpeanutpeanutpeanut4";
char IDE[] = "128f9970-fcae-11e8-b82d-f12a91579eed";  

char echo[] = "v1/971da7b0-fd21-11e6-ac86-a9a56e70acce/things/128f9970-fcae-11e8-b82d-f12a91579eed/data/json";

unsigned long lastMillis = 0;
int error;

//~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
void setup() {
  Serial.begin(115200);


  WiFi.begin(ssid, wifiPassword);
  Serial.println("Checking wifi...");
  while (WiFi.status() != WL_CONNECTED) {
    Serial.print(".");
    delay(1000);
  }

   Serial.println(" ");
   Serial.println("Create clients ...");
  
   clientA.begin("mqtt.mydevices.com", 1883, netA);
   clientJ.begin("mqtt.mydevices.com", 1883, netJ);
   clientE.begin("mqtt.mydevices.com", 1883, netE);
  
   clientA.onMessage(messageReceivedA);
   clientJ.onMessage(messageReceivedJ);
   clientE.onMessage(messageReceivedE);

   Serial.println("Connecting to MQTT ...");

   delay(3000);
    
   int res = 0;
  
   res = clientA.connect(clientIDA, usernameA, passwordA, false);
   while (res == 0){
    Serial.print(".");
    delay(3000);
    res = clientA.connected();
   }

   Serial.println("MQTT client A connected");
   
   int result = clientA.subscribe(desk);
   Serial.print("Subscribed desk topic from client A, status = "); Serial.println(result);

   res = clientJ.connect(clientIDJ, usernameJ, passwordJ, false);
   while (res == 0){
    Serial.print(".");
    delay(3000);
    res = clientJ.connected();
  }

  Serial.println("MQTT client J connected");
  
  result = clientJ.subscribe(green);
  Serial.print("Subscribed green topic from client J, status =  "); Serial.println(result);
  result = clientJ.subscribe(echo);
  Serial.print("Subscribed echo topic from client J, status =   "); Serial.println(result);
  // since green and echo are both owned by User J/E (J and E are the same), then client J can request both

 
  
   res = clientE.connect(IDE, usernameE, passwordE, false);
   while (res == 0){
    Serial.print(".");
    delay(3000);
    Serial.println(IDE);                              // some additional details for debugging the connection
    Serial.println(usernameE);
    Serial.println(passwordE);
    Serial.println(clientE.lastError());
    Serial.println(clientE.returnCode());
    res = clientE.connected();
   }

   Serial.println("MQTT client E connected");
   
   result = clientE.subscribe(echo);
   Serial.print("Subscribed echo topic from client E, status =  "); Serial.println(result);
  
}


//~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
// incoming AAA: v1/8bc435a0-3367-11e8-aeac-8375e928efd4/things/9e506830-f390-11e8-809d-0f8fe4c30267/data/2 - temp,c=19.500

void messageReceivedA(String &topic, String &payload) {
  int ind;
  String temp;
  String dev;
  Serial.println("incoming AAA: " + topic + " - " + payload);
  ind = payload.indexOf('=', 0 );
  temp = payload.substring(ind+1, ind+5);   // 4 digits of number
  
  ind = topic.length();     
  dev = topic.substring(ind-9, ind-7);      // last 2 digits of client id
  Serial.println(dev + " " + temp);
}

//incomng JJJ: v1/971da7b0-fd21-11e6-ac86-a9a56e70acce/things/d4fcd880-e9a7-11e7-80de-0f99a32d7a93/data/2 - AM Temp,C=21.500
//incomng JJJ: v1/971da7b0-fd21-11e6-ac86-a9a56e70acce/things/128f9970-fcae-11e8-b82d-f12a91579eed/data/json - [{"channel":90,"type":"time","unit":"ms","name":"millis","value":58002}]

// messageReceivedJ receives mqtt in two formats, from two different 8266s - so you have to figure out how to parse it!

void messageReceivedJ(String &topic, String &payload) {
  int ind;
  String temp;
  String dev;
  Serial.println("incomng JJJ: " + topic + " - " + payload);
  
 
}

//incomng EEE: v1/971da7b0-fd21-11e6-ac86-a9a56e70acce/things/128f9970-fcae-11e8-b82d-f12a91579eed/data/json - [{"channel":90,"type":"time","unit":"ms","name":"millis","value":377107}]

void messageReceivedE(String &topic, String &payload) {
  int ind;
  int ind2;
  String temp;
  String dev;
  Serial.println("incomng EEE: " + topic + " - " + payload);
  ind = payload.lastIndexOf(':' );            // colon before the number 
  ind2 = payload.lastIndexOf('}' );           // curly bracket after the number
  temp = payload.substring(ind+1, ind2);    
  //temp = payload.substring(ind+1, ind+5);     // i only want 4 digits, so dont go to end
  ind = topic.length();
  dev = topic.substring(ind-12, ind-10);
  Serial.println(dev + " " + temp);
}


//~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
void loop() {
    
  // publish a message roughly every 29 seconds
  
  if (millis() - lastMillis > 29000) {
    lastMillis = millis();
    
    String s1 = "[{\"channel\":90,\"type\":\"time\",\"unit\":\"ms\",\"name\":\"millis\",\"value\":";
    String s2 = "}]";
    String s3 = s1 + String(lastMillis) + s2;
    
    clientE.publish(echo, s3);
    Serial.print("Sending ===> "); Serial.print(echo); Serial.println(s3);
  }

  clientA.loop();
  clientJ.loop();
  clientE.loop();
    
}

This program uses https://github.com/256dpi/arduino-mqtt which can be fetched from github, or installed from the Arduino Compiler used Tools - Manage Libraries … You also have to make a small edit in the code to handle the long client-id names that cayenne uses. It is written up in the code what you have to do - it is just changing a 128 to a 196.

The problem I had the the cayenne mqtt library is that it creates one mqtt client called Cayenne that is used in the Cayenne.begin(), Cayenne.virtualWrite() etc but when I created additional CayenneMQTTWiFiClients they seemed to create okay, but then the started interacting so only the last client created would work, so there must be a global variable somewhere connecting then all together. I tried CayenneMQTTClients and CayenneArduinoMQTTClients, but switched to the other one before I got to the bottom of the problem. Multiple MQTT clients might not be supported in that library.

The Arduino monitor produces the following showing the data from the 3 devices all arriving at this device.

checking wifi...
...Create clients ...
Connecting to MQTT ...
MQTT client A connected
Subscribed desk topic from client A, status = 1
MQTT client J connected
Subscribed green topic from client J, status =  1
Subscribed echo topic from client J, status =   1
MQTT client E connected
Subscribed echo topic from client E, status =  1
incoming AAA: v1/8bc435a0-3367-11e8-aeac-8375e928efd4/things/9e506830-f390-11e8-809d-0f8fe4c30267/data/2 - temp,c=21.125
67 21.1
Sending ===> v1/971da7b0-fd21-11e6-ac86-a9a56e70acce/things/128f9970-fcae-11e8-b82d-f12a91579eed/data/json[{"channel":90,"type":"time","unit":"ms","name":"millis","value":29001}]
incomng EEE: v1/971da7b0-fd21-11e6-ac86-a9a56e70acce/things/128f9970-fcae-11e8-b82d-f12a91579eed/data/json - [{"channel":90,"type":"time","unit":"ms","name":"millis","value":29001}]
ed 29001
incomng JJJ: v1/971da7b0-fd21-11e6-ac86-a9a56e70acce/things/128f9970-fcae-11e8-b82d-f12a91579eed/data/json - [{"channel":90,"type":"time","unit":"ms","name":"millis","value":29001}]
incomng JJJ: v1/971da7b0-fd21-11e6-ac86-a9a56e70acce/things/ef7d67f0-339f-11e8-b6d4-35a0ad51f849/data/1 - temp,c=1.000
incoming AAA: v1/8bc435a0-3367-11e8-aeac-8375e928efd4/things/9e506830-f390-11e8-809d-0f8fe4c30267/data/2 - temp,c=21.187
67 21.1

Help in transmitting between arduino
#2

Great work @jameszahary. You are right the current cayenne MQTT library can only subscribe to cmd topic.
To make it work you might have to change it subscribe to data topics of other client_id but then it might have created more problem. So using a separate library was a good option.