MQTT Manually Publishing / Subscribing

I add the “@reboot python /home/pi/tinkerforge/brick-mqtt-proxy.py --brickd-host 192.168.1.41 --brickd-port 4223 --broker-host localhost --broker-port 1883 --update-interval 5&” to the crontab, rebooted and it seems to work.

here is the latest used cayenne-mqtt.py that gives me the data in my last post

import paho.mqtt.client as mqtt
import time
import sys
import subprocess


mqttc = mqtt.Client(client_id="xxxx")
mqttc.username_pw_set("yyyy", password="zzzz")
mqttc.connect("mqtt.mydevices.com", port=1883, keepalive=60)

topic_temp = "v1/username/things/xxxx/data/1"  #change username/clientid here without quotes
topic_humidity = "v1/username/things/xxxx/2"
topic_airpress = "v1/username/things/xxx/data/3"

while True:
    try: 
        temp = subprocess.Popen("mosquitto_sub -t /tinkerforge/bricklet/temperature/t6Q/temperature",shell=True)
	humidity= subprocess.Popen("mosquitto_sub -t /tinkerforge/bricklet/humidity/uk9/humidity",shell=True)
        airpress= subprocess.Popen("mosquitto_sub -t /tinkerforge/bricklet/barometer/vGZ/air_pressure",shell=True)
	
        if temp is not None:
                temp = "temp,c=" + str(temp)
                mqttc.publish(topic_temp, payload= temp , retain=True)	

        if humidity is not None:
                humidity = "humidity,%=" + str(humidity)
                mqttc.publish(topic_humidity, payload= humidity , retain=True)

        if airpress is not None:
                airpress = "airpress,mBar=" + str(airpress)
                mqttc.publish(topic_airpress, payload= airpress , retain=True)

        time.sleep(5)
    except (EOFError, SystemExit, KeyboardInterrupt):
        mqttc.disconnect()
        sys.exit()

Sorry, it seems I have problems with keeping the indents at the beginning of the script.
Did a ctl-c an ctl-v from the idle editor and only the last part is good.
Could not upload the .py file as it is not allowed.
Any hint how to do it as it should for the future?

How do I get the sensor data on the dashboard?

Thanks

Patrick

Just highlight all the code and hit the preformatted text icon at the top of the editor window

It all looks good to me, double check that this info below is filled in correctly. That would be a reason why the widgets wouldn’t show up on the dashboard. Also refresh the dashboard page after the device shows up as online. Sometimes they don’t come up automatically.

(ignore the spaces - apparently the bold tag doesn’t work when preceded by a /)
topic_temp = “v1/ username /things/ clientid /data/1” #change MQTT username/clientid here
topic_humidity = “v1/ username /things/ clientid / data /2” #change MQTT username/clientid here - also missing /data/
topic_airpress = “v1/ username /things/ clientid /data/3” #change MQTT username/clientid here

Hi Adam,

Ok, made a mistake in the topics lines; I copied and pasted the username with a " add the end in the three topics.
Removed the quote at the end of username in the three topics and bingo …saw three sensors in cayenne dashboard.

But exempt for temp widget (still without data) that all the widgets where completely empty: just a channel placeholder.
Then found at the end of the mqtt manually publishing/subscribing a paragraph called: Supported Data Types.
Saw that humidity(hvac_hum) and air pressure(press) had other data type and units.

So I tried to modified the last part of the cayenne-mqtt.py to:

while True:
try: 
	temp = subprocess.Popen("mosquitto_sub -t /tinkerforge/bricklet/temperature/t6Q/temperature",shell=True)
	hvac_hum= subprocess.Popen("mosquitto_sub -t /tinkerforge/bricklet/humidity/uk9/humidity",shell=True)
	press= subprocess.Popen("mosquitto_sub -t /tinkerforge/bricklet/barometer/vGZ/press",shell=True)

	if temp is not None:
			temp = "temp,c=" + str(temp)
			mqttc.publish(topic_temp, payload= temp , retain=True)	

	if hvac_hum is not None:
			hvac_hum = "hvac_hum,P=" + str(hvac_hum)
			mqttc.publish(topic_humidity, payload= hvac_hum , retain=True)

	if press is not None:
			press = "press,bar=" + str(press)
			mqttc.publish(topic_airpress, payload= press , retain=True)

	time.sleep(5)
	
except (EOFError, SystemExit, KeyboardInterrupt):
	mqttc.disconnect()
	sys.exit()

And got this on my dashboard:

Huge progress!
I begin to understand the logic of the cayenne mqtt api.

But I still not have the sensor data on all channels and also in channel 2 (humidity in %) and 3 (air pressure in mbar) no units.
Here I am stuck; did not find why in the docs.

Any idea why I do not have any sensor data?

See the finish line and need a little push to get there:slight_smile:

Thanks for your help,

Patrick

Hi Adam,

After some googling, found that data type {“_timestamp”:1487085328.256301,“temperature”:2525} a dictory is

Then here is what if did:

pi@pi20:~/tinkerforge $ python cayenne-mqtt.py
{“_timestamp”:1487085328.256301,“temperature”:2525}
{“_timestamp”:1487085358.444162,“humidity”:291}
{“_timestamp”:1487085363.488732,“humidity”:290}
{“_timestamp”:1487085363.46522,“temperature”:2518}
{“_timestamp”:1487085363.46522,“temperature”:2518}
{“_timestamp”:1487085363.488732,“humidity”:290}
^Cpi@pi20:~/tinkerforge $ python
Python 2.7.9 (default, Sep 17 2016, 20:26:04)
[GCC 4.9.2] on linux2
Type “help”, “copyright”, “credits” or “license” for more information.

dict={“_timestamp”:1487085328.256301,“temperature”:2525}
dict.get(“temperature”)
2525

Copied the first {“_timestamp”:1487085328.256301,“temperature”:2525} to dict
when dict.get(“temperature”) I receive the temp value of 2525 => the right value

But when I tried this in the cayenne-mqtt.py

temp = “temp,c=” + str(temp.get(“temperature”) I receive a message that Popen doesn’t have a get module.

I seems that the
temp = subprocess.Popen(“mosquitto_sub -t /tinkerforge/bricklet/temperature/t6Q/temperature”,shell=True)
doen’t give the same values as from the prompt.
Some guys speaks about adding a PIPE and stdout,…
But this a bit over my small python knowledge.

Hou do I check what is in temp just after temp = subprocess.Popen(“mosquitto_sub -t /tinkerforge/bricklet/temperature/t6Q/temperature”,shell=True) ???

Thanks

Patrick

Just taking a guess here since I can’t really test, but try

temp = “temp,c=” + str(dict(temp).get(“temperature”))

Also, if you are still getting errors about popen, you can try subprocess.call instead of subprocess.popen

temp = subprocess.call(“mosquitto_sub -t /tinkerforge/bricklet/temperature/t6Q/temperature”,shell=True)

1 Like

After reading the whole post, I can think of a better solution.

Since if you are launching a subscriber for a mqtt channel, at the time of making the subscription statement (on that command line) you can run a script (.py) each time that data is received (when that subscriber is activated) And in that script already make the connection to cayenne and publish as a client (which is very simple, using libraries).

I remember reading in an MQTT tutorial how you can have a subscription that automatically when you get new values execute a bash script, I think it’s a matter of handling more bash commands.

1 Like

I do not know if my idea is crazy or impossible, but it came about when I read about MQTT a long time ago in this article

Specifically this line:
mosquitto_sub -h m20.cloudmqtt.com -p 15162 -t "GPIO" -u raspberry -P raspberry > /sys/class/gpio/gpio17/value

hi Adam,

Spend along time trying Pope and call subprocess scripts and could not get the temp value out the
temp = subprocess.Popen(“mosquitto_sub -t /tinkerforge/bricklet/temperature/t6Q/temperature”,shell=True)

removing all the humidity, air-pressure I made a cayenne-mqtt-test.py file with

while True:
try: 
	temp = subprocess.call("mosquitto_sub -t /tinkerforge/bricklet/temperature/t6Q/temperature",shell=True)
	print (temp)			
	temp = str(dict(temp).get("temperature"))
	print (temp)
	
	time.sleep(5)
	
except (EOFError, SystemExit, KeyboardInterrupt):
	mqttc.disconnect()
	sys.exit()

keeping the fist part intact to try to see what value is in temp and different stages. Runinning it gives me this:

pi@pi20:~/tinkerforge $ python cayenne-mqtt-test.py

{“_timestamp”:1487165024.991951,“temperature”:2550}
{“_timestamp”:1487165095.287481,“temperature”:2556}
^Cpi@pi20:~/tinkerforge $

tried print with () and without but nothing prints on the screen.

How to I “debug” the value of temp during the running of this test script?

Thanks

Patrick

Maybe try stdout=subprocess.PIPE. Sorry, I don’t have mosquitto_sub installed to test right now. I’ll try to get that installed tonight and do a proper test to see what the issue is instead of stabbing in the dark.

Try this with Omega Onion2:

Hi Adam,

ok will wait.

But I have some doubts about the value of temp in the

temp = subprocess.call(“mosquitto_sub -t /tinkerforge/bricklet/temperature/t6Q/temperature”,shell=True)

So I put the value {“_timestamp”:1487085328.256301,“temperature”:2525} in temp

while True:
try: 
	
	temp = {"_timestamp":1487085328.256301,"temperature":2525}
	print (temp)			
	temp = str(dict(temp).get("temperature"))
	print (temp)
					
	time.sleep(5)
	
except (EOFError, SystemExit, KeyboardInterrupt):
	mqttc.disconnect()
	sys.exit()

and receive this when running

pi@pi20:~/tinkerforge $ python cayenne-mqtt-test.py
{‘_timestamp’: 1487085328.256301, ‘temperature’: 2525}
2525
{‘_timestamp’: 1487085328.256301, ‘temperature’: 2525}
2525
{‘_timestamp’: 1487085328.256301, ‘temperature’: 2525}
2525
{‘_timestamp’: 1487085328.256301, ‘temperature’: 2525}
2525
^Cpi@pi20:~/tinkerforge $

What is the right solution.

Now I am convinced that the value of temp with the
temp = subprocess.call(“mosquitto_sub -t /tinkerforge/bricklet/temperature/t6Q/temperature”,shell=True)
is not a " {‘_timestamp’: 1487085328.256301, ‘temperature’: 2525} " type value but something else.

Is there a way to see/print the value of temp just after the line
temp = subprocess.call(“mosquitto_sub -t /tinkerforge/bricklet/temperature/t6Q/temperature”,shell=True)
when running the script, just to be 100% sure ?

Thanks

Patrick

Ok, so it’s been a while since I’ve used subprocess…I was doing it all wrong. I’m not sure if it’s possible or not, but it probably would have been much easier to just read the values natively in python instead of using subprocess. Might look in to that sometime to see if it’s possible to connect to more than one server in the same script. Anyway…This should work for you:

import paho.mqtt.client as mqtt
import time
import sys
import subprocess


mqttc = mqtt.Client(client_id="xxxx")
mqttc.username_pw_set("yyyy", password="zzzz")
mqttc.connect("mqtt.mydevices.com", port=1883, keepalive=60)

topic_temp = "v1/username/things/xxxx/data/1"  #change username/clientid here without quotes
topic_humidity = "v1/username/things/xxxx/data/2"
topic_airpress = "v1/username/things/xxx/data/3"

while True:
    try: 
        temp = subprocess.Popen(['mosquitto_sub', '-t', '/tinkerforge/bricklet/temperature/t6Q/temperature', stdout=subprocess.PIPE)
        humidity= subprocess.Popen(['mosquitto_sub', '-t', '/tinkerforge/bricklet/humidity/uk9/humidity', stdout=subprocess.PIPE)
        airpress= subprocess.Popen(['mosquitto_sub', '-t', '/tinkerforge/bricklet/barometer/vGZ/air_pressure', stdout=subprocess.PIPE)
	
        if temp is not None:
                temp = "temp,c=" + str(int(temp.stdout.readline()))
                mqttc.publish(topic_temp, payload= temp , retain=True)	

        if humidity is not None:
                humidity = "humidity,%=" + str(int(humidity.stdout.readline())
                mqttc.publish(topic_humidity, payload= humidity , retain=True)

        if airpress is not None:
                airpress = "airpress,mBar=" + str(int(airpress.stdout.readline()))
                mqttc.publish(topic_airpress, payload= airpress , retain=True)

        time.sleep(5)
    except (EOFError, SystemExit, KeyboardInterrupt):
        mqttc.disconnect()
        sys.exit()

Hi Adam,

Tried out your latest solution and got this error:

pi@pi20:~/tinkerforge $ python cayenne-mqtt-test.py

File “cayenne-mqtt-test.py”, line 15
temp = subprocess.Popen([‘mosquitto_sub’, ‘-t’, ‘/tinkerforge/bricklet/temperature/t6Q/temperature’, stdout =subprocess.PIPE)
^
SyntaxError: invalid syntax
pi@pi20:~/tinkerforge $

Saw that a ] was missing after …/t6Q/temperature’ ] , stdout =subprocess.PIPE)

This problem was solved, but the I receive this

pi@pi20:~/tinkerforge $ python cayenne-mqtt-test.py
Traceback (most recent call last):
File “cayenne-mqtt-test.py”, line 18, in
temp = “temp,c=” + str(int(temp.stdout.readline()))
ValueError: invalid literal for int() with base 10: ‘{“_timestamp”:1487318726.163536,“temperature”:2431}\n’
pi@pi20:~/tinkerforge $

does it has to do with the fact that the value is a dictand not an int() ???
I only need the temperature value and not the full {“_timestamp”:1487318726.163536,“temperature”:2431} data value.
could this be the error???

Then to replace int with dic
temp = “temp,c=” + str(dict(temp.stdout.readline()))
and rafter running got this error

pi@pi20:~/tinkerforge $ python cayenne-mqtt-test.py
Traceback (most recent call last):
File “cayenne-mqtt-test.py”, line 18, in
temp = “temp,c=” + str(dict(temp.stdout.readline()))
ValueError: dictionary update sequence element #0 has length 1; 2 is required

Still not the temp value we need…

Thanks
Patrick

Lets just get rid of subprocess…it’s not really a good thing to use anyway. Try this:

import paho.mqtt.client as mqtt
import time
import sys
import subprocess
import json

temp = None
humidity = None
airpress = None

def on_message_temp(mosq, obj, msg):
    global temp
    temp = json.loads(msg.payload)

def on_message_humidity(mosq, obj, msg):
    global humidity
    humidity = json.loads(msg.payload)

def on_message_airpress(mosq, obj, msg):
    global airpress
    airpress = json.loads(msg.payload)

Cayenne = mqtt.Client(client_id="clientid")
Cayenne.username_pw_set("username", password="password")
Cayenne.connect("mqtt.mydevices.com", port=1883, keepalive=60)

localmqtt = mqtt.Client()
localmqtt.message_callback_add("tinkerforge/bricklet/temperature/t6Q/temperature", on_message_temp) #do not include leading slash on topic
localmqtt.message_callback_add("tinkerforge/bricklet/humidity/uk9/humidity", on_message_humidity) #do not include leading slash on topic
localmqtt.message_callback_add("tinkerforge/bricklet/barometer/vGZ/air_pressure", on_message_airpress) #do not include leading slash on topic
localmqtt.connect("localhost", port=1883, keepalive=60)
localmqtt.subscribe([("/tinkerforge/bricklet/temperature/t6Q/temperature",0),("/tinkerforge/bricklet/humidity/uk9/humidity",0),("/tinkerforge/bricklet/barometer/vGZ/air_pressure",0)])
localmqtt.loop_start()

topic_temp = "v1/username/things/clientid/data/1"  #change username/clientid here
topic_humidity = "v1/username/things/clientid/data/2" #change username/clientid here
topic_airpress = "v1/username/things/clientid/data/3" #change username/clientid here

while True:
    try: 
        if temp is not None:
            cayennetemp = "temp,c=" + str(temp['temperature'])
            Cayenne.publish(topic_temp, payload=cayennetemp , retain=True)	

        if humidity is not None:
            cayennehumidity = "humidity,%=" + str(humidity['humidity'])
            Cayenne.publish(topic_humidity, payload=cayennehumidity , retain=True)

        if airpress is not None:
            cayenneairpress = "airpress,mBar=" + str(airpress['airpress'])
            Cayenne.publish(topic_airpress, payload=cayenneairpress , retain=True)

        time.sleep(5)
    except (EOFError, SystemExit, KeyboardInterrupt):
        localmqtt.loop_stop()
        Cayenne.disconnect()
        localmqtt.disconnect()
        sys.exit()

Hi Adam,

Put your new code in my cayenne-mqtt-test.py and replaced clientid, username and password with the values of my bring your own thing page.

Gat a strange error, the code for airpress seems similar as for temp and humidity but I still receive this error:

pi@pi20:~/tinkerforge $ python cayenne-mqtt-test.py
Traceback (most recent call last):
File “cayenne-mqtt-test.py”, line 50, in
cayenneairpress = “airpress,mBar=” + str(airpress[‘airpress’])
KeyError: ‘airpress’

When I open my dashboard, I see this:

I have a value in temp ,to be divided by 100 to have the real temp.
Used this code cayennetemp = “temp,c=” + str(temp[‘temperature’]/100)
And the value became 25,00 °C

But no value or unit for humidity or airpress when this strange error will be gone.

Do you have to use the “hvac_hum,P=” instead of “humidity,%=” to see the value and unit as I found at one of the last pages of the MQTT Manually publishing / Subscripbing documentation?
It is a very large table with all sorts of units for a lot of possible sensors.
Or can you keep “humidity,%=” ???

Thanks

Patrick

What is the output of:

“mosquitto_sub -t /tinkerforge/bricklet/humidity/uk9/humidity”
and
“mosquitto_sub -t /tinkerforge/bricklet/barometer/vGZ/air_pressure”

Adam,

Made some progress.
As I have a keyerror: ‘airpress’ and do not see from where the error originate, I remarked the lines

#if airpress is not None:
        #cayenneairpress = "airpress,mBar=" + str(airpress['airpress'])
        #Cayenne.publish(topic_airpress, payload=cayenneairpress , retain=True)

Now the program runs without error and I receive the value for humidity but without the % unit (see arrow)

So temp and humidy are working and I see the values changing.

Rest the airpress problem.When removing the 3 remarks from the airpress lines, the program stops with error:

pi@pi20:~/tinkerforge $ python cayenne-mqtt-test.py
Traceback (most recent call last):
File “cayenne-mqtt-test.py”, line 50, in
cayenneairpress = “airpress,mBar=” + str(airpress[‘airpress’])
KeyError: ‘airpress’

As you ask I ran mosquitto_sub -t /tinkerforge/bricklet/barometer/vGZ/air_pressure and receive

pi@pi20:~/tinkerforge $ mosquitto_sub -t /tinkerforge/bricklet/barometer/vGZ/air_pressure
{“_timestamp”:1487407795.6225,“air_pressure”:1023000}
{“_timestamp”:1487407800.684803,“air_pressure”:1022990}
{“_timestamp”:1487407805.756374,“air_pressure”:1022996}
{“_timestamp”:1487407810.811612,“air_pressure”:1022989}
{“_timestamp”:1487407815.842195,“air_pressure”:1022987}

So that seems good.

I try to solve the keyerror and the unit missing without success.

Any idea’s

Thanks

Patrick

Use this:

cayenneairpress = “airpress,mBar=” + str(airpress[‘airpressure’])

Adam,
Tried it and got the same mistake.
Then changed the last airpressure to air_pressure and it works.
cayenneairpress = “airpress,mBar=” + str(airpress[‘air_pressure’]/1000) #divided by 1000 to have the right value.

Here is the proof:

Thanks you, you are the best.

Patrick.

P.S. any idea why we do not see the % in humidity and mBar in the air pressure widget? Should appear as Celcius in the temp widget or am I wrong?