Goodbye thingsboard – welcome grafana Part 1

Thingsboard had kind of a short lifespan in the weather station project. The main issues I have

  • resource hungry – Java, tomcat server
  • buggy – kept crashing on me 5 times yesterday and now doesn’t even startup anymore
  • controls not implemented yet, multi-axis missing
  • basically too much boilerplate

On paper thingsboard looks like a good and easy way to get your MQTT data archived and displayed, but I don’t need a multi tenant solution where I need to invest a lot of resources to debug, fix and implement the functionality I need.

So here’s the deal: Mosquitto as MQTT broker, InfluxDB, Telegraf as bridge between Mosquitto and InfluxDB and Grafana for data visualization.

In part one we get the basic system running on localhost, in part 2 we add security and deploy to our web server.

Mosquitto MQTT broker

Setup mosquitto. I just use the one packaged with my distribution.

Test mosquitto

Open up 2 terminal windows and publish a message and in the other subscribe to the topic

mosquitto_pub -h localhost -t house/s1 -m "test" -d
mosquitto_sub -h localhost -t house/#

Now power up your ESP and publish something to a topic. I’m still using PubSubClient, so code could look something like that:

// Prepare a JSON payload string
// Prepare a JSON payload string
  String payload = "{";
  payload += "\"temperature\":"; payload += temperature; payload += ",";
  payload += "\"pressure\":"; payload += pressure; payload += ",";
  payload += "\"humidity\":"; payload += humidity; payload += ",";
  payload += "\"altitude\":"; payload += altitude; payload += ",";
  payload += "\"light\":"; payload += light; payload += ",";
  payload += "\"wind\":"; payload += wind; payload += ",";
  payload += "\"batvoltage\":"; payload += batvoltage; payload += ",";
  payload += "\"pvvoltage\":"; payload += pvvoltage; payload += ",";
  payload += "\"batperc\":"; payload += batperc;
  payload += "}";
 
  // Send payload
  char attributes[payload.length() + 1];
  payload.toCharArray( attributes, payload.length() + 1);
  Serial.print( attributes ); Serial.println(payload.length());
  if (  client.publish( "v1/devices/me/telemetry", attributes )) {
    Serial.println("Success");
  } else {
    Serial.println("Fail");
 
  }

Again, subscribe with mosquitto_sub to the topic and watch the data coming in.

~ $ mosquitto_sub -h localhost -t "v1/#"
{"temperature":19.84,"pressure":1017,"humidity":55.54,"altitude":-38.77,"light":0,"wind":0,"batvoltage":4039,"pvvoltage":2942,"batperc":83}
{"temperature":19.84,"pressure":1017,"humidity":55.54,"altitude":-39.04,"light":0,"wind":0,"batvoltage":4043,"pvvoltage":3225,"batperc":84}

influxdb

Download from the influx website. Packaged version was kinda old

~ $ influx
Visit https://enterprise.influxdata.com to register for updates, InfluxDB server management, and monitoring.
Connected to http://localhost:8086 version 0.10.0
InfluxDB shell 0.10.0
> create database telegraf

telegraf

Same thing, download and install from influx website.
Save original config at /etc/telegraf/telegraf.conf and create a sample config with an mqtt input and an influx output.
Then go to the last line and change the data_format from influx to json since where sending json encoded payload with PubSub.

~ $ telegraf -sample-config -input-filter mqtt_consumer -output-filter influxdb > telegraf.conf
~ $ grep data_format /etc/telegraf/telegraf.conf
  data_format = "json"
#    data_format = "value"

influx again

Fire everything up and take a look inside the database

~ $ influx
Visit https://enterprise.influxdata.com to register for updates, InfluxDB server management, and monitoring.
Connected to http://localhost:8086 version 0.10.0
InfluxDB shell 0.10.0
> use telegraf
> show measurements
> select * from mqtt_consumer

Grafana

Get it from the grafana website and install. I consider this as mandatory because my packaged version was something like 2.6 with an interface straight outta hell while the latest stable as of now is version is 4.6.

Run the thing, goto http://localhost:3000 and watch the magic.
Add a datasource, point it to your influxDB in proxy mode, no username for now.
Add a dashboard, add a graph or singlestat and play around a little.

Things in the screenshot I had to lookup:

  • Multiaxis can either be set dynamically by clicking on a legend item or can be setup in advance in Display->Series overrides
  • Table on the right is just a legend with everything ticked on
  • The battery gauge meter is a singlestat item with show gauge turned on in options
  • my metrics for the battery voltage are in mV but you can add a ton of transformations and aggregations in the metrics tab to any field. One of them being math(/1000)
  • Proxymode basically makes the Javascript in your browser talk to grafana and grafana to the (influx-)DB. In direct mode JS would directly talk to the (influx-)DB. So NAT needed, credentials in browser, bad things happen

That concludes Part 1 – my quick and dirty setup.
There isn’t really anything more to tell since the installation is so easy and there’s nearly no configuration required.

In Part 2 we’ll look into security and retention times because as of now, the database might just overflow some time.