Thursday, April 4, 2019

Nodered, influxdb and Grafana

I had installed nodered to help monitor some of the systems on the boat and elsewhere some time ago. It has the ability to display some data, charts and plots with the extra "dashboard" module.
https://nodered.org/docs/getting-started/installation

I also installed mosquitto to collect data from the ESP devices and the OPis. I probably followed someone's notes, but I didn't seem to log it.
https://mosquitto.org/download/

It was OK for a while, but each time I restarted a display, it would start with no historical data. There was also no way to remove erroneous data. As I change things, plug in or forget to plug in things, I do get erroneous data points on occasion.

I tried sqlite/sqlite3 on op2e and croc for a while. It is like mysql, only a little simpler. I found an example flow for nodered and tried it for a while.
 https://www.youtube.com/watch?v=ccKspiI8FRw&feature=youtu.be
  https://flows.nodered.org/flow/36f2de38ed4bb62cf3a5e8fdc76d779e

Then I saw the youtube from Andreas https://www.youtube.com/watch?v=JdV4x925au0 and decided to try it.

It didn't take too long to get some data going into influxdb, but it did take some learning. The data had to be formatted just right to be inserted in a way that grafana could read it back.

For the simple case of a two values JSON string this is it. It did take a while fo make it this simple:


var tokens  = msg.topic.split("/");
msg.measurement   =tokens[1];       //get device  name from topic  level 3 /v1.6/devices/tokens[3]
var dest    = tokens[tokens.length-1];
inputjson   =JSON.parse(msg.payload);
msg.payload =inputjson;
return msg;
// End

For the more complex report for the disk space it took several days:


var tokens  = msg.topic.split("/");
sys = tokens[1];       

function extract(elem) {
    let extr = elem;// { fileSystem: "/home", percent: "96" }
   // node.warn(extr)
    var tag = {"host": sys, "filesystem": extr.fileSystem};
    msg.measurement = "Disk";
    msg.payload= [{"percent": extr.percent}, tag];
    node.send(msg);
    return extr;
    }
    
msg.payload.disk.forEach(extract);
// End

But it is working now.

The next hurdle was to get the data out into a graph with Grafana. A longer learning curve. And I'm not sure how to show it. Several windows for input all have to agree for the data to show up.

Overview


Just the battery monitor