Victron CerboGX MQTT help please

[gmoscalu]

Hello,

I am trying to pull Victron Energy's MQTT data into mqtt2prometheus and i have some issues.

I installed v0.1.7 on Ubuntu 22.04.4 LTS, hit the #140 #139 #138 and fixed them all (I discovered the issues after fixing them), started mqtt2prometheus service with the following config file:

mqtt:
  # The MQTT broker to connect to
  server: tcp://<LAN_IP>:1883
  # The Topic path to subscribe to. Be aware that you have to specify the wildcard.
  topic_path: N/xxxxxxxxxxxxxxx/#
  # The MQTT QoS level
  qos: 0
cache:
  timeout: 24h
json_parsing:
  separator: .

Service startup looks OK-ish:

May 27 22:17:34 Ubuntu-Monitoring mqtt2prometheus[620306]: 2024-05-27T22:17:34+03:00#011info#011mqttclient/mqttClient.go:20#011Connected to MQTT Broker
May 27 22:17:34 Ubuntu-Monitoring mqtt2prometheus[620306]: 2024-05-27T22:17:34+03:00#011info#011mqttclient/mqttClient.go:21#011Will subscribe to topic#011{"topic": "N/xxxxxxxxxxxxxxx/#"}
May 27 22:17:34 Ubuntu-Monitoring mqtt2prometheus[620306]: 2024-05-27T22:17:34+03:00#011info#011web/tls_config.go:195#011#011{"level": "info", "msg": "TLS is disabled.", "http2": false}

Whenever i query the metrics, with curl http://127.0.0.1:9641/metrics aside the go metrics, i only get received_messages{status="success",topic="N/xxxxxxxxxxxxxxx/[...........]"} lines, no metrics.

If i put it in debug by switching the command lines

#ExecStart=/usr/bin/mqtt2prometheus -config /etc/mqtt2prometheus/mqtt2prom_victron.yaml -listen-port 9641  -log-level debug $ARGS
ExecStart=/usr/bin/mqtt2prometheus -config /etc/mqtt2prometheus/mqtt2prom_victron.yaml -listen-port 9641  $ARGS

and then:

root@Ubuntu-Monitoring:/etc/mqtt2prometheus# vi /etc/systemd/system/mqtt2prometheus.service
root@Ubuntu-Monitoring:/etc/mqtt2prometheus# systemctl daemon-reload
root@Ubuntu-Monitoring:/etc/mqtt2prometheus# service mqtt2prometheus restart

I can see it connects to CerboGX, subscribes and gets many many many info from the MQTT.

Sample debug log:
syslog_mqtt2prometheus_sample.txt

i checked #82 - my JSON seems valid,

Is there a way i could use/configure this tool to export the data for Prometheus without actually defining 2 million metrics? The complete listing of paths is available on https://github.com/victronenergy/venus/wiki/dbus, so they're way too many to manually define...

Thanks,
Gabriel