Observability, logging and MQTT

I like to have systems that monitor my home automation, but I’m finding this a bit challenging with the blaQ. ESPHome only has two logging methods that I know of, local when connected to a device or sending to MQTT. However it looks like there is no MQTT support, so that is out. As I continue to have occasional issues with my device going into a bad state I want to watch the logs. Any ideas on how to do this? I suppose one way would be to write a script that connects to the API and pulls the logs, but that seems a bit much right now. I probably will if I have to though. I’m just trying to think of ideas on how to monitor the logs. I think someone made a plugin for ESPHome that will send to syslog, but I’m a little hesitant to use that, not sure why.

1 Like

I am interested in this also. One of my blaQ openers is having issues.

It’s actually really easy to log remotely via esphome dashboard or command line and I’ve been meaning to write some documentation on this.

ESPHome dashboard (in browser)

If you’re using the ESPHome add-on for Home Assistant (or running ESPHome dashboard separately) you should be able to stream logs directly from there in the browser:

Screenshot 2024-06-05 at 11.48.51 AM

Command line method

  1. Install esphome if you don’t have it already. I work on a mac and I use homebrew to easily manage esphome CLI:

    brew install esphome and brew upgrade esphome when new releases are published.

  2. Download the base firmware config for your product from konnected-esphome. For the blaQ that’s garage-door-GDOv2-Q.yaml. Click the download button to save the file to your computer or just clone the open-source repo.

  3. Find the IP address (or internal hostname) of the blaQ on your network. You can use the Konnected app, your router, ESPhome dasboard, etc to find this.

  4. Run esphome logs in a terminal:

    esphome logs garage-door-GDOv2-Q.yaml --device 192.168.86.11
    

    or

    esphome logs garage-door-GDOv2-Q.yaml --device gdo-blaq-abc123.local
    
  5. Colorized logs will scroll in the terminal window. Press CTRL-C or ⌘-C to exit.

@nate thanks for the feedback. Yes, I do know how to connect and get logs no problem. I’m talking about something different here. I went to enable logging, to leave it running all the time and sending logs to some location to be stored. For example, I run Opensearch, I’m sure others run other central logging systems. I have multiple ways and places to accept the logs centrally, I just see no way to get ESPhome / GDO-Q to send them. I don’t want to leave my computer connected collecting logs. Other ESPhome apps will send all messages to MQTT, and I can subscribe to them that way. I realize this is not necessarily a GDO specific issue, more generally ESPHome, but there is a work around with some other apps. I’m curious your thoughts on this. I hope I clarified my question here. Thanks.

@danielgoepp gotcha. I understand what you’re looking for now. You certainly can enable MQTT for logging if you want to. The only limitation there is that we use the MQTT component for communication with Konnected Cloud (if enabled) so if you are using Konnected Cloud (for SmartThings and/or Alexa) then you cannot use your own MQTT broker at the same time.

To enable MQTT using your own broker, you’ll just need to edit and re-build your ESPHome config manually. It’s pretty simple to do, just adopt the ESPHome device into the ESPHome add-on/dashboard, edit the configuration, and add the mqtt: component. Here’s an example config on how to enable error logging to an MQTT topic.

Here’s a quick overview how to adopt an ESPHome device and edit the config.

I also found this custom component which adds syslog capability to ESPHome. I haven’t used it personally but I don’t see why it wouldn’t work.

Odd, okay. I thought I had tried that and it errored not supported, so that’s why I figured I would ping here to see what you thought. I will try again though to play around. I have my second unit now (haven’t install it yet), so I can play around without breaking my now finally working main garage door :slight_smile:

I don’t use Konnected Cloud for SmartThings or Alexa, so I should be good there.

Thanks again for the info.

Oh, and yes, I found that same plugin too, I mention it in my original post. I was nervous to use it…so for now I’m staying away. I might try it though, what’s the harm?

I was able to reproduce the error I think I was getting before, however I was able to resolve the issue. I’ll post here for anyone in the future that finds this post and experiences the same thing. When you first add mqtt you might getting errors. I will attach the log. However, all I needed to do to resolve it was to just a “clean build files” and I was good after that.
logs_garage-door-opener-car_run.txt (16.5 KB)

Okay, I think this will be my last post here. Like my last comment, this is in the spirit of sharing for future readers.

I have mqtt enabled now, and it works a charm. First, the support for Home Assistant works out of the box. The reason it works better for me is that I run Home Assistant and ESPHome in different subnets than my blaQ devices themselves. With MQTT enabled, when the device comes online, Home Assistant sees it no problem, without mDNS or adopting in ESPHome as an add-on, which I don’t run it that way. With my first device I was able to manually add it to HA by providing the IP. However with MQTT enabled, it publishes to the “homeassistant” topic and then HA prompts to add automatically. Just a little nicer for folks in my same situation.

I’m not sure yet if I’m in some split brain mode though. Other ESP Home devices I have use MQTT to control them. However, in the docs it only says this about alarm panels, not GDOs.

I’m assuming that control is still via the API, since I needed to enter my key still. It does beg the question of why? The API would not be needed if all the control was supported over MQTT. I think I read something somewhere about this, but I couldn’t find it. So, do both methods work? When I command the device from Home Assistant, is it using the API or MQTT?

Glad you figured it out!

Home Assistant can talk both the ESPHome native API and MQTT. Which connection method you’re using depends on how it’s added under Settings > Devices & Services. If it’s discovered/added using ESPHome, then it’s using the ESPHome native API. If it’s showing under the MQTT integration header, then it’s using that.

From ESPHome’s documentation:

Advantages over MQTT

The ESPHome native API has many advantages over using MQTT for communication with Home Automation software (currently only Home Assistant and ioBroker). But MQTT is a great protocol and will never be removed. Features of native API (vs. MQTT):

  • Much more efficient: ESPHome encodes all messages in a highly optimized format with protocol buffers - for example binary sensor state messages are about 1/10 of the size.
  • One-click configuration: ESPHome just needs one click to set up in Home Assistant - no more messing around with retained MQTT discovery messages and alike.
  • One less single point of failure: In the ESPHome native API each ESP is its own server. With MQTT, when the broker shuts off nothing can communicate anymore.
  • Stability: Since ESPHome has far more control over the protocol than with MQTT, it’s really easy for us to roll out stability improvements.
  • Low Latency: The native API is optimized for very low latency, usually this is only a couple of milliseconds and far less than can be noticed by the eye.
1 Like