Well I'm glad you finally found me! I probably don't use enough of the right "keywords" in my titles or descriptions to 'optimize' my videos for ranking in the search results. But the UKposts algorithm is a big secret mystery and I simply don't waste time trying to figure out the best way to get a video ranked or shown on the home page, when I could be using that time to make additional videos. Although this one had both MQTT and Home Assistant in the title... so who knows how it works? Regardless, I'm happy that you found the video helpful... and don't hesitate to reach out if you have any follow-up questions. Thanks for watching (after you finally found me)!

You are most welcome. And thank you so much for the kind comment. It is much appreciated!

You are most welcome! I know a lot of folks have asked me about it... and it can be a bit confusing when first starting out. There are other alternatives, like ESPHome (as someone else mentioned), but depending on the platform and complexity of your project, ESPHome might not work or be a good fit. Even if not building your own code, other platforms like Tasmota or the newer Shelly devices either use or make MQTT available, so it's a good idea to at least have a basic understanding of how it works. Thanks for watching.. and taking time to comment!

You are welcome. I'm glad my video was able to help you out. Thanks for taking time to leave a comment.

You are welcome! I was much the same way. I started using MQTT with Tasmota devices (well before there was a Home Assistant integration for Tasmota) and was really just copying and pasting the YAML needed without really understanding it either. It wasn't until I needed to add my own MQTT to a project that I really started to both understand it and see how useful it could be. Thanks for watching... and taking time to leave a comment. I appreciate it!

ESPHome is fantastic... if you are using an ESP (or one of the handful of other related MCUs). But it is when you want or need to use something like a Raspberry Pi, Arduino or other platform/language where ESPHome cannot be used that MQTT becomes the easy and simple way to integrate into Home Assistant. I just have MQTT included in my standard Arduino/C++ template I use for any new projects so it is there to integrate into Home Assistant if or when I decide to do so. Thanks for watching and taking time to comment. Let me know if you run into any questions along the way.

You are very welcome... and I'm glad you found the video helpful. Most people (myself included) seem to struggle a bit with the concepts behind MQTT at first. I tried to break it down to something that folks might be able to relate to. But once you get the hang of it, it is one of the best ways to integrate custom devices into Home Assistant that don't have a native or add-on integration. I use timers a lot in my automations. They are very handy for a lot of different purposes. For me, timers and input_booleans are a couple of the most useful "helpers" when it comes to creating and managing automations. Thanks for watching and thanks so much for taking time to post your kind comment. I greatly appreciate it!

Yeah... I do tend to make lengthy videos compared to the average . I try to always include the 'why' as well as the 'how' for beginners which result in long videos, but usually include chapter links for those that are a bit more advanced and want to skip certain parts. Glad this one didn't bore you too much! As I mentioned in the video, I struggled with the concepts of MQTT at first. But now it is my go-to for any of my projects that do not have a native integration. My C++ and Python starting frameworks both have MQTT stubs included... even if I don't initially intend to integrate them into Home Assistant. The MQTT is there if I ever want to integrate it.... or even control the device through another system as long as that system can publish an MQTT message. ESPHome and other firmware like Tasmota have their place. For ESPHome, if I find that I'm creating a lot of lambda raw C++ to accomplish what I need, then I figure I'm better off just writing in directly in C++ myself (I'm a somewhat late learner of C++, but I love it and prefer it over other options like Python in most cases). YAML will come to you. It's biggest issues (in my opinion) are trying to figure out if something is a dictionary without leading dashes or a list with leading dashes. And of course the very strict indentation rules. While I stick to YAML and don't really use the UI, the improvements is the capabilities with the UI means you can get away without even using YAML in Home Assistant these days, except for some really advanced or corner cases or you want complete control over where your code lives. Thanks for watching... and for taking the time to share such a nice comment. It is comments like yours that motivate me to continue making videos.

Glad it helped out! I too struggled to understand and use the retain flag properly when I first started implementing MQTT on my own ESP devices and it took a lot of troubleshooting of a device that I had that was exhibiting the "ghosting" behavior until I finally figured out what was going on and how the retain flag was causing that behavior. It's also why I now create separate state and command topics for my MQTT devices. Yes, you can get a way with using the same topic for both if handled carefully, but by having separate topics, one retained and one not, it is much easier to manage (at least for me) and avoids the issue of having a topic retained when it shouldn't be. Thanks for watching... and taking the time to leave a nice comment.

@@ResinChemTech I have used Shelly devices. I can see now that they use something similar, but it never clicked. Looking at documents never really conveys the concept, but pictures and words work WAY better. I am really ready to make a detection device for my front door and doorbell. Maybe even a light included. Thanks again! (I also subscribed. I will look over your earlier vids tomorrow. I found this one by accident. I know that is NOT what a UKpostsr wants to hear, but at least it came up.)

Yeah, Shelly and Tasmota devices tend to use /tele (for telemetry) instead of /stat (for state). But both are basically serving the same purpose. I just selected /stat for my own devices as it keeps all my devices grouped together and separate from Shelly/Tasmota devices when I look at them using something like MQTT Explorer. But as you know, you can pretty much use any valid topic you like. Thanks for subscribing! I don't do a lot of social media promoting of my content, and really only use Discord with any regularity, which is why most people probably just stumble across my channel much like you did. But I'm glad you found me! Feel free to shoot me any questions or thoughts you might have on my other videos.

As I mentioned, there are other ways of doing this... the MQTT device can also update the same topic as the command. But if not handled carefully, it can still lead to occasional ghosting. It takes a little bit more code up front to do it the way I am, with different stat/ and cmnd/ topics for states and 'command's, but it makes it very easy to know which messages should be retained and which shouldn't. But everyone has their own techniques or methods. A lot of people and devices, use tele/ instead of stat/. I use stat/ because when I look at my MQTT Explorer, everything under the stat/ branch is basically mine... then broken down by the device name... stat/brlamp... stat/garagelight, etc. It's just my own personal way of organizing all my MQTT topics. But it by no means the only way! Thanks for watching... and taking the time to leave a comment!

Also like the separation of command and status. This way you can see if the device got its message correctly. Thanks for showing more options how to use mqtt.

You are welcome! Glad you found something that might be helpful for you.

You are most welcome and I hope you found it helpful. And thanks for taking the time to leave your kind comment. I greatly appreciate it.

You are most welcome. Thanks for taking time to leave a comment.

You are most welcome. I'm glad you found it helpful. I know that MQTT was a bit confusing for me at first, but it is extremely handy for your own DIY projects once you get the hang of it.

You are welcome. Thanks for taking time to post a comment!

As long as you have a broker all those devices can communicate with each other. And if your broker is standalone, Home Assistant wouldn't even be required... or you could substitute another MQTT-compliant automation platform like NodeRed. While other options like APIs can have advantages, it's hard to beat the simplicity, speed and local lightweight nature of MQTT. Thanks for watching... and the kind words. Glad you found the video useful!

I appreciate it... and hope it made some sense for those looking to use and better understand MQTT.

@@ResinChemTech Helped me- MQTT is not really the problem when you are starting it is understanding how to write JSON / Python/ etc. at least for me!

I actually just finished up another project today that uses Arduino/C++ on an ESP8266 (a D1 Mini) at this point. And of course it is integrated into Home Assistant. It has local push buttons for some of its features, but it is nice to have control of those features via Home Assistant and to be able to also use those in automations. I've gotten to the point, that my starting Arduino templates that I use for ESP8266 and ESP32 already have the MQTT code built-in.... even if I don't use it (although that is rare... I try to integrate everything into Home Assistant... even if it is nothing more than on/off or rebooting the controller remotely). And I hear you on the 'never finished'! I can't stop tweaking and adding new things to pretty much anything I build. I always think of some other feature or option that I could include in MQTT and control via Home Assistant. Thanks for watching and taking time to comment. Don't hesitate to let me know if I can help you out if or when you decide to move to Home Assistant.

Thanks! Glad you found it helpful. Just let me know if you run into any questions for this or any of my other videos.

Thanks for the kind words and I'm glad you found the video helpful. I do attempt to try to explain both the why as well as the how in most of my videos, as I find the "why" was lacking in a lot of the 'how-to' videos I watched when first starting out. It does tend to make my videos much longer, but I try to always include the chapter links so if there are parts of less interest, those sections can be skipped. I agree that the official Home Assistant documentation could be improved. Part of the issue is how rapidly Home Assistant changes. Often, I will make a video about Home Assistant and within six months that video is already out of date or contains incorrect info due to new changes. But I remember how confusing it all was when I first started, so if I can do just a little to help others just starting out, I feel like I've made at least a little bit of a contribution. Thanks for watching and taking the time to post such a kind comment. I appreciate it.

You are welcome. And thanks for the comment!

I agree that MQTT can be a bit confusing and overwhelming initially. I too struggled to grasp it at first. But after a little bit of study... and just playing with it by sending commands to MQTT devices and watching data come back, it started to make sense. And once you get the hang of it, you'll soon discover how great it is for tying different devices together and allowing them to "talk" to one another via the broker... even if they are speaking different base "languages". Stick with it... for me, it suddenly clicked into place once I started using it in one of my own projects.

In the interest of time, I could not go into all the details of the various MQTT options. But LWT and keep alive can be configured on the client side when establishing a connection to the broker (the exact format will vary slightly based on the language used). LWT is a message published by the broker on the client's behalf if the connection is lost. The keep alive time is how long to maintain an active connection between the client and broker. If you want to know more about these options (and many others), the following site is a really good source for MQTT information: www.hivemq.com/blog/mqtt-essentials-part-9-last-will-and-testament/

Well, it's a little hard to tell what is going on for sure based on the information above and without being able to see the full code, but it does sound like an MQTT issue. I would experiment with changing the retained status (and maybe use something like MQTT Explorer to monitor exactly what is being sent/retained) and see if the ghost switching goes away. If it does, then you know it is a retained flag issue. If the random switching continues, then it is due to something other then MQTT messaging. Let me know what you find.

I just wanted to let you know that UKposts blocked your last comment. It will automatically block certain comments (those with links, certain words, etc.) But I was able to see the code. First, I see where pub3 and sub3 are defined, but your code uses other topics like pub1 and sub1 that aren't defined in the code you supplied. I'm guessing these are defined elsewhere and refer to the other relays? Again, I'd recommend that you try altering some of the publish statements and change the retained parameter from true to false and see if it resolves the ghost switching. At least that way, you can determine whether the random switching is a result of MQTT or caused by another issue. If all your MQTT topics are published with retain 'false' (at least temporarily for testing) and the random switching still occurs, then it is either not MQTT or something else (like a Home Assistant automation) is sending an MQTT topic... but you should be able to see if that occurs by watching the MQTT messages with MQTT Explorer (or other tool). But also verify with the multiple sub/pub topics that you aren't inadvertently publishing to the topic for one relay when you meant to publish for another (e.g. using the incorrect pubX topic). This could also appear to be 'random' switching. That's about the best I can offer at this point. UKposts comments are somewhat limiting in terms of trying to assist in troubleshooting more advanced issues... especially those that involve large code blocks.

Glad my video was helpful! Like you, I'm "old school" and do everything in YAML in Home Assistant. Not only automations and scripts, but helpers, etc. I don't think I've used the UI for anything (when I show the UI in a video, it is just for the video and I don't actually save it). Part of the reason is that I had to learn YAML when I started out with Home Assistant (that was the _only_ way back then) and since I took the time to learn it, I'm going to use it! The other reason is that I like to keep all my automations, entities, etc. in Home Assistant packages. The UI throws all automations into a single great big automations.yaml file and you can't add your own comments to the code, control where the files are placed, etc. But for those new to Home Assistant, the UI is fantastic and I don't fault those that use it. I just prefer my YAML. As far as documentation... I hear ya' when it comes to the "official" Home Assistant documentation. Home Assistant updates so often that the documentation often doesn't keep up... or new features really aren't documented at all. I don't know if I can point you at a single source for MQTT sensors in Home Assistant as anything I learned from years ago is sure to be out of date. Learning to use templates is a huge help (but then you have to learn a bit of Jinja as well!). If you haven't watched any of Jeff's videos over at Slacker Labs ( www.youtube.com/@SlackerLabs ), he has a few tutorials on creating and using templated sensors that are pretty good. And of course I'm happy to try to help and answer any questions you might have as well. Thanks for taking time to leave a comment!

Here are the brief answers to your questions: a) MQTT does use the TCP protocol for communication b) This is a bit too long to fully answer here, but I showed what I consider as the pros and cons of each method. Personally, I'd like to be able clone my entire Home Assistant production over to a test system. Having MQTT integrated into Home Assistant makes this more difficult. But maintaining a separate VM/Docker container, etc. also comes with its own additional overhead. As I said, the HA add-on works great, but for additional flexibility in my own use case, I'd put it in a separate install/container if I were to start over. c) For me... there isn't much of anything. If I were a NodeRed user, I would consider that in its own VM/Docker, etc. This is just because, similar to MQTT, if it was separate from Home Assistant, it would continue to run and function even if Home Assistant were to go down for some reason. Other than that, I don't really have anything else that would make sense to split out. Thanks for taking time to post a comment. If you want more details or to discuss further, you already know how to reach me!

c) You could have a look at 'IOTstack'. Some clever people have thought about this and done a load of work using Docker containers running: NodeRed, InfluxDB, Grafana, Zigbee2MQTT etc.

I'm not sure I completely understand what you are asking. You normally only have one broker... this can be either the HA MQTT Add-on broker _or_ a standalone MQTT broker. You generally would not have both unless you have multiple networks or other special situations. Each device or application either publishes data TO the broker or subscribes to get data FROM the broker (or both). You connect to the broker for each device or application... and normally that connection is done by specifying the IP address of the broker, along with a User ID and password. I hope that answers the question you were asking.

I'm not completely sure if I'm answering what you are asking, but do you have these entities integrated and showing in Home Assistant? In other words, can you see things like the accelerometer and gyro in the dev tools->states? If so, note that while a device class isn't necessary, to be able to plot a historical graph, the value has to be numeric. So check the actual values being returned by MQTT. If not numeric, then you may need to create a templated sensor based on the return value so that you can get a true numeric value. You should then be able to graph the history of this numeric templated sensor. Again, sorry if I am misunderstanding what you are asking.

You are welcome! You can find the full YAML code for the dashboard, MQTT entities and automations for the project shown (matrix clock) in my Github repo: github.com/Resinchem/LED-Matrix-Clock_WS2812b If you want to see more examples of Home Assistant/MQTT YAML, then you might also take a look at my repo on the DIY Arylic amp project, as it also makes heavy use of MQTT... and also has all the Home Assistant YAML: github.com/Resinchem/Arylic-Amp-MQTT

There really isn't a 'rule' other than those imposed by the broker, but there are some best practices or recommendations... which of course vary widely. Some common themes however are to always use lower case letters, avoid spaces and special characters, don't start with a forward slash, etc. You can find a lot of different recommendations just by Googling "MQTT Topic best practices". Personally, I think it is more important that you just create your own standard and stick to it. Your example above is perfectly acceptable. If you don't have a utility like MQTT Explorer yet, you might install that and take a look at the topics that are currently being created by Home Assistant and any other MQTT devices already in use. For me, I already had a lot of Tasmota MQTT devices (pre-integration) which used base topics of tele/ stat/ and cmnd/ followed by a device name then the actual 'action' part of the topic (like power, brightness, etc.) so I more or less just adopted that as the standard I use. But that's no more valid than starting with a device name, followed by state or command, then the actual 'action'. Or you could group your topics by location, starting with livingroom/ bedroom/ kitchen/ and then the device and rest of the topic after that. It's really what works for you.

@@ResinChemTech Thanks for your feedback.

There is really nothing special about the dashboard. It does use the custom button card, which I covered in a recent video. And you can find all the code, including the YAML code for the dashboard, in the Github repo for this project: github.com/Resinchem/LED-Matrix-Clock_WS2812b The dashboard code is published as lovelace.yaml in the /homeassistant folder of the repo. And there is some additional information in the wiki of that same Github repo.

Noted. I do try to assure I'm using dark mode for most code windows (like I did for the Arduino/Python code), but the Home Assistant code editor is that blinding white. I'll be sure to try to use another method moving forward.

@@ResinChemTech Maybe code-editor/VS Code... Thanks, I have bad vision but everybody loves dark mode...👍

Not completely sure what you are asking here... may be a bit different use of terminology for things. But my setup is basically the same as yours... I'm running Home Assistant on a Proxmox box and accessing it via the browser (Chrome) using Home Assistant's IP address. But I am able to cut/copy/paste code in both the File Editor and in the ESPHome editor. I'm using Windows, so the Ctrl-C, Ctrl-V for copy/paste works. I can copy text from really just about anywhere... Github, blog, another text editor, etc... and paste it into either the File Editor or the ESPHome editor within Home Assistant. You can also try a 'right-click' of the mouse to bring up the cut, copy, paste options... but for some reason, I do see cut/copy but not paste in the pop up menu in the ESPHome editor. Out of curiosity, if you copy something from say a Github example, can you paste that into a local text file (not in Home Assistant), even using something like Notepad? If you are asking something else please let me know, but I am able to copy text from outside files and paste those into either the File editor for YAML files or into the ESPHome node editor. One thought... are you logged into Home Assistant as the owner or an administrator? And do you have 'advanced' mode enabled? If either of those are not the case, it might prevent you from editing any files... and therefore, paste would not be available. Click on your icon in the lower left corner. In the profile, at the top see if the account is an owner or administrator. Then scroll down and see if the 'advanced' mode is toggled on. If you are unsure of accounts, go to the System -> People. This could likely be your issue.

@@ResinChemTech Hello and thanks for the help. Much appreciate it, so yes, advanced is selected and the keyboard shortcuts do work so everyone that’s copying pasting must be using the keyboard shortcuts, which I never have before. Mine is the same as yours. The copy is there, but not the paste with the mouse. Also, just curious how you got the side-by-side view and not sure some are using the built-in file editor and some are using the VS code not sure which is the best and some have mentioned that you need to be using Samba Share to edit the config. So I wasn’t sure what the issues were. Your videos are good. You’re very good at explaining things you must be a teacher.

Well, jumping multiple versions of Home Assistant generally always results in things breaking. Between version 2021.6.5 and 2022.12.8 (I'm guessing .8 as there wasn't a .18), there is a year and a half of breaking changes to contend with! That is a LOT of breaking changes... and I know that there were multiple MQTT breaking changes in there as well. In fact, 2022.12 introduced a major change in the way MQTT entities are defined in YAML. I know you won't find this very helpful, but I always recommend that you don't fall more than a couple of versions before upgrading, or you will often find yourself in this situation. There are so many things it could possibly be, I can't even begin to tell you where to look. You can try using the Home Assistant logs as a source to see if you can find errors. I don't think it has anything to do with moving from a Pi to a VM. If you can't find the particular error, the only resort may be to drop back to the 2021.6.5 version, then apply upgrades in order until the problem occurs. And I was running the current 2022.12 when I made this video, so it is something between the 2021.6 and 2022.12 that has broken your install/MQTT. Also be sure that your MQTT add-on has been upgraded to match Home Assistant too. Don't feel too bad. I made this same mistake early in my Home Assistant journey of skipping multiple upgrades and then faced a daunting task of trying to repair Home Assistant. I've since learned and now upgrade each month so I can address any "breaking changes" as they occur and impact my install. Sorry I can't pinpoint the exact issue. I'm happy to try to help out further, but with so many version upgrades, I'm not sure where to tell you to start looking.

I think this video is probably what you are looking for: ukposts.info/have/v-deo/j4mqhIN_qIOHjqs.html As long as you know the MQTT topics and types of payloads used by the device, this video shows you how to manually integrated and control that device from Home Assistant. Hope this helps you out!

I think I sort of covered that in the video when I talked about the retain flag. I write my code, and most well designed devices have a different topic for commands vs. states. So the command is sent on one topic (not retained) but the state is read on a different topic, which is retained. The command is sent to the device, which then returns the state... but the state change received does not cause a new command to be issued. This avoids the circular issue you are describing (if I understand what you are asking) and also prevents 'ghosting' issues when a device disconnects and reconnects to the broker and the state/command topics are the same and have been retained. If you didn't watch it, you might watch the section of the video on the use of the retained flag as it shows what I'm trying to describe.

Well, my matrix actually has dual controllers. If you are talking about the controller that shows patterns, etc. then that is the WLED controller and it doesn't use MQTT... it integrates into Home Assistant via an API and isn't even my code. If you are talking about the controller that shows a clock, scoreboard, countdown timer, etc. then that is custom Arduino code that I developed. You can see that complete code in the related Github repo: github.com/Resinchem/LED-Matrix-Clock_WS2812b If you look in the matrix_clock_main.ino file in the /arduino folder, you can see how I implemented MQTT. The sections that are probably of the most interest are the MQTT setup under the Setup() function (starting at line 371) and the callbacks (starting at line 581), although there are a lot of calls to publish MQTT values scattered throughout the rest of the code. If you are using a Raspberry Pi, then my guess is that you are also using Python. If so, then as I show in the video, the logic is basically the same between Arduino code and Python code... the syntax is just a bit different. So, you will have to take what I show in the Arduino code and modify it for Python. But the logic and processes are still the same. If you are new to the code and MQTT, this is probably still a bit overwhelming. It was to me at first as well. I recommend just trying to start out very simple. First, see if you can individually control a few pixels on your matrix. Once you have that, see if you can create a single MQTT command to call the procedure that turns on those few pixels. Then you just have to build out from there. It took me months to develop the code for my matrix, but I learned a ton in the process. So, hang in there, study online documentation and take it in small steps and you can get there! Good luck... if you run into any particular questions, let me know and I'll try to address them as best as I can (although there are limitations as to what I can do here in the UKposts comments).

@@ResinChemTech thank you so much for the in depth reply! I am new and yes, it is overwhelming lol.

Unless it's changed (I haven't installed a new Tasmota device in a while), Tasmota uses tele/ instead of stat/ for its "telemetry" values. You can use the same command topic for both (commands and states), but it can be tricky and certain situations can still lead to occasional ghosting. As I mentioned, there are other ways to do it, but using separate stat/ and cmnd/ topics is just the way I opt to handle retained vs. non-retained values. It leads to having to write a bit more code up front, but it's a bit easier (at least for me) to keep values that I want retained vs. ones that don't separate.

@@ResinChemTech thanks!

Well, Home Assistant by default does not have an MQTT integration. But it's extremely easy to add it. It is a standard add-on that you can find in the add-on store under Settings. This is the add-on, so it works both as your MQTT broker and as the integration for Home Assistant. Lewis over at @EverythingSmartHome has a really good video on installing and setting up the MQTT add-on in Home Assistant: ukposts.info/have/v-deo/nKGFomN-omqCxqM.html Of course, you can create your own standalone MQTT broker on another device (like a Raspberry Pi, VM, old PC, etc.) and then install the MQTT integration via Home Assistant integrations. Both methods have pros and cons, but using the Home Assistant add-on broker is definitely the easiest way and doesn't require a different machine/VM, etc. Sorry to point you at other sources. Unfortunately I sometimes have to make a call as to what to show and what to skip over (and assume that folks already know... or point them elsewhere) in some of my videos simply due to the length. But I do appreciate you watching. And don't hesitate to follow up with any additional questions and I'll do my best to help you out.

Well, it could be a number of things causing the disconnect. First, you didn't mention the broker you are using (e.g Home Assistant MQTT add-on, standalone, etc.), but whichever one it is, assure that you are updated to the latest version. Next, each client connecting to the broker must have a unique client ID. If multiple devices share the same ID, when one connects, MQTT will disconnect the other. I'm not a NodeRED user, so I'm not sure about the process that is used to connect to, and communicate with, MQTT. So I'm not much help on that front. But the 'not authorized' message generally means that you have the IP address, password or user name configured incorrectly when attempting to connect via the device. Be sure that you are using the local IP address of the broker and that you have a valid username/password for the broker. See this, which may help: community.home-assistant.io/t/mqtt-refusing-connection/437138

I have a /packages folder because I've opted to use Home Assistant packages for my configuration. This is not the default, which is to place everything within one big configuration.yaml file. The other alternative (which is what I used prior to packages) is to use a 'split' configuration, where you put things like lights, switch, sensors, etc. (or all your MQTT entities) in their own YAML files and then 'include:' the individual files in the main configuration.yaml file. I cover packages in another video: ukposts.info/have/v-deo/jph5rniPkKGHyas.html (If you just want to know about packages, you can skip forward to about the 10 minute mark in this video). In addition, I'll have another video coming out this Saturday that focuses solely on the Home Assistant side of setting up MQTT entities. This will significantly expand on the short segment that talked about the Home Assistant side in this video. I think it should provide much more information and details on the Home Assistant side of setting up MQTT devices and entities.

I'm not sure I follow. If the device is disconnected when the CMND is published with an "on" command, the STAT won't be updated... since as in my example, the STAT is _only_ published by the end device... which is disconnected. When the device reconnects, it will publish its current state... which in your example is "off". In this particular example, the only thing that happens is that the CMND that was issued is simply ignored (and dropped, since it has no retain flag and the device is disconnected). If the state of the light was "off" before the disconnection... and it isn't programmed to turn on when powered up or booted... then when it reconnects, the published state will still be off. At no point in this particular example does the "state" topic become out of sync with the actual state of the light.... as it is only the light itself that is updating the retained state topic. I've seen this exact situation in Home Assistant where I have a switch tied to a custom-developed MQTT device. When I toggle the switch to 'on', the switch in Home Assistant briefly turns on... then turns right back off again on its own. This is because it toggles on briefly when the command is sent, but because the device is offline, the retained state is still off. Home Assistant quickly updates the state of the switch right back to off from the last retained STAT topic (which was off at disconnection). This generally occurs within 1-2 seconds, and is one way I know when a device is offline (or at least dropped the MQTT connection). For this particular video, it was about DIY devices that you do completely control. In my case, I am the one that wrote the firmware/code that runs on the end devices. If it is a retail device, or firmware that you didn't develop, then you are correct. You are somewhat at the mercy of what the device/firmware developer provides and if it both sends and receives on the same topic, then you can still have issues. Some devices/firmware are (in my opinion) better than others. I like Tasmota if I want to use MQTT with prebuilt firmware because it does use different topics for commands vs. states, so I can follow the same process to assure I don't get ghosting or states that are out of sync. Sorry if I am misunderstanding what you are saying... but in my experience with my own MQTT devices, using the techniques I described has never resulted in an 'out-of-sync' situation between what Home Assistant is showing for a state and the actual state of the device itself.

@@ResinChemTechYou understood and answered wonderfully. I missed a few points in the video that you clarified above. My fault. The thing I am trying to wrap my head around is how to support in the complete system, the ability to have an automated "prime" that will set state that even is one part is down, when the part comes back up, it will return to state it "should be in". I have run into some of things you mentioned. I think my solution is to follow your "scheme" but have this "prime" subscribe to the state (from the broker) and make some "decisions" based on said state. The wrinkle is managing temporary state caused by human interaction compared to the automated state in "this failure state of a device disappearing and when it reappears it may default to wrong state as to what the prime wants. (in other words, I have a more complex situation to solve). Example is outside driveway lights, tune on at sundown, turn off an hour later. But if a user turns them off stay off. If the car proximity triggers turn them on and return to the timed turn off time OR if after a short timer to turn off. Several similar scenarios of automated lights with possible user override but also trying to maintain robustness when a device gets disconnected (from reboot caused by poor code and a watchdog, bad wifi, battery power lost due to poor solar, etc).

Hey, no problem! Sometimes I tend to ramble a bit in my videos and don't always make my points as clearly as I should. For the more complex situations, I will often make use of things like input_booleans and/or timers within Home Assistant. My interior garage lights (controlled by a switch relay that uses MQTT) is sort of a more complex situation similar to what you described. The lights come on via door switches then turn off after a set time. But the wall switch can also be flipped manually by a user, regardless of door state. For this, I use an input_boolean helper (along with timers) that sort of acts like that "prime" that you mention. I can toggle or change the state of that input_boolean via various automations depending on what happens with the lights... and can return the light to a desired state based on this input_boolean. This may or may not work in your situation, but I use input_booleans a lot in my automations to help control devices and states that might also be altered by a user with a different physical device. Don't know if any of that might be helpful to you or not!

I could be incorrect, but when you do a full HA restart, it also restarts all the add-ons... including MQTT. I do know that when I restart HA, I can watch the status at the bottom and I see it specifically say "Starting MQTT, not everything will be available until it is finished...." So, obviously MQTT goes down when the whole machine and/or host goes down, but I believe the MQTT add-on, and other add-ons, are also restarted on a full system restart as well, meaning it is down.. at least briefly, which can lead to the entity errors I showed. Now, this is in the full Home Assistant install (used to be called HassOS) and not a containerized version. If you have some other type of of install, then it may well be that you can restart the core without impacting MQTT. But if using the add-on with full Home Assistant, I do believe that MQTT restarts as part of the process.

@@ResinChemTech I certainly don't know everything but what I think: "MQTT is starting..." you mentioned means that MQTT integration is starting, not the broker (which is in its dedicated container and is running uninterrupted throughout HA restart). "Entity errors" you see also because of the MQTT integration starting - so you should see the errors also if the MQTT broker runs on an independent machine. Finally, I think that HA restart itself doesn't restart any of the add-ons. Anyone can confirm or deny our polemic?

@@ResinChemTech I forgot one more thing: I think that HassOS is also containerized.

No disagreement here! I am far from an expert on the inner workings of HA! You may well be correct (and probably are!). In my particular case, I do have 3 times a week that I stop the VM to do a full backup... so for me in that case, MQTT goes down then because the entire VM is stopped and restarted. And now that I think about it, you are correct about the integration vs. the add-on... and I would see those same errors even if I had an external broker with non-retained values. Sorry for my own confusion... and lack of knowledge about the inner workings of HA! That's why I love the feedback... I often learn new things myself!

@@ResinChemTech Absolutely no problem with that. I correct your mistakes, you correct mine - beneficial for both of us 😉

You might try watching the follow up video to this one, if you haven't already, that focuses specifically on how to set up the Home Assistant side for MQTT devices. ukposts.info/have/v-deo/j4mqhIN_qIOHjqs.html This video was a bit more of an MQTT overview. The linked video above has much more info on the specifics of the Home Assistant side. I hope that will help a bit.

And ESPHome is great for many cases.... I use it for a lot of my own devices. But for more complex project, like the clock I show in this video, ESPHome is not an option. I also do not believe you can install ESPHome on devices like the Raspberry Pi... or other devices that might use Python or other non-Arduino based languages. I believe that he only supported platforms currently are ESP8266, ESP32 and RP2040. ESPHome is a quick and easy way to put code on custom devices, especially those that just need to connect sensors, switches, etc. But for devices that need to do more advanced local processing with highly customized code... or if you want to develop in another language like Java, then MQTT offers an easy alternative to integrating your devices into Home Assistant.

I just watched it back on both the computer and TV and I'm not seeing any sync issues between the audio and video on my end. It may be a temporary UKposts thing.

​@@ResinChemTech, yep all good for me

Thanks for the confirmation! It always worries me a bit when I get a report like this, but I generally check and watch back all videos after I upload them and I didn't see any issues then... or after watching again based on this comment. But it's good to know that others aren't seeing the same issue as the original comment. Thanks again for taking a moment to let me know it is OK on your end.