One of my New Year’s initiatives is to move even more of my services from third-party providers back under my control. I have implemented a lot of home automation using Apple’s HomeKit, but I am hoping to move that to an open source solution, and I landed on Home Assistant (HA).
The main driver for choosing HA is to implement a “panic button” that can be used by my Dad. He is 86 years old and lives with us in a basement apartment, and recently he fell getting out of bed. It took us awhile to hear him. I don’t want to spend money on those “I’ve fallen and I can’t get up” lanyards so I figure if I put enough cheap Zigbee buttons around I can generate a notification like “Dad Pressed the Button Next to The Bed” or some such. I’m really close to getting that to work, so expect an update in another post.
But what I wanted to talk about now was an issue I ran into with HA.
I looked at various ways to install Home Assistant, and I went with the Home Assistant Operating System on a Raspberry Pi.
We had a winter storm come through this weekend, and while they were expecting things to be much worse, it wasn’t that bad. Still, I ended up staying inside most of the day so I had time to play. When I saw that HA could run on a Raspberry Pi 4, I knew the only challenge I would face would be finding mine. After searching for about an hour I found it in my office in a box clearly labeled “Raspberry Pis”.
(sigh)
Finding a 128GB MicroSD card was easier, and after installing the Raspberry Pi Imager and following the instructions, I was good to go.
For anyone who thinks open source applications are hard to use or don’t have a good user interface, you should try out HA. It was very easy to add my home automation devices to the instance, and if I had any complaint it would be that there were too many options that could be customized, and that is never a complaint I make.
I didn’t think too much about it until I ran into an issue this morning. I was on my work laptop and I suddenly couldn’t reach internal Amazon resources. Since there are layoffs planned for this week and I’ve been told the first sign is that your e-mail stops working, I figured maybe I had been affected. It was a little strange since they aren’t supposed to happen until tomorrow, so I investigated further.
I keep my work computer on the guest network, so I switched it to my personal network and things started working. After ruling out issues with the router I figured out it was DNS.
It is always a DNS problem.
I used Quad9 (9.9.9.9) as my default DNS. A few tests showed that it was, indeed, not responding to requests.
$ nslookup linkedin.com 9.9.9.9
;; connection timed out; no servers could be reached
$ nslookup linkedin.com 1.1.1.1
Server: 1.1.1.1
Address: 1.1.1.1#53
Non-authoritative answer:
Name: linkedin.com
Address: 150.171.22.12
$ nslookup linkedin.com 8.8.8.8
Server: 8.8.8.8
Address: 8.8.8.8#53
Non-authoritative answer:
Name: linkedin.com
Address: 150.171.22.12
$ ping 9.9.9.9
PING 9.9.9.9 (9.9.9.9): 56 data bytes
64 bytes from 9.9.9.9: icmp_seq=0 ttl=58 time=19.509 ms
64 bytes from 9.9.9.9: icmp_seq=1 ttl=58 time=19.946 ms
My personal network used 1.1.1.1 as a backup so that’s why it started working.
I went and looked online to see if there were any reported issues with Quad9, but their site said everything was okay. While I am not a paying customer of theirs I decided to open a support ticket in any case.
Quad9 support is amazing.
I sent the above snippet and Zachery replied asking for my IP address and a traceroute.
He then wrote back saying that my IP had been rate-limited for excessive DNS requests, which I found odd, but then he asked: Are you using Home Assistant by any chance?
It turns out that HA has a reputation for making lots of PTR requests to try and reverse-lookup the IP addresses it discovers.
There are a couple of ways to mitigate this.
The first is to disable the “DHCP Discovery” integration, which is actually a bit of a pain. Like many applications, HA is configured using a file, configuration.yaml, and the first entry is default_config:. That configuration starts a bunch of services, such as dhcp:, but to disable it you have to delete the default_config: line from the configuration file and then manually add back in all of the services, except the ones you want to disable.
The second is to use your router as the DNS provider. This will allow the router to respond to local network PTR requests.
So I did that and waited. I verified that the router was serving up the PTR requests:
$ nslookup 172.20.100.13 172.20.100.1
Server: 172.20.100.1
Address: 172.20.100.1#53
13.100.20.172.in-addr.arpa name = homeassistant.
and after waiting a bit I can now use Quad9 again:
$ nslookup linkedin.com 9.9.9.9
Server: 9.9.9.9
Address: 9.9.9.9#53
Non-authoritative answer:
Name: linkedin.com
Address: 150.171.22.12
I did get an alert from my OpenNMS instance that the CPU on my router was spiking, so maybe I haven’t stopped HA from making lots of requests, but I hope I’ve reduced the number of requests going to Quad9.
Just one more plug for Quad9. I got better support from them than I do many of the services for which I pay, and Zachary deserves a raise.