Building highly-available DNS for my home lab
Every device on my network asks DNS before it does anything else. For a while that sat on a single Raspberry Pi running Pi-hole. It worked, but a dead SD card or a bad update would take name resolution down for the whole house until I fixed it.
This is Project 1 of my home lab. I turned that single node into a
redundant DNS layer with automatic failover, local recursion, and
monitoring that checks what clients actually experience. Every address
below is a private RFC1918 lab address on 192.168.68.0/24.
The design
| Component | Role |
|---|---|
| Pi-hole | DNS filtering and query visibility, one instance per node |
| Unbound | Local recursive resolver, one per node, on 127.0.0.1#5335 |
| Keepalived | VRRP virtual IP that clients point at, floats between nodes |
| Gravity Sync | Replicates Pi-hole config (blocklists, allowlists, local records) between nodes |
| Prometheus, Blackbox, Grafana, Alertmanager | Probes the DNS path and alerts on failure |
Two nodes:
ashpi-1 192.168.68.60 higher VRRP priority (primary)
ashpi-2 192.168.68.61 VRRP priority 90 (backup)
DNS VIP 192.168.68.20 owned by whichever node is MASTERClients get 192.168.68.20 as their only DNS server, handed out by DHCP.
They never talk to a node directly, so a node can drop off without any
client-side change.
One thing to get right: the VIP has to sit outside the DHCP pool. If Keepalived claims an address the router has already leased, failover turns into an address conflict.
Recursion instead of a forwarder
Each Pi-hole forwards to its own local Unbound on 127.0.0.1#5335 instead
of a public resolver like 1.1.1.1 or 8.8.8.8. Unbound resolves from
the root servers itself, using a local root.hints file.
This matters in an HA setup. A shared upstream forwarder would be one dependency sitting behind both nodes. If it rate-limited me or went down, failover wouldn’t help, because both nodes would fail the same way. A local resolver on each node keeps the two paths independent.
Setup per node:
sudo apt install unbound
# drop in the Pi-hole Unbound config, fetch root.hints
sudo unbound-checkconf
sudo systemctl enable --now unbound
dig google.com @127.0.0.1 -p 5335 # direct recursion worksThen point Pi-hole’s upstream at 127.0.0.1#5335 and check that a normal
domain and a blocked domain both still behave.
Keepalived and the VIP
Keepalived runs VRRP between the two nodes. The higher-priority node holds the VIP. If its advertisements stop, the backup promotes itself and sends a gratuitous ARP so the switch updates its table. The VIP moves in about a second.
The VRRP log tail is the useful part. You can watch the instance go
BACKUP (init), then compare priorities, then MASTER STATE. That
transition log is the first place I look when failover misbehaves.
Keeping the two nodes in sync
A second node is only useful if it’s configured like the first one. When I built this, Gravity Sync was the standard tool for it. It replicates the Pi-hole gravity database, blocklists, allowlists, and local DNS records between nodes over SSH.
gravity-sync compare returning “No replication is required at this time”
is the healthy state. The nodes already match. I run it after any
blocklist change.
Gravity Sync has since been archived upstream. That doesn’t change the architecture, since the replication layer is a swappable piece, and there’s a separate write-up coming on how I noticed and what I moved to. It’s a good reminder to track a dependency’s health, not just its version.
Testing failover
Adding a second node is not the same as having high availability. You have to confirm the second node takes over. The test is blunt: stop Keepalived on the active node and watch what a client sees.
# on ashpi-1 (current MASTER)
sudo systemctl stop keepalived
The VIP shows up on ashpi-2 within a second or two. Then, from a client, the checks that matter:
dig google.com @192.168.68.20 # normal resolution still works
dig doubleclick.net @192.168.68.20 # filtering still works
Resolution and filtering both survive the failover because ashpi-2 has its own Unbound and its own synced Pi-hole config. Restart Keepalived on ashpi-1 and the VIP moves back. I ran this test before and after adding Unbound, because a new per-node dependency is exactly the kind of change that can quietly break the standby path.
Monitoring the path, not just the boxes
A dashboard that says “both Pis are up” can still miss a DNS outage. Unbound can wedge while the host looks fine. So the Blackbox Exporter probes DNS three ways:
192.168.68.60:53, ashpi-1 directly192.168.68.61:53, ashpi-2 directly192.168.68.20:53, the VIP, which is what clients use
The VIP probe is the important one. If the per-node probes are green but the VIP probe fails, the problem is Keepalived, not Pi-hole.
Alerting gets tested the same way as failover. It isn’t trusted until it
has fired on purpose. Stopping a node produces a SingleNodeDown warning
that routes through Alertmanager to Discord, and I confirmed both the
firing and the recovery notification.
Access
No SSH port is forwarded from the internet. Administration goes over Tailscale, so the management plane isn’t exposed even though the service itself is critical. The node-to-node SSH that Gravity Sync needs uses key auth between the two Pis only.
Closing notes
A few things I’d carry into the next build. Point clients at a VIP, never at a node, so everything behind it can be replaced without touching a client. Give each node its own recursive resolver so the two paths stay independent. Trigger the failure conditions yourself instead of assuming the feature works, and re-run those tests after any change to a node. And watch the client-facing path, because the VIP probe catches a class of failure that per-host monitoring can’t see.
None of this needs to be large. Two Raspberry Pis and Keepalived teach the same failover discipline as a rack of servers.
The physical Pis are the fallback layer now. The next phase moves the DNS stack into Proxmox VMs, with a controlled cutover that keeps the Pis as a tested rollback path.