stayz3ro.dev

Moving HA DNS off the Raspberry Pis and onto Proxmox

The last DNS post ended with two Raspberry Pis running Pi-hole and Unbound behind a Keepalived VIP. That pair did its job for a while. But the whole network depends on it, and it lived on SD cards and USB power. Two single points of failure sitting in front of the thing meant to remove single points of failure.

This phase rebuilt that pair as Proxmox VMs and cut the production DNS path over to them. The client-facing VIP stayed the same the whole time. Everything behind it changed.

The shape of the move

The design is identical to the Pi version, only the hardware changed. Two nodes, each with its own Pi-hole and its own local Unbound, behind a Keepalived VIP. Clients keep pointing at 192.168.68.20. The new part is that the nodes are VMs on the Proxmox hosts instead of physical Pis.

NodeAddressRole
pihole-vm-01192.168.68.4primary, on pve01
pihole-vm-02192.168.68.5backup, on pve02
DNS VIP192.168.68.20owned by whichever node is MASTER
ashpi-1192.168.68.60retained as rollback
ashpi-2192.168.68.61retained as rollback

The physical Pis were not removed. They stayed plugged in as a tested fallback layer through the whole migration, which turned out to be the decision that made the cutover boring.

Pi-hole dashboard on pihole-vm-01 (192.168.68.4): Active, handling live
query traffic

Pi-hole dashboard on pihole-vm-02 (192.168.68.5): Active, the second
independent node behind the same VIP

Provisioning the VMs with OpenTofu

Rather than click through the Proxmox UI twice, I provisioned the VM shells with OpenTofu and a Debian 12 cloud-init image. OpenTofu defines the VM, cloud-init hands over the baseline, then Ansible-style validation confirmed the result. Same shape I’d use anywhere else, just smaller.

The order matters. Build the primary first, install and validate Pi-hole and Unbound on it, then do the same for the backup. Two nodes you cannot tell apart are the point, but you build them one at a time so a mistake is isolated to a machine that is not in production yet.

Two build-time lessons

Two things bit me on the first VM, both worth writing down.

The new VM had working IP connectivity but could not resolve deb.debian.org. Direct queries to the production VIP worked, so DNS itself was fine. The VM had inherited Tailscale MagicDNS resolver entries from the image baseline. A DNS server should not be using Tailscale for its own resolution. The fix was an explicit lab resolver baseline, set before any package install. Now a VM does not get packages until a DNS and apt-get update preflight passes.

The other one was snapshots. I tried qm snapshot before installing Pi-hole and it failed, because the disk layout on that storage does not support snapshots. So I used a vzdump checkpoint instead. Snapshots are a storage feature, not a Proxmox feature, and the storage was not snapshot-capable. Worth knowing before you count on one.

Keepalived, installed and deliberately off

Keepalived was installed on both VMs during the build and then left disabled. Installing it is not the same as activating it. The VIP stays on the physical Pis until the cutover phase. Turning Keepalived on early would mean two pairs fighting over the same address, which is exactly the failure you do not want to introduce by accident.

The discipline was: build everything, validate everything, and only then move the VIP in one controlled step.

The cutover

Phase 23 was the actual move. Keepalived configs were staged on both VMs. Then the VIP migrated from the physical pair to the VM pair in one window, with the ER605 DHCP and DNS settings untouched. Clients never noticed, because they already point at the VIP and the VIP is what moved.

Then the work that actually proves the thing. Phase 24 validated failover from pihole-vm-01 to pihole-vm-02, failback, and a controlled rollback to the physical Pis, with DNS continuity checked at each step. A cutover you cannot roll back is a bet, not a migration. The physical pair being there the whole time is what let me run the tests with confidence.

keepalived on pihole-vm-01: ip addr show includes the VIP
192.168.68.20, service active (running) — this node is
MASTER

keepalived on pihole-vm-02: the VIP is absent from ip addr show, and
the VRRP log shows the node entering BACKUP STATE — service active
(running), just not holding the
address

Hardening the pair

Phase 24.5 put the VMs on the production footing the Pis never really had. Tailscale went on both for administration, with --accept-dns=false so the MagicDNS problem from the build could not come back. UFW went on both with a default-deny inbound policy allowing only LAN DNS, SSH, the Pi-hole web UI, Tailscale, and VRRP. Then the failover and failback tests were run again with the firewall enabled, because a new per-node dependency is the kind of change that quietly breaks the standby path.

The monitoring targets were retargeted from the physical pair to the VM pair, with the Pis kept as fallback targets. Along the way I found a Grafana admin password sitting in plaintext in a Docker Compose inventory, moved it to a .env reference, and rotated it. Doing the firewall work meant looking at the monitoring config, and looking at the monitoring config surfaced a credential problem that would have sat there otherwise.

The cluster underneath

The last piece was standing up the Proxmox cluster the VMs now live on. A two-node cluster between pve01 and pve02, named stayz3ro-lab, over the management subnet. A two-node cluster has no quorum with two votes, so Proxmox HA manager stayed off. This cluster is a management foundation, not full infrastructure HA. The DNS pair handles its own failover through Keepalived, which is why the cluster does not need to.

pvecm status: Quorate: Yes, 3 nodes, quorum 2/3 met. The cluster has
since grown to three nodes (pve01, pve02, pve03) from the original
pve01/pve02 pair described
above

What carried over

The same discipline as the Pi build, on better hardware. Point clients at a VIP, never a node. Give each node its own recursive resolver so the two paths stay independent. Trigger the failures yourself and re-run them after every change. Keep a physical fallback that is tested, not assumed. None of that changed when the hardware did, which is the point of building it this way in the first place.

The physical Pis are now the fallback layer. They stay until the VM pair has proven itself for long enough that retiring them is its own decision.