Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pi-Hole does not appear to be forwarding all requests to unbound upstream #2235

Closed
fongd opened this issue Feb 19, 2025 · 21 comments · Fixed by #2239
Closed

Pi-Hole does not appear to be forwarding all requests to unbound upstream #2235

fongd opened this issue Feb 19, 2025 · 21 comments · Fixed by #2239

Comments

@fongd
Copy link

fongd commented Feb 19, 2025

Versions

Core version is v6.0 (Latest: v6.0)
Web version is v6.0 (Latest: v6.0)
FTL version is v6.0 (Latest: v6.0)

Platform

  • OS and version: Ubuntu 22.04.5 LTS
  • Platform: Bare metal install

Expected behavior

Pi-Hole is configured to use my unbound server running on localhost (port 5335) as a forwarder (the only one configured in Pi-Hole). Unbound contains the following stanza:

server:
	private-domain: "home.lan"
	domain-insecure: "home.lan"

	local-zone: "web.devel" redirect
	local-data: "web.devel 3600 IN A 192.168.8.10"

	local-zone: "enterprise.home.lan" redirect
	local-data: "enterprise.home.lan 3600 IN A 192.168.8.15"

This allows me to create any subdomain on the enterprise.home.lan domain such as scrypted.enterprise.home.lan and have it resolve to my server at 192.168.8.15. For example:

[user@Enterprise ~ ]$ dig scrypted.enterprise.home.lan @127.0.0.1 -p 5335

; <<>> DiG 9.18.30-0ubuntu0.22.04.2-Ubuntu <<>> scrypted.enterprise.home.lan @127.0.0.1 -p 5335
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 52666
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;scrypted.enterprise.home.lan.	IN	A

;; ANSWER SECTION:
scrypted.enterprise.home.lan. 3600 IN	A	192.168.8.15

;; Query time: 0 msec
;; SERVER: 127.0.0.1#5335(127.0.0.1) (UDP)
;; WHEN: Wed Feb 19 02:59:40 EST 2025
;; MSG SIZE  rcvd: 73

Since upgrading to Pi-Hole 6, Pi-Hole no longer seems to forward requests to unbound and instead returns an NXDOMAIN result.

If I query Pi-Hole for the same domain name (which should forward the query to unbound):

[user@Enterprise ~ ]$ dig scrypted.enterprise.home.lan @192.168.8.15

; <<>> DiG 9.18.30-0ubuntu0.22.04.2-Ubuntu <<>> scrypted.enterprise.home.lan @192.168.8.15
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 5099
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;scrypted.enterprise.home.lan.	IN	A

;; Query time: 20 msec
;; SERVER: 192.168.8.15#53(192.168.8.15) (UDP)
;; WHEN: Wed Feb 19 03:04:38 EST 2025
;; MSG SIZE  rcvd: 57

NXDOMAIN is returned instead of an IP address that unbound would be returning.

Here is how this is represented in the query log immediately after restarting the DNS resolver (i.e., cache flushed but query status indicates it was somehow served from cache):

2025-02-19 03:02:00 |   | A | scrypted.enterprise.home.lan | pi.hole | 11.8 ms | Deny
-- | -- | -- | -- | -- | -- | --
Query received on:  2025-02-19 03:02:00.008 Client:  pi.hole (192.168.8.15) Query Status:  Served from cache Reply:  NXDOMAIN Database ID:  2522

And from the FTL log:

2025-02-19 03:04:38.511 DEBUG_FLAGS Processing FTL hook from src/dnsmasq/forward.c:1910 (type: ?, name: "scrypted.enterprise.home.lan", id: 26)...
2025-02-19 03:04:38.511 DEBUG_FLAGS      Flags: F_FORWARD F_IPV4 F_QUERY 
2025-02-19 03:04:38.528 DEBUG_FLAGS Processing FTL hook from src/dnsmasq/domain-match.c:411 (type: ?, name: "scrypted.enterprise.home.lan", id: 26)...
2025-02-19 03:04:38.528 DEBUG_FLAGS      Flags: F_FORWARD F_NEG F_IPV4 F_NXDOMAIN F_CONFIG

However, if I query for something.web.devel (which also provides a similar wildcard feature in my unbound config), Pi-Hole correctly forwards that to unbound and resolves it:

[user@Enterprise ~ ]$ dig tester.web.devel @192.168.8.15

; <<>> DiG 9.18.30-0ubuntu0.22.04.2-Ubuntu <<>> tester.web.devel @192.168.8.15
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 9619
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;tester.web.devel.		IN	A

;; ANSWER SECTION:
tester.web.devel.	3600	IN	A	192.168.8.10

;; Query time: 3 msec
;; SERVER: 192.168.8.15#53(192.168.8.15) (UDP)
;; WHEN: Wed Feb 19 03:09:56 EST 2025
;; MSG SIZE  rcvd: 61

Actual behavior / bug

Pi-Hole does not appear to be forwarding all requests to upstream DNS (in my case, unbound running on localhost#5335).

Steps to reproduce

See above, not really sure how to reproduce this from scratch.

Debug Token

Screenshots

Image

Additional context

I upgraded from Pi-Hole v5.18.4. This configuration worked as expected then but no longer does following the upgrade to v6.0.

@yubiuser
Copy link
Member

How does the corresponding log line of /var/log/pihole/pihole.log looks like for

dig scrypted.enterprise.home.lan @192.168.8.15

@fongd
Copy link
Author

fongd commented Feb 19, 2025

This is all I get when I dig from .15 itself:

Feb 19 09:12:31 dnsmasq[91732]: query[A] scrypted.enterprise.home.lan from 192.168.8.15
Feb 19 09:12:31 dnsmasq[91732]: config scrypted.enterprise.home.lan is NXDOMAIN

@Pythbit
Copy link

Pythbit commented Feb 19, 2025

I am also having this issue with cloudflared as an upstream. I've rolled back to 5.18 for now.

@yubiuser
Copy link
Member

@fongd

You set home.lan as domain for your Pi-hole. This DNS domain is then purely local and Pi-hole won't forward any queries for this domain to an upstream server. From /etc/pihole/dnsmasq.conf

# DNS domain for both the DNS and DHCP server
# This DNS domain in purely local. FTL may answer queries from
# /etc/hosts or DHCP but should never forward queries on that
# domain to any upstream servers

@ejpenney
Copy link

Maybe I'm in the wrong place but I was getting no successful DNS resolutions whatsoever after upgrading to 6.0, neither internal nor external hosts were resolving. I do have a wildcard DNS configured for internal stuff, but I wasn't really able to test the bounds of the failure. My priority was to restore everything from backups and get everything to stop complaining, restore internet access. If/when I have a couple of hours I can try to do an upgrade on a test system and see if I can get better data.

I use unbound as my upstream, that's the reason I subscribed to this issue... It certainly looked this morning like PiHole was failing to forward to Unbound.

@yubiuser
Copy link
Member

Please open a new issue and post a debug log

@fongd
Copy link
Author

fongd commented Feb 19, 2025

@fongd

You set home.lan as domain for your Pi-hole. This DNS domain is then purely local and Pi-hole won't forward any queries for this domain to an upstream server. From /etc/pihole/dnsmasq.conf

# DNS domain for both the DNS and DHCP server
# This DNS domain in purely local. FTL may answer queries from
# /etc/hosts or DHCP but should never forward queries on that
# domain to any upstream servers

Thanks @yubiuser, clearing out the DNS domain restores forwarding of my home.lan domain to unbound.

However, there seems to be a disconnect between the comments in dnsmasq.conf vs what is described in the web UI:

Image

The DNS domains for your Pi-hole. If no domain is specified and you are using Pi-hole's DHCP server, then any hostnames with a domain part (i.e., with a period) will be disallowed. If a domain is specified, then hostnames with a domain parts matching the domain here are allowed. In addition, when a suffix is set then hostnames without a domain part have the suffix added as an optional domain part.

In my case, I am not using Pi-hole's DHCP server but I did have a domain specified so I was led to believe that setting this acted more like a search domain feature rather than telling Pi-hole to be authoritative for the domain.

Incidentally, I think the upgrade to Pi-hole v6 brought this over from my v5.x setup but I can't be sure. This is what I had in my setupVars.conf file of my previously-working v5.x:

PIHOLE_DOMAIN=home.lan

So maybe this is a documentation issue? Or maybe something was inadvertently changed with regard to this setting.

@ejpenney
Copy link

Hmm, started an entirely new LXC with PiHole Unbound on it, upgraded to 6.0 and every instance of unbound on my network crashed instantly and simultaneously. Can reproduce this 100% of the time: restore from backup, turn on the 6.0 machine, they all suddenly go offline and must be restored from backup, restarts don't help. I don't see anything in the obvious logs and unfortunately to dig deeper I'd need to do this in a safer manner, and I'm frankly not equipped to do any more isolation at this time. Looks like to thoroughly test this I would need to set up a separate router and a few test systems. Well outside the time I have to sink into this.

I guess my issue is different than OP, but I can't submit a ticket as the bug is far too invasive to analyze in my environment.

@norrist
Copy link

norrist commented Feb 20, 2025

I have a very similar issue.

  • I use local.lan as my home network domain.
  • Pihole is set for forward requests to my home router which does DHCP and serves DNS using unbound.
  • following the upgrade to 6.0, PiHole stopped forwarding requests for local.lan to unbound.

debug token is: https://tricorder.pi-hole.net/PqCtgDTt/

@yubiuser
Copy link
Member

@norrist

Queries for Pi-hole's 'domain' are never forwarded upstream. Change the setting in Settings > DNS

@araemo
Copy link

araemo commented Feb 21, 2025

I would agree with others on two counts:

  1. This should be documented better. It doesn't say anything about this preventing forwarding/upstream resolution of names ending with this domain.
  2. This should not be a default setting unless DHCP is enabled, or just not a default at all. (Or maybe default to 'pihole.lan' or something like that, that is unlikely to block a whole variety of in-use domains)

I also use a .myname.lan domain hosted on bind servers in my lan, and put pihole between certain clients and those servers to provide ad blocking. But I also need clients behind the piholes to be able to resolve my internal *.myname.lan addresses. Having this silently intercept all traffic for any *.lan domain out of the box is going to break a lot of peoples networks.

@yubiuser yubiuser transferred this issue from pi-hole/pi-hole Feb 21, 2025
@yubiuser
Copy link
Member

Transferred to FTL repo as changes in pihole.toml and dnsmaq default settings come from here.

@DL6ER
Copy link
Member

DL6ER commented Feb 21, 2025

The default domain is lan. If you configure a revServer for this domain, Pi-hole will NOT make it a purely local domain but instead it should work as you expect it to. This is a clash with your expectations and security considerations.

With domain = lan and no revServer, Pi-hole will never forward a.b.c.lan upstream for security reasons. Okay, in your case you trust your upstream because you control it. However, it could also be Google DNS or similar which you don't want to show internal network details. Local subnet IP addresses of the upstream are no reliable mechanism to determine if the server is trustworthy as you may have a local DoH or DoT gateway which would still forward your internal details upstream to the server. This restriction is meaningful independent of any DHCP settings.

Hence, Pi-hole enforces this "lan is purely local" unless you also specify a dns,revServer for the same domain. In this case, we lift the purely local nature and trust this upstream also for this domain.

Do you think there is an error in this logic (I don't think so)? But I see we should extend the documentation.

@DL6ER
Copy link
Member

DL6ER commented Feb 21, 2025

  # The DNS domain used by your Pi-hole.
  #
  # This DNS domain in purely local. FTL may answer queries from its local cache and
  # configuration but *never* forwards any requests upstream *unless* you have
  # configured a dns.revServer exactly for this domain. In the latter case, all queries
  # for this domain are sent exclusively to this server (including reverse lookups).
  #
  # For DHCP, this has two effects; firstly it causes the DHCP server to return the
  # domain to any hosts which request it, and secondly it sets the domain which it is
  # legal for DHCP-configured hosts to claim. The intention is to constrain hostnames so
  # that an untrusted host on the LAN cannot advertise its name via DHCP as e.g.
  # "google.com" and capture traffic not meant for it. If no domain suffix is specified,
  # then any DHCP hostname with a domain part (ie with a period) will be disallowed and
  # logged. If a domain is specified, then hostnames with a domain part are allowed,
  # provided the domain part matches the suffix. In addition, when a suffix is set then
  # hostnames without a domain part have the suffix added as an optional domain part.
  # For instance, we can set domain=mylab.com and have a machine whose DHCP hostname is
  # "laptop". The IP address for that machine is available both as "laptop" and
  # "laptop.mylab.com".
  #
  # You can disable setting a domain by setting this option to an empty string.
  #
  # Possible values are:
  #     <any valid domain>
  domain = "lan"

This would be my proposal - any suggestions for changes/improvements?

@DL6ER
Copy link
Member

DL6ER commented Feb 21, 2025

#2239

@yubiuser
Copy link
Member

I think the web interface text for this option also needs clarification

@DL6ER
Copy link
Member

DL6ER commented Feb 21, 2025

The web interface text is automatically generated from the description in pihole.toml to realize a true single-source-of-truth.

@yubiuser
Copy link
Member

Only in All Settings, not on the DNS page

@yubiuser yubiuser linked a pull request Feb 21, 2025 that will close this issue
5 tasks
@Matatu
Copy link

Matatu commented Feb 22, 2025

Seems it is still something wrong with the upgrade process, After upgrading to 6.0.x, domain "lan" was added, there was no domain entry in setupVars.conf before. Also "bogus-priv" was set, which was disabled before.

@rdwebdesign
Copy link
Member

After upgrading to 6.0.x, domain "lan" was added, there was no domain entry in setupVars.conf before.

This is somewhat expected. Not setting a value in setupVars.conf means "use the default value". In this case, lan is the default.

@DL6ER
Copy link
Member

DL6ER commented Feb 22, 2025

I guess this is now considered 'fixed' after changing the documentation? If not, please excuse this oversight, and simply reopen the issue

@DL6ER DL6ER closed this as completed Feb 22, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

9 participants