x/net/http/httpproxy: no_proxy no longer matches an ip suffix when used as hostname

[nicholasngai]

Go version

go version go1.24.1 darwin/amd64

Output of go env in your module/workspace:

AR='ar'
CC='cc'
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_ENABLED='1'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
CXX='c++'
GCCGO='gccgo'
GO111MODULE=''
GOAMD64='v1'
GOARCH='amd64'
GOAUTH='netrc'
GOBIN=''
GOCACHE='/Users/nngai/Library/Caches/go-build'
GOCACHEPROG=''
GODEBUG=''
GOENV='/Users/nngai/Library/Application Support/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFIPS140='off'
GOFLAGS=''
GOGCCFLAGS='-fPIC -arch x86_64 -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -ffile-prefix-map=/var/folders/ts/hlpvmfc143v104lrvqjg58040000gn/T/go-build4286700706=/tmp/go-build -gno-record-gcc-switches -fno-common'
GOHOSTARCH='amd64'
GOHOSTOS='darwin'
GOINSECURE=''
GOMOD='/Users/nngai/test/go.mod'
GOMODCACHE='/Users/nngai/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='darwin'
GOPATH='/Users/nngai/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/usr/local/Cellar/go/1.24.1/libexec'
GOSUMDB='sum.golang.org'
GOTELEMETRY='local'
GOTELEMETRYDIR='/Users/nngai/Library/Application Support/go/telemetry'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/usr/local/Cellar/go/1.24.1/libexec/pkg/tool/darwin_amd64'
GOVCS=''
GOVERSION='go1.24.1'
GOWORK=''
PKG_CONFIG='pkg-config'

What did you do?

We are leveraging no_proxy in our environment which includes .1 in the list. In Go 1.24.0 and 1.23.6 and prior, this matched IP addresses such as 1.1.1.1. However, in Go 1.24.1 and 1.23.7, this hostname is no longer matched.

Here’s a simple repro:

package main

import (
	"fmt"
	"net/http"
	"os"
)

func main() {
	req, err := http.NewRequest(http.MethodGet, os.Args[1], http.NoBody)
	if err != nil {
		panic(err)
	}

	proxyUrl, err := http.ProxyFromEnvironment(req)
	if err != nil {
		panic(err)
	}
	fmt.Println("Proxy:", proxyUrl)
}

What did you see happen?

In Go 1.23.6 and 1.24.0, the behavior is as so:

$ http_proxy=http://non.existent no_proxy=.1 GOTOOLCHAIN=go1.23.6 go run main.go http://1.1.1.1/
Proxy: <nil>

$ http_proxy=http://non.existent no_proxy=.1 GOTOOLCHAIN=go1.24.0 go run main.go http://1.1.1.1/
Proxy: <nil>

In Go 1.23.7 and 1.24.1, the behavior is as so:

$ http_proxy=http://non.existent no_proxy=.1 GOTOOLCHAIN=go1.23.7 go run main.go http://1.1.1.1/
Proxy: http://non.existent

$ http_proxy=http://non.existent no_proxy=.1 GOTOOLCHAIN=go1.24.1 go run main.go http://1.1.1.1/
Proxy: http://non.existent

This is a breaking change to Go. 😞

What did you expect to see?

The same behavior that worked in Go 1.23.6 and 1.24.0 should still function in 1.23.7 and 1.24.1 according to the Go compatibility promise.

[nicholasngai]

This appears to be related to #71984 and its associated CLs.

[gabyhelp]

Related Issues

• net/http: provide better error message when proxy is misconfigured #29809
• net/url: proxy settings will be ignored when user credentials have a % sign in it #38166 (closed)
• net/http: ProxyFromEnvironment panics on malformed no_proxy contents #19536 (closed)
• net/http: $no_proxy handling requires domains to be prefixed with a . #4574 (closed)
• net/http: bad host for WriteProxy for HTTP schemes #36831 (closed)
• x/net/http/httpproxy: http_proxy breaking changes in 1.16.x #45227 (closed)
• affected/package: net/http #61047 (closed)
• net/http: http.Get fails with "connection reset by peer" after update to Go 1.23 but works flawlessly up to 1.22.8 #70139 (closed)
• net/http: http client not support `socks://` proxy #65766 (closed)

Related Code Changes

• vendor: update vendored x/net/http/httpproxy

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

[prattmic]

cc @neild @golang/security

[seankhliao]

This is for #71984, note that this was never documented to work
What's allowed is a suffix match against a domain, or a prefix (cidr) match against IP, which in general maps to control over the unmatched parts.
https://1.1.1.1 is a rare case where an ip address is a canonical name, but it doesn't change the fact that it's still an ip address that should be matched by cidr, rather than be treated as a dns hostname.

https://pkg.go.dev/golang.org/x/net/http/httpproxy#Config

[nicholasngai]

Sadly, there is no standardization of behavior of no_proxy matching. GitLab has a good write-up about this.

Notably, most software out there always treats IP addresses as hostnames like any other for purposes of matching, according to the table on the GitLab page. wget, Python, .NET, etc. always does string matching (no_proxy=.1 matches 1.1.1.1), and these pieces of software don’t support CIDR notation.

Would it be reasonable to support both for IPv4? IMO this is well-supported enough in the broader ecosystem among other runtimes and programming languages, and it feels like something that should be covered in the Go 1 compatibility guarantee since this used to work.

[seankhliao]

Notably curl doesn't support a suffix ip address match.
There's a security carve out for go1 compat, and it can be insecure to only match the last part of an ip address, like matching only the subdomain of a dns hostname.

In your case, actually specifying the full match doesn't seem to be much more work, and should result in overall more correct behaviour across a range of inputs.

[neild]

Suffix matching on IP addresses is incredibly weird. The IP hierarchy doesn't go in that direction; X.X.X.1 doesn't have any relation to Y.Y.Y.1.

Prefix matching does make some sense, and we support that.

I am curious whether any of the implementations that string match on IPs are also vulnerable to #71984, where an IPv6 address with a zone ID is improperly matched against a host pattern: "::1%.example.com" definitely should not match ".example.com".

I see several possible approaches we could take here:

1. Say that while it makes very little sense, we used to match .1 with 127.0.0.1 and therefore we will continue to do so in the future. Put in a special case to handle this. Be careful not to match against zone IDs. Backport.
2. More complicated version of (1): Say that while we shouldn't match .1 with 127.0.0.1, we shouldn't change our behavior in a minor release. Put in a special case to handle this for 1.23/1.24. Leave the new behavior in 1.25 and ongoing.
3. More complicated version of (2): Put in a GODEBUG as well.
4. Say that matching partial IPv4 addresses was a bug and leave the bugfix as-is.

[nicholasngai]

There's a security carve out for go1 compat, and it can be insecure to only match the last part of an ip address, like matching only the subdomain of a dns hostname.

Sure! I’m not saying that there aren’t security implications to doing suffix matching on IP addresses. The fact is this is a fairly precedented behavior among the majority of software outside of Go.

I see several possible approaches we could take here:

IMO (3) would be my preferred approach. It aligns with the philosophy presented in the compatibility blog post, and this doesn’t feel like a case for not doing GODEBUG because “doing so is infeasible.”

It definitely is a security risk (I hope people aren’t generally implementing a no_proxy=.1 use case in production lol) but suffix matching IP addresses is not urgent one unlike the IPv6 zone IDs exploit. It’s probably worth removing in Go, but gingerly in a new 1.25 version rather than in 1.24.1 and 1.23.7.