cmd/go: GO111MODULE=on go build requires gcc

[nqv]

What version of Go are you using (go version)?

go version go1.11rc1 linux/amd64

Does this issue reproduce with the latest release?

Not sure

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOBIN=""
GOCACHE="/home/nqv/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/nqv/go"
GOPROXY=""
GORACE=""
GOROOT="/home/nqv/Downloads/go"
GOTMPDIR=""
GOTOOLDIR="/home/nqv/Downloads/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build126805613=/tmp/go-build -gno-record-gcc-switches"

What did you do?

Build a minimal http server failed with GO111MODULE=on go build

$ cd ~/go/src/httpd
$ cat httpd.go
package main

import "net/http"

func main() {
	http.ListenAndServe(":8000", nil)
}
$ GO111MODULE=on go mod init
go: creating new go.mod: module httpd
$ GO111MODULE=on go build
# net
exec: "gcc": executable file not found in $PATH

What did you expect to see?

GO111MODULE=on go build produces binary.

What did you see instead?

Build failed due to missing gcc.
Either of these commands works fine though:

$ go build
$ GO111MODULE=on CGO_ENABLED=0 go build

[mvdan]

It's surprising that GO111MODULE=on go build is trying to build the net package, or any other part of the standard library. How did you install Go? Does go build std do nothing, like it should?

cc @myitcv

[myitcv]

Edit: fixed bad copy-paste.

Reproduced:

docker run --rm -i -t ubuntu:16.04 bash
apt-get -y update < /dev/null > /dev/null
apt-get -y install wget < /dev/null > /dev/null
cd /tmp
wget -q https://dl.google.com/go/go1.11rc1.linux-amd64.tar.gz
tar -zxf go1.11rc1.linux-amd64.tar.gz
export PATH=/tmp/go/bin:$PATH
cd
mkdir -p go/src/httpd
cd go/src/httpd
cat <<EOD > httpd.go
package main

import "net/http"

func main() {
http.ListenAndServe(":8000", nil)
}
EOD
GO111MODULE=on go mod init
GO111MODULE=on go build

exits non-zero with:

# net
exec: "gcc": executable file not found in $PATH

whereas:

go build

just works.

/cc @rsc @bcmills @ianlancetaylor

[nqv]

I installed go1.11 from archive

$ cd ~/Downloads
$ curl -L -O https://dl.google.com/go/go1.11rc1.linux-amd64.tar.gz
$ tar xf go1.11rc1.linux-amd64.tar.gz

FYI, go build -v outputs

$ go build -v
httpd

$ rm -r ~/.cache/go-build/
$ GO111MODULE=on go build -v
golang_org/x/crypto/cryptobyte/asn1
golang_org/x/net/dns/dnsmessage
golang_org/x/crypto/internal/chacha20
golang_org/x/crypto/poly1305
golang_org/x/crypto/cryptobyte
golang_org/x/crypto/curve25519
golang_org/x/crypto/chacha20poly1305
golang_org/x/text/transform
golang_org/x/text/unicode/bidi
golang_org/x/net/http2/hpack
net
# net
exec: "gcc": executable file not found in $PATH
golang_org/x/text/unicode/norm
golang_org/x/text/secure/bidirule
golang_org/x/net/idna

$ rm -r ~/.cache/go-build/
$ GO111MODULE=on CGO_ENABLED=0 go build -v
golang_org/x/crypto/cryptobyte/asn1
golang_org/x/net/dns/dnsmessage
golang_org/x/crypto/curve25519
golang_org/x/crypto/internal/chacha20
golang_org/x/crypto/cryptobyte
golang_org/x/crypto/poly1305
golang_org/x/text/transform
golang_org/x/crypto/chacha20poly1305
golang_org/x/text/unicode/bidi
golang_org/x/text/unicode/norm
net
golang_org/x/net/http2/hpack
golang_org/x/text/secure/bidirule
golang_org/x/net/idna
golang_org/x/net/http/httpproxy
net/textproto
crypto/x509
golang_org/x/net/http/httpguts
crypto/tls
net/http/httptrace
net/http
httpd

[rsc]

It's not that surprising, and it's fine for Go 1.11.

[pda]

I find it surprising. It's a pain when there's no gcc.
Here's a minimal reproduction:

$ docker run --rm -w /notgo golang:1.11-alpine sh -c 'go mod init x && go build std'
go: creating new go.mod: module x
# net
exec: "gcc": executable file not found in $PATH

It looks like setting CGO_ENABLED=0 avoids the error, but I'm not sure what the implications are…

$ docker run --rm -w /x golang:1.11-alpine sh -c 'go mod init x && CGO_ENABLED=0 go build std'
go: creating new go.mod: module x

[tylercloke]

@rsc Can you expand a bit? It was certainly a surprise for me :) I started seeing this after upgrade to 1.11. Feels like a breaking change to the end-user if my build worked before upgrade and doesn't work now. Just want to understand what changed in more detail and what proper remediation is. Thanks!

[thepudds]

Regarding this issue, Russ commented:

Because the pre-built packages are non-module builds and can’t be reused. Sorry. Disable cgo for now or install gcc.

Note this issue is specific to opting in to modules (and related to the fact that Go 1.11 ships non-module based packages because the modules system overall is still opt-in only in Go 1.11).

This particular issue should not be affecting non-module based builds as far as I understand it.

[tylercloke]

@thepudds Thanks! Can you link me to the original context for that comment? Also, would we expect this to affect builds that pull in deps that might use modules? We did not opt into modules and were seeing issues on 1.11.

[bcmills]

We did not opt into modules and were seeing issues on 1.11.

The original report of this issue required GO111MODULE=on in order to trigger. If you're seeing these symptoms outside of module mode, please file a separate issue for them.

(You can check whether module mode is active by running go env: if you're in module mode, it will have a non-empty GOMOD line.)

[tylercloke]

There are two issues marked as dups of this that make no mention of modules. I can also repro without modules enabled:

#27303
#27285

I'll open another issue when I have time if you want but what I'm seeing looks pretty similar to the above.

[mfridman]

Noticed a similar issue in our pipeline when switching from 1.10.x to 1.11.0. Specifically with the 1.11.0-alpine3.7 docker image.

We explicitly set GO111MODULE=off and were seeing similar error as reported above.

exec: "gcc": executable file not found in $PATH

The solution, as mentioned before, was to explicitly set CGO_ENABLED=0. I suspect many folks will come across this particular issue when using lightweight containers such as alpine.

[thepudds]

@mfridman if you look at the symptoms in #27285, do those line up with what you are seeing? #27285 has an initial CL.

[mvdan]

We ran into the same issue when upgrading from 1.10.3-alpine to 1.11.1-alpine - errors due to gcc not being installed.

To clarify, this is when building a package inside GOPATH and without setting any Go env vars other than GOPATH and GOCACHE:

$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/go/src/brank.as/brankas-api/.go_cache"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/go"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build759719652=/tmp/go-build -gno-record-gcc-switches"
$ go test -p 8 -parallel 8 -coverprofile=coverage.out -count=1 -v ./...
# runtime/cgo
exec: "gcc": executable file not found in $PATH

For now I'll just globally set CGO_ENABLED=0, but I presume there's something to fix here since this worked in 1.10.

[thepudds]

Hi @mvdan , given you are seeing that behavior outside of modules (including an empty GOMOD reported by go env), what would you think about opening a new issue that covers the problem you saw in your immediately prior comment?

I think that was the suggestion from @bcmills several comments back:

The original report of this issue required GO111MODULE=on in order to trigger. If you're seeing these symptoms outside of module mode, please file a separate issue for them.

As far as I am following things, it sounds like there might be at least three distinct behaviors:

1. This original issue reported here in cmd/go: GO111MODULE=on go build requires gcc #26988, which might be surprising to a user but sounds like it is currently "expected" behavior with 1.11 due to the opt in nature of modules and because "the pre-built packages are non-module builds and can’t be reused".

2. Issue cmd/go: 'go build' in module mode rebuilds vendored dependencies in GOROOT #27285, which sounds like it is related to the -i flag (and which has one pending CL, but sounds like the solution might be to make -i a no-op in module mode)

3. The issue you are reporting, which sounds like it happened when modules are disabled (and some chance might be the same thing that @mfridman and @tylercloke reported, but that might be easiest to discuss and determine in a new/separate issue. Edit: @mvdan has now opened cmd/go: 'go test' on alpine and ubuntu requires gcc in GOPATH mode #28065 for this issue 3).

[mvdan]

Agreed, will open a new issue. I posted here first, in case I was missing something obvious or misunderstanding how GO11MODULE affected this issue.

[thepudds]

Hi @mvdan, makes sense that you posted here first. Overall, not 100% clear what's going on with the more recent reports here, and some chance there might be multiple underlying causes that might initially seem to have similar symptoms...

[rsc]

This is expected: we can only ship one pre-compiled copy of the standard library, and the pre-compiled copy we ship is for non-modules (GOPATH) mode. If we need standard library packages in module mode then we may need to rebuild them if there are any changes needed. In this case the slightly different effect of the vendor folder means that changes are needed.

One way to fix this for Go 1.12 would be to rename the top-level vendor folder to something different (internal/...) and fix the imports to match. We should probably do that.

[rsc]

Note the fix is to change the import path of the vendor folder.

[gopherbot]

Change https://golang.org/cl/147443 mentions this issue: vendor/golang_org: move to internal/golang_org

[bcmills]

CL 147443 fixed the known cause of these symptoms. Please open a new issue if you continue to observe them with a go command built after commit 2012227.