cmd/link: sectionForAddress(0xA9D67F) address not in any section file [1.15 backport] #40974
Description
What version of Go are you using (go version)?
go version go version go1.15 darwin/amd64
Does this issue reproduce with the latest release?
Yes.
What operating system and processor architecture are you using (go env)?
go env Output
$ go env GO111MODULE="on" GOARCH="amd64" GOBIN="" GOCACHE="/Users/wk/Library/Caches/go-build" GOENV="/Users/wk/Library/Application Support/go/env" GOEXE="" GOFLAGS="" GOHOSTARCH="amd64" GOHOSTOS="darwin" GOINSECURE="" GOMODCACHE="/Users/wk/gopath/pkg/mod" GONOPROXY="gitlab.com/segmed" GONOSUMDB="gitlab.com/segmed" GOOS="darwin" GOPATH="/Users/wk/gopath" GOPRIVATE="gitlab.com/segmed" GOPROXY="https://proxy.golang.org,direct" GOROOT="/usr/local/Cellar/go/1.15/libexec" GOSUMDB="sum.golang.org" GOTMPDIR="" GOTOOLDIR="/usr/local/Cellar/go/1.15/libexec/pkg/tool/darwin_amd64" GCCGO="gccgo" AR="ar" CC="clang" CXX="clang++" CGO_ENABLED="1" GOMOD="/Users/wk/r/segmed/dataops/cli/go.mod" CGO_CFLAGS="-g -O2" CGO_CPPFLAGS="" CGO_CXXFLAGS="-g -O2" CGO_FFLAGS="-g -O2" CGO_LDFLAGS="-g -O2" PKG_CONFIG="pkg-config" GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/gx/4jcx3wqs70195ygr767r2xsr0000gn/T/go-build586495030=/tmp/go-build -gno-record-gcc-switches -fno-common"
Darwin wkoszek_mba 19.5.0 Darwin Kernel Version 19.5.0: Thu Apr 30 07:42:45 PDT 2020; root:xnu-6153.121.1~3/RELEASE_X86_64 x86_64
What did you do?
I decided to upgrade today:
20:04:15][wk@wkoszek_mba:~/r/segmed/dataops/cli]$ brew upgrade golang
==> Upgrading 1 outdated package:
golang 1.14.6 -> 1.15
==> Upgrading golang 1.14.6 -> 1.15
==> Downloading https://homebrew.bintray.com/bottles/go-1.15.catalina.bottle.tar.gz
==> Downloading from https://d29vzk4ow07wi7.cloudfront.net/36ef2e4cc3ecc84e0c1e98580a63f8d417efc8831f0c59b0a7e13d517c61353b?response-content-disposition=attachment%3Bfilename%3D%22go-1.15.catalina.bottle.tar.gz%22&Poli
######################################################################## 100.0%
==> Pouring go-1.15.catalina.bottle.tar.gz
🍺 /usr/local/Cellar/go/1.15: 9,769 files, 494.3MB
Removing: /usr/local/Cellar/go/1.14.6... (9,453 files, 424.7MB)
[20:05:29][wk@wkoszek_mba:~/r/segmed/dataops/cli]$ make
I upgraded only Golang, nothing else.
This resulted in my project crashing at building of test executable:
$ make t
../tools/pre-make.py
../tools/segmed_lint
gofmt -w *.go # formatting of Golang according to standard
goimports -w *.go # getting rid of too many imports/too few of them
gotags *.go > tags # tags for easy browsing
golint . # linting (static analysis) of Go code
go test -o segmed.test -v -cover -c # compile tests to the binary segmed.test. It doesn't run anything
# gitlab.com/segmed/dataops/cli.test
/usr/local/Cellar/go/1.15/libexec/pkg/tool/darwin_amd64/link: running clang failed: exit status 1
ld: sectionForAddress(0xA9D67F) address not in any section file '/var/folders/gx/4jcx3wqs70195ygr767r2xsr0000gn/T/go-link-819363190/go.o' for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [segmed.test] Error 2
I parametrized my makefile and fetched 2 versions of Golang from the website:
go1.14.7.darwin-amd64.tar.gzgo1.15.darwin-amd64.tar.gz
Reproducing of 1.15 issue:
$ make GODIRPREFIX=./go1.15.darwin-amd64/go/bin/
../tools/pre-make.py
../tools/segmed_lint
./go1.15.darwin-amd64/go/bin/gofmt -w *.go # formatting of Golang according to standard
goimports -w *.go # getting rid of too many imports/too few of them
gotags *.go > tags # tags for easy browsing
golint . # linting (static analysis) of Go code
go build -ldflags "-w -s" -o segmed *.go # actually building stuff
GOARCH=amd64 GOOS=linux go build -ldflags "-w -s" -o segmed.linux *.go
./go1.15.darwin-amd64/go/bin/go test -o segmed.test -v -cover -c # compile tests to the binary segmed.test. It doesn't run anything
# gitlab.com/segmed/dataops/cli.test
/Users/wk/r/segmed/dataops/cli/go1.15.darwin-amd64/go/pkg/tool/darwin_amd64/link: running clang failed: exit status 1
ld: sectionForAddress(0xA9D67F) address not in any section file '/var/folders/gx/4jcx3wqs70195ygr767r2xsr0000gn/T/go-link-400480254/go.o' for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [segmed.test] Error 2
Proof 1.14.7 is OK:
$ make GODIRPREFIX=./go1.14.7.darwin-amd64/bin/
../tools/pre-make.py
../tools/segmed_lint
./go1.14.7.darwin-amd64/bin/gofmt -w *.go # formatting of Golang according to standard
goimports -w *.go # getting rid of too many imports/too few of them
gotags *.go > tags # tags for easy browsing
golint . # linting (static analysis) of Go code
go build -ldflags "-w -s" -o segmed *.go # actually building stuff
GOARCH=amd64 GOOS=linux go build -ldflags "-w -s" -o segmed.linux *.go
./go1.14.7.darwin-amd64/bin/go test -o segmed.test -v -cover -c # compile tests to the binary segmed.test. It doesn't run anything
$
What did you expect to see?
I'd expect 1.15 to behave like 1.14.7
What did you see instead?
# gitlab.com/segmed/dataops/cli.test
/Users/wk/r/segmed/dataops/cli/go1.15.darwin-amd64/go/pkg/tool/darwin_amd64/link: running clang failed: exit status 1
ld: sectionForAddress(0xA9D67F) address not in any section file '/var/folders/gx/4jcx3wqs70195ygr767r2xsr0000gn/T/go-link-400480254/go.o' for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
More findings
The structure of the project is dataops/{cli, dephi, debug}. In cli/ dir I have work-in-progress file asset.go which depends on our private dephi module published on GitLab. dephi module just carries 1.8MB of string constants which are meant to be compiled into the package. In dataops/dephi, I use pkger to get:
-rw-rw-r-- 1 wk staff 1855629 Aug 21 20:47 pkged.go
Other than that, "dephi" is very trivial -- 1 .go file -- which returns constants as []string. Now the funny part: cli/asset.go is very-work-in-progress: it has AssetsList() which isn't called from anywhere. But by moving asset.go out of the project, I made it compile fine with go1.15.
To isolate whether asset.go is a culprit I made debug/ directory, with some mocks, but I also depend on asset.go and I use dephi. It works OK:
$ make
../tools/pre-make.py
../tools/segmed_lint
gofmt -w *.go # formatting of Golang according to standard
goimports -w *.go # getting rid of too many imports/too few of them
gotags *.go > tags # tags for easy browsing
golint . # linting (static analysis) of Go code
config.go:3:6: exported type Config should have comment or be unexported
main.go:5:5: exported var GlobalVar should have comment or be unexported
go build -ldflags "-w -s" -o segmed *.go # actually building stuff
GOARCH=amd64 GOOS=linux go build -ldflags "-w -s" -o segmed.linux *.go
go test -o segmed.test -v -cover -c # compile tests to the binary segmed.test. It doesn't run anything
[21:57:28][wk@wkoszek_mba:~/r/segmed/dataops/debug]$ ls -la
total 19380
drwxrwxr-x 15 wk staff 480 Aug 21 21:57 .
drwxrwxr-x 46 wk staff 1472 Aug 21 21:52 ..
-rw-rw-r-- 1 wk staff 533 Aug 21 21:39 asset.go
-rw-rw-r-- 1 wk staff 37 Aug 21 21:40 config.go
-rw-rw-r-- 1 wk staff 438 Aug 21 21:41 go.mod
-rw-rw-r-- 1 wk staff 5711 Aug 21 21:40 go.sum
-rw-rw-r-- 1 wk staff 107 Aug 21 21:41 helper.go
-rw-rw-r-- 1 wk staff 116 Aug 21 21:42 main.go
-rw-rw-r-- 1 wk staff 85 Aug 21 21:37 main_test.go
-rw-rw-r-- 1 wk staff 2763 Aug 21 21:35 makefile
-rwxrwxr-x 1 wk staff 6237020 Aug 21 21:57 segmed
-rwxrwxr-x 1 wk staff 5578752 Aug 21 21:57 segmed.linux
-rwxrwxr-x 1 wk staff 7980412 Aug 21 21:57 segmed.test
-rw-rw-r-- 1 wk staff 1139 Aug 21 21:57 tags
-rw-rw-r-- 1 wk staff 70 Aug 21 21:57 unused_to_test.go
I attempt to emulate impact of unused code by making unused_to_test.go with 1 global function not called from anywhere. No luck in reproducing -- this project builds fine with Golang 1.15.
Summary: asset.go itself and probably pkger aren't the main culprit. It appears like a combination of other factors exposes this linker issue.