Skip to content

cmd/go: miscompilation of codependent global var assigments in go1.12 #30977

New issue
New issue
Closed
Closed
cmd/go: miscompilation of codependent global var assigments in go1.12#30977
Labels
FrozenDueToAgeNeedsInvestigationSomeone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone
Go1.13
@ijc

Description

@ijc
ijc
opened on Mar 21, 2019

What version of Go are you using (go version)?

$ PATH=$HOME/tmp/go/bin:$PATH go version
go version go1.12.1 linux/amd64

Does this issue reproduce with the latest release?

It reproduces with go 1.12.1 so I believe yes. I downloaded it from https://dl.google.com/go/go1.12.1.linux-amd64.tar.gz

$ sha256sum ~/tmp/go1.12.1.linux-amd64.tar.gz
2a3fdabf665496a0db5f41ec6af7a9b15a49fbe71a85a50ca38b1f13a103aeec  /home/ijc/tmp/go1.12.1.linux-amd64.tar.gz

I've also reproduced on darwin/amd64 from https://golang.org/doc/install?download=go1.12.1.darwin-amd64.pkg:

$ sha256sum ~/Downloads/go1.12.1.darwin-amd64.pkg 
8da4d8a7c5c4fb5d144b5c18e48ca5f0a35677dc71517822ab41bdc7766d3175  /home/ijc/Downloads/go1.12.1.darwin-amd64.pkg

What operating system and processor architecture are you using (go env)?

linux/amd64 (Debian).

go env Output on linux/amd64
$ PATH=$HOME/tmp/go/bin:$PATH go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/ijc/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/ijc/go"
GOPROXY=""
GORACE=""
GOROOT="/home/ijc/tmp/go"
GOTMPDIR=""
GOTOOLDIR="/home/ijc/tmp/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/ijc/development/Docker/go1.12-initvar-miscompile/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build327430398=/tmp/go-build -gno-record-gcc-switches"

I have also reproduced on darwin/amd64:

go env Output on darwin/amd64
$ /usr/local/go/bin/go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/ijc/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/ijc/go"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/ijc/go1.12-initvar-miscompile/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/4k/lv46d_kx3sn71k23yltwyhf40000gn/T/go-build732469803=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

I have a program which includes:

import "github.com/blang/semver"

var (
	Version0_2_0  = semver.MustParse("0.2.0")
	VersionLatest = Version0_2_0
)

The types are semver.Version which is a struct with a handful of fields (Major, Minor etc).

I am seeing corruption of VersionLatest where the major field is 824638865200 (always that number, AFAICS) after init and not 0. Version0_2_0 is never corrupted.

I have narrowed it down to this minimal case: go1.12-initvar-miscompile.zip

$ PATH=$HOME/tmp/go/bin:$PATH go build ./
$ ./miscompile  # this is a pass
"0.2.0"
$ ./miscompile # this is a fail
panic: VersionLatest corrupt: "0.2.0" "824638766896.2.0"

goroutine 1 [running]:
main.main()
	/home/ijc/development/Docker/go1.12-initvar-miscompile/main.go:186 +0x1b5

Note that the test program includes a massive bunch of anonymous imports, these seem to be necessary to trigger the runtime/gc load and cause the issue (they are derived from all the vendoring in my actual project which have init() functions). Trying to minimise the set of imports is non-linear (since the issue is non-deterministic AFAICT) and also reduces the probability of the issue occurring. I got it down to a dozen or so imports and a 1/10,000 incidence, since it is much easier to reproduce with the larger import set that is what I included here.

What did you expect to see?

On success VersionLatest is "0.2.0".

What did you see instead?

It is corrupted to "824638766896.2.0".

Analysis

After the call to semver.MustParse there is a runtime.writeBarrier slow path:

  main.go:177		0x1799233		e8182f30ff		CALL github.com/blang/semver.MustParse(SB)	
  main.go:177		0x1799238		833d31cc050100		CMPL $0x0, runtime.writeBarrier(SB)		
  main.go:177		0x179923f		0f8588000000		JNE 0x17992cd

The issue occurs precisely when this jump is taken and is during this sequence:

// Result of semver.MustParse is on the stack at 0x10(SP). First copy it
// to `Version0_2_0`, effectively `memcpy(Version0_2_0, «stack temporary»)`

  main.go:177		0x17992cd		488b442410		MOVQ 0x10(SP), AX				
  main.go:177		0x17992d2		48898424a0000000	MOVQ AX, 0xa0(SP)				
  main.go:177		0x17992da		0f10442418		MOVUPS 0x18(SP), X0				
  main.go:177		0x17992df		0f118424a8000000	MOVUPS X0, 0xa8(SP)				
  main.go:177		0x17992e7		0f10442428		MOVUPS 0x28(SP), X0				
  main.go:177		0x17992ec		0f118424b8000000	MOVUPS X0, 0xb8(SP)				
  main.go:177		0x17992f4		0f10442438		MOVUPS 0x38(SP), X0				
  main.go:177		0x17992f9		0f118424c8000000	MOVUPS X0, 0xc8(SP)				
  main.go:177		0x1799301		0f10442448		MOVUPS 0x48(SP), X0				
  main.go:177		0x1799306		0f118424d8000000	MOVUPS X0, 0xd8(SP)				

// This sequence overwrites 0x10(SP) with some value before calling
// `runtime.typedmemmove`

  main.go:177		0x179930e		488d058bd22800		LEAQ type.JyaB60kV(SB), AX			
  main.go:177		0x1799315		48890424		MOVQ AX, 0(SP)					
  main.go:177		0x1799319		488d0d00f00301		LEAQ main.Version0_2_0(SB), CX			
  main.go:177		0x1799320		48894c2408		MOVQ CX, 0x8(SP)				
  main.go:177		0x1799325		488d8c24a0000000	LEAQ 0xa0(SP), CX				
  main.go:177		0x179932d		48894c2410		MOVQ CX, 0x10(SP)				
  main.go:177		0x1799332		e8b9e520ff		CALL runtime.typedmemmove(SB)			

// Now we try to copy the result of `semver.MustParse` from the stack into
// `VersionLatest`, effectively `memcpy(VersionLatest, «stack temporary»)`,
// but the values at 0x10(SP) has been changed by the `CALL` above,
// thus `VersionLatest` is corrupted.

  main.go:178		0x1799337		488b442410		MOVQ 0x10(SP), AX				
  main.go:178		0x179933c		4889442458		MOVQ AX, 0x58(SP)				
  main.go:178		0x1799341		0f10442418		MOVUPS 0x18(SP), X0				
  main.go:178		0x1799346		0f11442460		MOVUPS X0, 0x60(SP)				
  main.go:178		0x179934b		0f10442428		MOVUPS 0x28(SP), X0				
  main.go:178		0x1799350		0f11442470		MOVUPS X0, 0x70(SP)				
  main.go:178		0x1799355		0f10442438		MOVUPS 0x38(SP), X0				
  main.go:178		0x179935a		0f11842480000000	MOVUPS X0, 0x80(SP)				
  main.go:178		0x1799362		0f10442448		MOVUPS 0x48(SP), X0				
  main.go:178		0x1799367		0f11842490000000	MOVUPS X0, 0x90(SP)

On go1.11.4 the same code is compiled differently and the second copy is memcpy(VersionLatest, Version0_2_0) instead which does not use the corrupted stack copy

  main.go:178		0x170888a		488b056fbcef00		MOVQ main.Version0_2_0(SB), AX								
  main.go:178		0x1708891		488905c8bcef00		MOVQ AX, main.VersionLatest(SB)								
  main.go:178		0x1708898		0f100569bcef00		MOVUPS main.Version0_2_0+8(SB), X0							
  main.go:178		0x170889f		0f1105c2bcef00		MOVUPS X0, main.VersionLatest+8(SB)							
  main.go:178		0x17088a6		0f10056bbcef00		MOVUPS main.Version0_2_0+24(SB), X0							
  main.go:178		0x17088ad		0f1105c4bcef00		MOVUPS X0, main.VersionLatest+24(SB)							
  main.go:178		0x17088b4		0f10056dbcef00		MOVUPS main.Version0_2_0+40(SB), X0							
  main.go:178		0x17088bb		0f1105c6bcef00		MOVUPS X0, main.VersionLatest+40(SB)							
  main.go:178		0x17088c2		0f10056fbcef00		MOVUPS main.Version0_2_0+56(SB), X0							
  main.go:178		0x17088c9		0f1105c8bcef00		MOVUPS X0, main.VersionLatest+56(SB)

BTW, the code pattern is the same in the fast pass, but that lacks the call to runtime.typedmemmove which overwrites the stack value, thus the issue only occurs if we manager to hit the slow path.

Full go tool objdump -s '^main.init.ializers' Output for go 1.12.1 compilation
```
$ go tool objdump -s '^main.init.ializers$' miscompile 
TEXT main.init.ializers(SB) «...»/go1.12-initvar-miscompile/main.go
  main.go:177		0x17991f0		64488b0c25f8ffffff	MOVQ FS:0xfffffff8, CX				
  main.go:177		0x17991f9		488d442490		LEAQ -0x70(SP), AX				
  main.go:177		0x17991fe		483b4110		CMPQ 0x10(CX), AX				
  main.go:177		0x1799202		0f8692010000		JBE 0x179939a					
  main.go:177		0x1799208		4881ecf0000000		SUBQ $0xf0, SP					
  main.go:177		0x179920f		4889ac24e8000000	MOVQ BP, 0xe8(SP)				
  main.go:177		0x1799217		488dac24e8000000	LEAQ 0xe8(SP), BP				
  main.go:177		0x179921f		488d05a63a2d00		LEAQ 0x2d3aa6(IP), AX				
  main.go:177		0x1799226		48890424		MOVQ AX, 0(SP)					
  main.go:177		0x179922a		48c744240805000000	MOVQ $0x5, 0x8(SP)				
  main.go:177		0x1799233		e8182f30ff		CALL github.com/blang/semver.MustParse(SB)	
  main.go:177		0x1799238		833d31cc050100		CMPL $0x0, runtime.writeBarrier(SB)		
  main.go:177		0x179923f		0f8588000000		JNE 0x17992cd					
  main.go:177		0x1799245		488b442410		MOVQ 0x10(SP), AX				
  main.go:177		0x179924a		488905cff00301		MOVQ AX, main.Version0_2_0(SB)			
  main.go:177		0x1799251		0f10442418		MOVUPS 0x18(SP), X0				
  main.go:177		0x1799256		0f1105cbf00301		MOVUPS X0, main.Version0_2_0+8(SB)		
  main.go:177		0x179925d		0f10442428		MOVUPS 0x28(SP), X0				
  main.go:177		0x1799262		0f1105cff00301		MOVUPS X0, main.Version0_2_0+24(SB)		
  main.go:177		0x1799269		0f10442438		MOVUPS 0x38(SP), X0				
  main.go:177		0x179926e		0f1105d3f00301		MOVUPS X0, main.Version0_2_0+40(SB)		
  main.go:177		0x1799275		0f10442448		MOVUPS 0x48(SP), X0				
  main.go:177		0x179927a		0f1105d7f00301		MOVUPS X0, main.Version0_2_0+56(SB)		
  main.go:178		0x1799281		488b442410		MOVQ 0x10(SP), AX				
  main.go:178		0x1799286		488905f3f00301		MOVQ AX, main.VersionLatest(SB)			
  main.go:178		0x179928d		0f10442418		MOVUPS 0x18(SP), X0				
  main.go:178		0x1799292		0f1105eff00301		MOVUPS X0, main.VersionLatest+8(SB)		
  main.go:178		0x1799299		0f10442428		MOVUPS 0x28(SP), X0				
  main.go:178		0x179929e		0f1105f3f00301		MOVUPS X0, main.VersionLatest+24(SB)		
  main.go:178		0x17992a5		0f10442438		MOVUPS 0x38(SP), X0				
  main.go:178		0x17992aa		0f1105f7f00301		MOVUPS X0, main.VersionLatest+40(SB)		
  main.go:178		0x17992b1		0f10442448		MOVUPS 0x48(SP), X0				
  main.go:178		0x17992b6		0f1105fbf00301		MOVUPS X0, main.VersionLatest+56(SB)		
  main.go:178		0x17992bd		488bac24e8000000	MOVQ 0xe8(SP), BP				
  main.go:178		0x17992c5		4881c4f0000000		ADDQ $0xf0, SP					
  main.go:178		0x17992cc		c3			RET						
  main.go:177		0x17992cd		488b442410		MOVQ 0x10(SP), AX				
  main.go:177		0x17992d2		48898424a0000000	MOVQ AX, 0xa0(SP)				
  main.go:177		0x17992da		0f10442418		MOVUPS 0x18(SP), X0				
  main.go:177		0x17992df		0f118424a8000000	MOVUPS X0, 0xa8(SP)				
  main.go:177		0x17992e7		0f10442428		MOVUPS 0x28(SP), X0				
  main.go:177		0x17992ec		0f118424b8000000	MOVUPS X0, 0xb8(SP)				
  main.go:177		0x17992f4		0f10442438		MOVUPS 0x38(SP), X0				
  main.go:177		0x17992f9		0f118424c8000000	MOVUPS X0, 0xc8(SP)				
  main.go:177		0x1799301		0f10442448		MOVUPS 0x48(SP), X0				
  main.go:177		0x1799306		0f118424d8000000	MOVUPS X0, 0xd8(SP)				
  main.go:177		0x179930e		488d058bd22800		LEAQ type.JyaB60kV(SB), AX			
  main.go:177		0x1799315		48890424		MOVQ AX, 0(SP)					
  main.go:177		0x1799319		488d0d00f00301		LEAQ main.Version0_2_0(SB), CX			
  main.go:177		0x1799320		48894c2408		MOVQ CX, 0x8(SP)				
  main.go:177		0x1799325		488d8c24a0000000	LEAQ 0xa0(SP), CX				
  main.go:177		0x179932d		48894c2410		MOVQ CX, 0x10(SP)				
  main.go:177		0x1799332		e8b9e520ff		CALL runtime.typedmemmove(SB)			
  main.go:178		0x1799337		488b442410		MOVQ 0x10(SP), AX				
  main.go:178		0x179933c		4889442458		MOVQ AX, 0x58(SP)				
  main.go:178		0x1799341		0f10442418		MOVUPS 0x18(SP), X0				
  main.go:178		0x1799346		0f11442460		MOVUPS X0, 0x60(SP)				
  main.go:178		0x179934b		0f10442428		MOVUPS 0x28(SP), X0				
  main.go:178		0x1799350		0f11442470		MOVUPS X0, 0x70(SP)				
  main.go:178		0x1799355		0f10442438		MOVUPS 0x38(SP), X0				
  main.go:178		0x179935a		0f11842480000000	MOVUPS X0, 0x80(SP)				
  main.go:178		0x1799362		0f10442448		MOVUPS 0x48(SP), X0				
  main.go:178		0x1799367		0f11842490000000	MOVUPS X0, 0x90(SP)				
  main.go:178		0x179936f		488d052ad22800		LEAQ type.JyaB60kV(SB), AX			
  main.go:178		0x1799376		48890424		MOVQ AX, 0(SP)					
  main.go:178		0x179937a		488d05ffef0301		LEAQ main.VersionLatest(SB), AX			
  main.go:178		0x1799381		4889442408		MOVQ AX, 0x8(SP)				
  main.go:178		0x1799386		488d442458		LEAQ 0x58(SP), AX				
  main.go:178		0x179938b		4889442410		MOVQ AX, 0x10(SP)				
  main.go:178		0x1799390		e85be520ff		CALL runtime.typedmemmove(SB)			
  main.go:177		0x1799395		e923ffffff		JMP 0x17992bd					
  main.go:177		0x179939a		e8611a25ff		CALL local.runtime.morestack_noctxt(SB)		
  main.go:177		0x179939f		e94cfeffff		JMP local.main.init.ializers(SB)	
```

Metadata

Metadata

Assignees

No one assigned

    Labels

    FrozenDueToAgeNeedsInvestigationSomeone must examine and confirm this is a valid issue and not a duplicate of an existing one.

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions