[BUG] make test panic with the error `fatal error: concurrent map read and map write` #2713

iziang · 2023-04-19T03:50:08Z

make test panic with the following error

/Users/ziang/git/kubeblocks/bin/controller-gen object:headerFile="hack/boilerplate.go.txt" paths="./apis/...;./externalapis/..."
gofmt -l -w -s $(git ls-files --exclude-standard | grep "\.go$")
GOOS=linux go vet -mod=mod ./...
/Users/ziang/go/bin/goimports -local github.com/apecloud/kubeblocks -w $(git ls-files|grep "\.go$")
golangci-lint is already installed
/Users/ziang/go/bin/staticcheck ./...
/Users/ziang/go/bin/golangci-lint run ./...
go generate -x ./internal/testutil/k8s/mocks/...
go run github.com/golang/mock/mockgen -copyright_file ../../../../hack/boilerplate.go.txt -package mocks -destination k8sclient_mocks.go sigs.k8s.io/controller-runtime/pkg/client Client
go generate -x ./internal/configuration/container/mocks/...
go run github.com/golang/mock/mockgen -copyright_file ../../../../hack/boilerplate.go.txt -package mocks -destination dockerclient_mocks.go github.com/docker/docker/client ContainerAPIClient
go run github.com/golang/mock/mockgen -copyright_file ../../../../hack/boilerplate.go.txt -package mocks -destination criclient_mocks.go k8s.io/cri-api/pkg/apis/runtime/v1 RuntimeServiceClient
go generate -x ./internal/configuration/proto/mocks/...
go run github.com/golang/mock/mockgen -copyright_file ../../../../hack/boilerplate.go.txt -package mocks -destination reconfigure_client_mocks.go github.com/apecloud/kubeblocks/internal/configuration/proto ReconfigureClient
/Users/ziang/git/kubeblocks/bin/controller-gen rbac:roleName=manager-role crd:generateEmbeddedObjectMeta=true webhook paths="./cmd/manager/...;./apis/...;./controllers/...;./internal/..." output:crd:artifacts:config=config/crd/bases
KUBEBUILDER_ASSETS="/Users/ziang/Library/Application Support/io.kubebuilder.envtest/k8s/1.25.0-darwin-arm64" go test -short -coverprofile cover.out ./internal/... ./apis/... ./controllers/... ./cmd/...
ok  	github.com/apecloud/kubeblocks/internal/class	0.539s	coverage: 51.9% of statements
ok  	github.com/apecloud/kubeblocks/internal/cli/cloudprovider	1.055s	coverage: 40.9% of statements
ok  	github.com/apecloud/kubeblocks/internal/cli/cluster	1.271s	coverage: 62.8% of statements
?   	github.com/apecloud/kubeblocks/internal/cli/cmd	[no test files]
ok  	github.com/apecloud/kubeblocks/internal/cli/cmd/accounts	2.695s	coverage: 51.9% of statements
ok  	github.com/apecloud/kubeblocks/internal/cli/cmd/addon	1.954s	coverage: 9.8% of statements
ok  	github.com/apecloud/kubeblocks/internal/cli/cmd/alert	2.244s	coverage: 74.2% of statements
ok  	github.com/apecloud/kubeblocks/internal/cli/cmd/bench	0.233s	coverage: 70.4% of statements
ok  	github.com/apecloud/kubeblocks/internal/cli/cmd/class	0.604s	coverage: 81.0% of statements
ok  	github.com/apecloud/kubeblocks/internal/cli/cmd/cluster	3.522s	coverage: 57.2% of statements
ok  	github.com/apecloud/kubeblocks/internal/cli/cmd/clusterdefinition	1.625s	coverage: 70.0% of statements
ok  	github.com/apecloud/kubeblocks/internal/cli/cmd/clusterversion	2.828s	coverage: 70.0% of statements
ok  	github.com/apecloud/kubeblocks/internal/cli/cmd/dashboard	2.762s	coverage: 66.0% of statements
ok  	github.com/apecloud/kubeblocks/internal/cli/cmd/kubeblocks	1.966s	coverage: 52.5% of statements
ok  	github.com/apecloud/kubeblocks/internal/cli/cmd/migration	0.636s	coverage: 34.8% of statements
ok  	github.com/apecloud/kubeblocks/internal/cli/cmd/options	2.218s	coverage: 100.0% of statements
ok  	github.com/apecloud/kubeblocks/internal/cli/cmd/playground	49.668s	coverage: 27.2% of statements
ok  	github.com/apecloud/kubeblocks/internal/cli/cmd/version	0.733s	coverage: 52.4% of statements
ok  	github.com/apecloud/kubeblocks/internal/cli/create	0.701s	coverage: 68.4% of statements
ok  	github.com/apecloud/kubeblocks/internal/cli/delete	1.192s	coverage: 85.9% of statements
ok  	github.com/apecloud/kubeblocks/internal/cli/edit	0.766s	coverage: 100.0% of statements
ok  	github.com/apecloud/kubeblocks/internal/cli/exec	0.975s	coverage: 71.4% of statements
ok  	github.com/apecloud/kubeblocks/internal/cli/list	1.043s	coverage: 75.6% of statements
ok  	github.com/apecloud/kubeblocks/internal/cli/patch	0.628s	coverage: 28.0% of statements
ok  	github.com/apecloud/kubeblocks/internal/cli/printer	1.559s	coverage: 49.3% of statements
ok  	github.com/apecloud/kubeblocks/internal/cli/testing	1.791s	coverage: 62.3% of statements
?   	github.com/apecloud/kubeblocks/internal/cli/types	[no test files]
?   	github.com/apecloud/kubeblocks/internal/cli/types/migrationapi	[no test files]
ok  	github.com/apecloud/kubeblocks/internal/cli/util	25.893s	coverage: 51.6% of statements
ok  	github.com/apecloud/kubeblocks/internal/cli/util/helm	0.684s	coverage: 58.6% of statements
ok  	github.com/apecloud/kubeblocks/internal/cli/util/prompt	0.866s	coverage: 100.0% of statements
ok  	github.com/apecloud/kubeblocks/internal/configuration	2.134s	coverage: 82.4% of statements
ok  	github.com/apecloud/kubeblocks/internal/configuration/config_manager	5.489s	coverage: 67.3% of statements
ok  	github.com/apecloud/kubeblocks/internal/configuration/container	1.682s	coverage: 72.1% of statements
?   	github.com/apecloud/kubeblocks/internal/configuration/container/mocks	[no test files]
?   	github.com/apecloud/kubeblocks/internal/configuration/proto	[no test files]
?   	github.com/apecloud/kubeblocks/internal/configuration/proto/mocks	[no test files]
ok  	github.com/apecloud/kubeblocks/internal/configuration/util	1.104s	coverage: 57.4% of statements
?   	github.com/apecloud/kubeblocks/internal/constant	[no test files]
ok  	github.com/apecloud/kubeblocks/internal/controller/builder	7.472s	coverage: 76.9% of statements
?   	github.com/apecloud/kubeblocks/internal/controller/client	[no test files]
ok  	github.com/apecloud/kubeblocks/internal/controller/component	11.034s	coverage: 80.5% of statements
ok  	github.com/apecloud/kubeblocks/internal/controller/graph	1.054s	coverage: 68.0% of statements
ok  	github.com/apecloud/kubeblocks/internal/controller/lifecycle	0.426s	coverage: 1.4% of statements
ok  	github.com/apecloud/kubeblocks/internal/controller/plan	10.980s	coverage: 63.7% of statements
?   	github.com/apecloud/kubeblocks/internal/controller/types	[no test files]
ok  	github.com/apecloud/kubeblocks/internal/controllerutil	6.849s	coverage: 59.2% of statements
?   	github.com/apecloud/kubeblocks/internal/generics	[no test files]
ok  	github.com/apecloud/kubeblocks/internal/gotemplate	0.478s	coverage: 77.5% of statements
ok  	github.com/apecloud/kubeblocks/internal/preflight	32.267s	coverage: 78.4% of statements
ok  	github.com/apecloud/kubeblocks/internal/preflight/analyzer	0.919s	coverage: 87.2% of statements
ok  	github.com/apecloud/kubeblocks/internal/preflight/collector	0.613s	coverage: 79.1% of statements
?   	github.com/apecloud/kubeblocks/internal/preflight/interactive	[no test files]
ok  	github.com/apecloud/kubeblocks/internal/preflight/testing	0.337s	coverage: 92.3% of statements
ok  	github.com/apecloud/kubeblocks/internal/preflight/util	0.471s	coverage: 100.0% of statements
ok  	github.com/apecloud/kubeblocks/internal/sqlchannel	1.036s	coverage: 65.9% of statements
ok  	github.com/apecloud/kubeblocks/internal/sqlchannel/engine	0.187s	coverage: 60.0% of statements
?   	github.com/apecloud/kubeblocks/internal/testutil	[no test files]
?   	github.com/apecloud/kubeblocks/internal/testutil/apps	[no test files]
?   	github.com/apecloud/kubeblocks/internal/testutil/k8s	[no test files]
?   	github.com/apecloud/kubeblocks/internal/testutil/k8s/mocks	[no test files]
ok  	github.com/apecloud/kubeblocks/internal/unstructured	0.463s	coverage: 79.2% of statements
ok  	github.com/apecloud/kubeblocks/internal/unstructured/redis	0.106s	coverage: 45.9% of statements
?   	github.com/apecloud/kubeblocks/internal/webhook	[no test files]
ok  	github.com/apecloud/kubeblocks/apis/apps/v1alpha1	16.066s	coverage: 28.7% of statements
ok  	github.com/apecloud/kubeblocks/apis/dataprotection/v1alpha1	0.198s	coverage: 2.5% of statements
ok  	github.com/apecloud/kubeblocks/apis/extensions/v1alpha1	0.757s	coverage: 23.4% of statements
ok  	github.com/apecloud/kubeblocks/controllers/apps	46.536s	coverage: 78.9% of statements
ok  	github.com/apecloud/kubeblocks/controllers/apps/components	20.157s	coverage: 67.6% of statements
ok  	github.com/apecloud/kubeblocks/controllers/apps/components/consensus	12.912s	coverage: 27.4% of statements
ok  	github.com/apecloud/kubeblocks/controllers/apps/components/replication	16.584s	coverage: 69.5% of statements
ok  	github.com/apecloud/kubeblocks/controllers/apps/components/stateful	15.143s	coverage: 75.0% of statements
ok  	github.com/apecloud/kubeblocks/controllers/apps/components/stateless	11.872s	coverage: 78.3% of statements
?   	github.com/apecloud/kubeblocks/controllers/apps/components/types	[no test files]
ok  	github.com/apecloud/kubeblocks/controllers/apps/components/util	14.957s	coverage: 76.4% of statements
ok  	github.com/apecloud/kubeblocks/controllers/apps/configuration	16.160s	coverage: 73.7% of statements
ok  	github.com/apecloud/kubeblocks/controllers/apps/operations	14.991s	coverage: 82.4% of statements
ok  	github.com/apecloud/kubeblocks/controllers/apps/operations/util	12.671s	coverage: 84.4% of statements
Running Suite: Data Protection Controller Suite - /Users/ziang/git/kubeblocks/controllers/dataprotection
========================================================================================================
Random Seed: 1681874075

Will run 19 of 19 specs
config settings: map[clearresourcepollinginterval:1ms clearresourcetimeout:10s cm_namespace: consistentlyduration:3s consistentlypollinginterval:1ms eventuallypollinginterval:1ms eventuallytimeout:10s kill_container_signal:SIGKILL kubeblocks_tools_image:apecloud/kubeblocks:latest maxconcurrentreconciles_dataprotection:20]
•••••••••••fatal error: concurrent map read and map write

goroutine 927 [running]:
github.com/spf13/viper.(*Viper).searchMap(0x140000fd6c0?, 0x105b52d40?, {0x14000901d20, 0x1, 0x1})
	/Users/ziang/go/pkg/mod/github.com/spf13/[email protected]/viper.go:659 +0x48
github.com/spf13/viper.(*Viper).find(0x140000fd6c0, {0x14001811650, 0x18}, 0x1)
	/Users/ziang/go/pkg/mod/github.com/spf13/[email protected]/viper.go:1329 +0x810
github.com/spf13/viper.(*Viper).Get(0x140000fd6c0, {0x1056704b8?, 0x105678801?})
	/Users/ziang/go/pkg/mod/github.com/spf13/[email protected]/viper.go:892 +0x48
github.com/spf13/viper.(*Viper).GetString(0x1059cd32a?, {0x1056704b8?, 0x10567882c?})
	/Users/ziang/go/pkg/mod/github.com/spf13/[email protected]/viper.go:959 +0x20
github.com/spf13/viper.GetString(...)
	/Users/ziang/go/pkg/mod/github.com/spf13/[email protected]/viper.go:956

The text was updated successfully, but these errors were encountered:

iziang · 2023-04-19T03:57:26Z

Viper is not concurrent safe, the related discussion can be found here: spf13/viper#268.

iziang · 2023-04-19T04:00:19Z

https://github.com/spf13/viper#is-it-safe-to-concurrently-read-and-write-to-a-viper

iziang · 2023-04-20T04:14:26Z

we can enable the go data race detector to find race condition bugs like this. I have opened another improvement issue #2760.

iziang · 2023-08-06T04:48:22Z

can't reproduce it now, just close this issue, we can reopen it in the future if necessary.

iziang added the kind/bug Something isn't working label Apr 19, 2023

iziang assigned nayutah Apr 19, 2023

apecloud-bot added bug test labels Apr 19, 2023

iziang changed the title ~~[BUG] test paniced with fatal error: concurrent map read and map write~~ [BUG] make test panic with the error fatal error: concurrent map read and map write Apr 19, 2023

nashtsai added this to the Release 0.5.0 milestone Apr 19, 2023

ldming assigned huyongqii and unassigned nayutah Apr 19, 2023

iziang mentioned this issue Apr 20, 2023

[Improvement] enable golang race detector to find potential race condition bugs #2760

Closed

iziang self-assigned this Apr 20, 2023

ahjing99 modified the milestones: Release 0.5.0, Release 0.6.0 May 19, 2023

iziang closed this as completed Aug 6, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[BUG] make test panic with the error `fatal error: concurrent map read and map write` #2713

[BUG] make test panic with the error `fatal error: concurrent map read and map write` #2713

iziang commented Apr 19, 2023 •

edited

Loading

iziang commented Apr 19, 2023

iziang commented Apr 19, 2023

iziang commented Apr 20, 2023

iziang commented Aug 6, 2023

[BUG] make test panic with the error fatal error: concurrent map read and map write #2713

[BUG] make test panic with the error fatal error: concurrent map read and map write #2713

Comments

iziang commented Apr 19, 2023 • edited Loading

iziang commented Apr 19, 2023

iziang commented Apr 19, 2023

iziang commented Apr 20, 2023

iziang commented Aug 6, 2023

[BUG] make test panic with the error `fatal error: concurrent map read and map write` #2713

[BUG] make test panic with the error `fatal error: concurrent map read and map write` #2713

iziang commented Apr 19, 2023 •

edited

Loading