fatal error: concurrent map read and map write from Config.Labels

[gengliqi]

Bug Report

fatal error: concurrent map read and map write

 goroutine 513 [running]:
 github.com/pingcap/tidb/config.(*Config).GetTiKVConfig(...)
 	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/config/config.go:312
 github.com/pingcap/tidb/sessionctx/variable.glob..func188({0xc04ef57a40?, 0xc001c01800?}, 0xc04e8fdcae?, {0xc04e8fdcae?, 0x2?})
 	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/sessionctx/variable/sysvar.go:1026 +0x33e
 github.com/pingcap/tidb/domain.(*Domain).rebuildSysVarCache(0xc000cefe00, {0x5025338, 0xc0008e2c80})
 	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/domain/sysvar_cache.go:147 +0x71e
 github.com/pingcap/tidb/domain.(*Domain).LoadSysVarCacheLoop.func1()
 	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/domain/domain.go:1415 +0x1ff
 created by github.com/pingcap/tidb/domain.(*Domain).LoadSysVarCacheLoop
 	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/domain/domain.go:1376 +0x172

tidb/server/http_handler.go

Lines 2221 to 2231 in b4183e1

	if len(labels) > 0 {
	cfg := *config.GetGlobalConfig()
	if cfg.Labels == nil {
	cfg.Labels = make(map[string]string, len(labels))
	}
	for k, v := range labels {
	cfg.Labels[k] = v
	}
	config.StoreGlobalConfig(&cfg)
	logutil.BgLogger().Info("update server labels", zap.Any("labels", cfg.Labels))
	}

The code above wants to update Labels in Config. It copies the global config first and then changes the Labels in the copy.
However, the Labels is a map[string]string so the copy is shallow, which causes this data race.

[ti-chi-bot]

@gengliqi: The label(s) sig/sql-infra, severity/major cannot be applied. These labels are supported: fuzz/sqlancer, challenge-program, compatibility-breaker, first-time-contributor, contribution, require-LGT3, good first issue, correctness, duplicate, proposal, security, needs-more-info, needs-cherry-pick-release-5.2, needs-cherry-pick-release-5.3, needs-cherry-pick-release-5.4, needs-cherry-pick-release-6.0, needs-cherry-pick-release-6.1, needs-cherry-pick-release-6.2, needs-cherry-pick-release-6.3, needs-cherry-pick-release-6.4, needs-cherry-pick-release-6.5, needs-cherry-pick-release-6.6, needs-cherry-pick-release-7.0, needs-cherry-pick-release-7.1, needs-cherry-pick-release-7.2, affects-5.2, affects-5.3, affects-5.4, affects-6.0, affects-6.1, affects-6.2, affects-6.3, affects-6.4, affects-6.5, affects-6.6, affects-7.0, affects-7.1, affects-7.2, may-affects-5.2, may-affects-5.3, may-affects-5.4, may-affects-6.0, may-affects-6.1, may-affects-6.2, may-affects-6.3, may-affects-6.4, may-affects-6.5, may-affects-6.6, may-affects-7.0, may-affects-7.1, may-affects-7.2.

In response to this:

/label sig/sql-infra
/label severity/major

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

[gengliqi]

/severity major

[ti-chi-bot]

@gengliqi: The label(s) sig/sql-infra, severity/major cannot be applied. These labels are supported: fuzz/sqlancer, challenge-program, compatibility-breaker, first-time-contributor, contribution, require-LGT3, good first issue, correctness, duplicate, proposal, security, needs-more-info, needs-cherry-pick-release-5.2, needs-cherry-pick-release-5.3, needs-cherry-pick-release-5.4, needs-cherry-pick-release-6.0, needs-cherry-pick-release-6.1, needs-cherry-pick-release-6.2, needs-cherry-pick-release-6.3, needs-cherry-pick-release-6.4, needs-cherry-pick-release-6.5, needs-cherry-pick-release-6.6, needs-cherry-pick-release-7.0, needs-cherry-pick-release-7.1, needs-cherry-pick-release-7.2, affects-5.2, affects-5.3, affects-5.4, affects-6.0, affects-6.1, affects-6.2, affects-6.3, affects-6.4, affects-6.5, affects-6.6, affects-7.0, affects-7.1, affects-7.2, may-affects-5.2, may-affects-5.3, may-affects-5.4, may-affects-6.0, may-affects-6.1, may-affects-6.2, may-affects-6.3, may-affects-6.4, may-affects-6.5, may-affects-6.6, may-affects-7.0, may-affects-7.1, may-affects-7.2.

In response to this:

/label sig/sql-infra
/label severity/major

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

[jebter]

/component pd

[seiya-annie]

/found customer