diff --git a/doc/SeparatingVttabletMysql.md b/doc/SeparatingVttabletMysql.md index f5c3ddca8d1..0ddaeecdfa8 100644 --- a/doc/SeparatingVttabletMysql.md +++ b/doc/SeparatingVttabletMysql.md @@ -20,9 +20,9 @@ The following adjustments need to be made to VTTablet command line parameters: * Do not use `-mycnf_socket_file`. There is no local MySQL unix socket file. -* Specify the host and port of the MySQL daemon for the `-db-config-XXX-host` - and `-db-config-XXX-port` command line parameters. Do not specify - `-db-config-XXX-unixsocket` parameters. +* Specify the host and port of the MySQL daemon for the `-db\_host` + and `-db\_port` command line parameters. Do not specify + `-db\_socket` parameters. * Disable restores / backups, by not passing any backup command line parameters. Specifically, `-restore_from_backup` and diff --git a/doc/ServerConfiguration.md b/doc/ServerConfiguration.md index 261f66f1c17..0124e470182 100644 --- a/doc/ServerConfiguration.md +++ b/doc/ServerConfiguration.md @@ -256,36 +256,45 @@ the binlogs in `/mnt/bin-logs`: VTTablet requires multiple user credentials to perform its tasks. Since it's required to run on the same machine as MySQL, it’s most beneficial to use the more efficient unix socket connections. +**connection** parameters + +* **db\_socket**: The unix socket to connect on. If this is specifed, host and port will not be used. +* **db\_host**: The host name for the tcp connection. +* **db\_port**: The tcp port to be used with the db\_host. +* **db\_charset**: Character set. Only utf8 or latin1 based character sets are supported. +* **db\_flags**: Flag values as defined by MySQL. +* **db\ssl\_ca, db\_ssl\_ca\_path, db\_ssl\_cert, db\_ssl\_key**: SSL flags. + + **app** credentials are for serving app queries: -* **db-config-app-unixsocket**: MySQL socket name to connect to. -* **db-config-app-uname**: App username. -* **db-config-app-pass**: Password for the app username. If you need a more secure way of managing and supplying passwords, VTTablet does allow you to plug into a "password server" that can securely supply and refresh usernames and passwords. Please contact the Vitess team for help if you’d like to write such a custom plugin. -* **db-config-app-charset**: The only supported character set is utf8. Vitess still works with latin1, but it’s getting deprecated. +* **db\_app\_user**: App username. +* **db\_app\_password**: Password for the app username. If you need a more secure way of managing and supplying passwords, VTTablet does allow you to plug into a "password server" that can securely supply and refresh usernames and passwords. Please contact the Vitess team for help if you’d like to write such a custom plugin. +* **db\_app\_use\_ssl**: Set this flag to false if you don't want to use SSL for this connection. This will allow you to turn off SSL for all users except for `repl`, which may have to be turned on for replication that goes over open networks. + +**appdebug** credentials are for the appdebug user: + +* **db\_appdebug\_user** +* **db\_appdebug\_password** +* **db\_appdebug\_use\_ssl** **dba** credentials will be used for housekeeping work like loading the schema or killing runaway queries: -* **db-config-dba-unixsocket** -* **db-config-dba-uname** -* **db-config-dba-pass** -* **db-config-dba-charset** +* **db\_dba\_user** +* **db\_dba\_password** +* **db\_dba\_use\_ssl** -**repl** credentials are for managing replication. Since repl connections can be used across machines, you can optionally turn on encryption: +**repl** credentials are for managing replication. -* **db-config-repl-uname** -* **db-config-repl-pass** -* **db-config-repl-charset** -* **db-config-repl-flags**: If you want to enable SSL, this must be set to 2048. -* **db-config-repl-ssl-ca** -* **db-config-repl-ssl-cert** -* **db-config-repl-ssl-key** +* **db\_repl\_user** +* **db\_repl\_password** +* **db\_repl\_use\_ssl** **filtered** credentials are for performing resharding: -* **db-config-filtered-unixsocket** -* **db-config-filtered-uname** -* **db-config-filtered-pass** -* **db-config-filtered-charset** +* **db\_filtered\_user** +* **db\_filtered\_password** +* **db\_filtered\_use\_ssl** ### Monitoring diff --git a/examples/compose/vttablet-up.sh b/examples/compose/vttablet-up.sh index 2735d1a25bc..63510ddfecb 100755 --- a/examples/compose/vttablet-up.sh +++ b/examples/compose/vttablet-up.sh @@ -14,22 +14,6 @@ if [ "$uid" = "1" ]; then tablet_role='master' fi -dbconfig_dba_flags="\ - -db-config-dba-uname root \ - -db-config-dba-charset utf8" - -dbconfig_flags="$dbconfig_dba_flags \ - -db-config-app-uname root \ - -db-config-app-charset utf8 \ - -db-config-appdebug-uname root \ - -db-config-appdebug-charset utf8 \ - -db-config-allprivs-uname root \ - -db-config-allprivs-charset utf8 \ - -db-config-repl-uname root \ - -db-config-repl-charset utf8 \ - -db-config-filtered-uname root \ - -db-config-filtered-charset utf8" - init_db_sql_file="$VTROOT/init_db.sql" echo "GRANT ALL ON *.* TO 'root'@'%';" > $init_db_sql_file @@ -52,7 +36,6 @@ fi $VTROOT/bin/mysqlctl \ -log_dir $VTDATAROOT/tmp \ -tablet_uid $uid \ - $dbconfig_dba_flags \ -mysql_port 3306 \ $action & @@ -78,5 +61,4 @@ exec $VTROOT/bin/vttablet \ -grpc_port $GRPC_PORT \ -service_map 'grpc-queryservice,grpc-tabletmanager,grpc-updatestream' \ -pid_file $VTDATAROOT/$tablet_dir/vttablet.pid \ - -vtctld_addr http://vtctld:$WEB_PORT/ \ - $dbconfig_flags + -vtctld_addr "http://vtctld:$WEB_PORT/" diff --git a/examples/kubernetes/vttablet-pod-benchmarking-template.yaml b/examples/kubernetes/vttablet-pod-benchmarking-template.yaml index 3cf0c1c3977..8f24da64f54 100644 --- a/examples/kubernetes/vttablet-pod-benchmarking-template.yaml +++ b/examples/kubernetes/vttablet-pod-benchmarking-template.yaml @@ -56,18 +56,6 @@ spec: -init_shard {{shard}} -init_tablet_type {{tablet_type}} -mysqlctl_socket $VTDATAROOT/mysqlctl.sock - -db-config-app-uname vt_app - -db-config-app-dbname vt_{{keyspace}} - -db-config-app-charset utf8 - -db-config-dba-uname vt_dba - -db-config-dba-dbname vt_{{keyspace}} - -db-config-dba-charset utf8 - -db-config-repl-uname vt_repl - -db-config-repl-dbname vt_{{keyspace}} - -db-config-repl-charset utf8 - -db-config-filtered-uname vt_filtered - -db-config-filtered-dbname vt_{{keyspace}} - -db-config-filtered-charset utf8 -queryserver-config-transaction-cap 300 -queryserver-config-schema-reload-time 1 -queryserver-config-pool-size 100 @@ -98,8 +86,6 @@ spec: -alsologtostderr -tablet_uid {{uid}} -socket_file $VTDATAROOT/mysqlctl.sock - -db-config-dba-uname vt_dba - -db-config-dba-charset utf8 -init_db_sql_file $VTROOT/config/init_db.sql" vitess env: - name: EXTRA_MY_CNF diff --git a/examples/kubernetes/vttablet-pod-template.yaml b/examples/kubernetes/vttablet-pod-template.yaml index ca87d9612d7..52e19aad80f 100644 --- a/examples/kubernetes/vttablet-pod-template.yaml +++ b/examples/kubernetes/vttablet-pod-template.yaml @@ -64,18 +64,6 @@ spec: -init_tablet_type {{tablet_type}} -health_check_interval 5s -mysqlctl_socket $VTDATAROOT/mysqlctl.sock - -db-config-app-uname vt_app - -db-config-app-dbname vt_{{keyspace}} - -db-config-app-charset utf8 - -db-config-dba-uname vt_dba - -db-config-dba-dbname vt_{{keyspace}} - -db-config-dba-charset utf8 - -db-config-repl-uname vt_repl - -db-config-repl-dbname vt_{{keyspace}} - -db-config-repl-charset utf8 - -db-config-filtered-uname vt_filtered - -db-config-filtered-dbname vt_{{keyspace}} - -db-config-filtered-charset utf8 -enable_semi_sync -enable_replication_reporter -orc_api_url http://orchestrator/api @@ -107,8 +95,6 @@ spec: -alsologtostderr -tablet_uid {{uid}} -socket_file $VTDATAROOT/mysqlctl.sock - -db-config-dba-uname vt_dba - -db-config-dba-charset utf8 -init_db_sql_file $VTROOT/config/init_db.sql" vitess env: - name: EXTRA_MY_CNF diff --git a/examples/local/vttablet-down.sh b/examples/local/vttablet-down.sh index 3d8640c2d1a..66191b4f44c 100755 --- a/examples/local/vttablet-down.sh +++ b/examples/local/vttablet-down.sh @@ -41,7 +41,6 @@ for uid_index in $uids; do echo "Stopping MySQL for tablet $alias..." $VTROOT/bin/mysqlctl \ - -db-config-dba-uname vt_dba \ -tablet_uid $uid \ shutdown & done diff --git a/examples/local/vttablet-up.sh b/examples/local/vttablet-up.sh index 9ff3f42282b..0bd71527b16 100755 --- a/examples/local/vttablet-up.sh +++ b/examples/local/vttablet-up.sh @@ -36,25 +36,6 @@ fi script_root=`dirname "${BASH_SOURCE}"` source $script_root/env.sh -dbconfig_dba_flags="\ - -db-config-dba-uname vt_dba \ - -db-config-dba-charset utf8" -dbconfig_flags="$dbconfig_dba_flags \ - -db-config-app-uname vt_app \ - -db-config-app-dbname vt_$keyspace \ - -db-config-app-charset utf8 \ - -db-config-appdebug-uname vt_appdebug \ - -db-config-appdebug-dbname vt_$keyspace \ - -db-config-appdebug-charset utf8 \ - -db-config-allprivs-uname vt_allprivs \ - -db-config-allprivs-dbname vt_$keyspace \ - -db-config-allprivs-charset utf8 \ - -db-config-repl-uname vt_repl \ - -db-config-repl-dbname vt_$keyspace \ - -db-config-repl-charset utf8 \ - -db-config-filtered-uname vt_filtered \ - -db-config-filtered-dbname vt_$keyspace \ - -db-config-filtered-charset utf8" init_db_sql_file="$VTROOT/config/init_db.sql" case "$MYSQL_FLAVOR" in @@ -93,7 +74,6 @@ for uid_index in $uids; do $VTROOT/bin/mysqlctl \ -log_dir $VTDATAROOT/tmp \ -tablet_uid $uid \ - $dbconfig_dba_flags \ -mysql_port $mysql_port \ $action & done @@ -142,7 +122,6 @@ for uid_index in $uids; do -pid_file $VTDATAROOT/$tablet_dir/vttablet.pid \ -vtctld_addr http://$hostname:$vtctld_web_port/ \ $optional_auth_args \ - $dbconfig_flags \ > $VTDATAROOT/$tablet_dir/vttablet.out 2>&1 & echo "Access tablet $alias at http://$hostname:$port/debug/status" diff --git a/go/cmd/mysqlctl/mysqlctl.go b/go/cmd/mysqlctl/mysqlctl.go index 43b7bd048ba..9e87249a9c7 100644 --- a/go/cmd/mysqlctl/mysqlctl.go +++ b/go/cmd/mysqlctl/mysqlctl.go @@ -44,17 +44,11 @@ var ( tabletAddr string ) -const ( - // dbconfigFlags is only set to DbaConfig, as mysqlctl only - // starts and stops mysqld, and that is only done with the dba user. - dbconfigFlags = dbconfigs.DbaConfig -) - func initConfigCmd(subFlags *flag.FlagSet, args []string) error { subFlags.Parse(args) // Generate my.cnf from scratch and use it to find mysqld. - mysqld, err := mysqlctl.CreateMysqld(uint32(*tabletUID), *mysqlSocket, int32(*mysqlPort), dbconfigFlags) + mysqld, err := mysqlctl.CreateMysqld(uint32(*tabletUID), *mysqlSocket, int32(*mysqlPort)) if err != nil { return fmt.Errorf("failed to initialize mysql config: %v", err) } @@ -71,7 +65,7 @@ func initCmd(subFlags *flag.FlagSet, args []string) error { subFlags.Parse(args) // Generate my.cnf from scratch and use it to find mysqld. - mysqld, err := mysqlctl.CreateMysqld(uint32(*tabletUID), *mysqlSocket, int32(*mysqlPort), dbconfigFlags) + mysqld, err := mysqlctl.CreateMysqld(uint32(*tabletUID), *mysqlSocket, int32(*mysqlPort)) if err != nil { return fmt.Errorf("failed to initialize mysql config: %v", err) } @@ -87,7 +81,7 @@ func initCmd(subFlags *flag.FlagSet, args []string) error { func reinitConfigCmd(subFlags *flag.FlagSet, args []string) error { // There ought to be an existing my.cnf, so use it to find mysqld. - mysqld, err := mysqlctl.OpenMysqld(uint32(*tabletUID), dbconfigFlags) + mysqld, err := mysqlctl.OpenMysqld(uint32(*tabletUID)) if err != nil { return fmt.Errorf("failed to find mysql config: %v", err) } @@ -104,7 +98,7 @@ func shutdownCmd(subFlags *flag.FlagSet, args []string) error { subFlags.Parse(args) // There ought to be an existing my.cnf, so use it to find mysqld. - mysqld, err := mysqlctl.OpenMysqld(uint32(*tabletUID), dbconfigFlags) + mysqld, err := mysqlctl.OpenMysqld(uint32(*tabletUID)) if err != nil { return fmt.Errorf("failed to find mysql config: %v", err) } @@ -125,7 +119,7 @@ func startCmd(subFlags *flag.FlagSet, args []string) error { subFlags.Parse(args) // There ought to be an existing my.cnf, so use it to find mysqld. - mysqld, err := mysqlctl.OpenMysqld(uint32(*tabletUID), dbconfigFlags) + mysqld, err := mysqlctl.OpenMysqld(uint32(*tabletUID)) if err != nil { return fmt.Errorf("failed to find mysql config: %v", err) } @@ -145,7 +139,7 @@ func teardownCmd(subFlags *flag.FlagSet, args []string) error { subFlags.Parse(args) // There ought to be an existing my.cnf, so use it to find mysqld. - mysqld, err := mysqlctl.OpenMysqld(uint32(*tabletUID), dbconfigFlags) + mysqld, err := mysqlctl.OpenMysqld(uint32(*tabletUID)) if err != nil { return fmt.Errorf("failed to find mysql config: %v", err) } @@ -241,7 +235,7 @@ func main() { fmt.Fprintf(os.Stderr, "\n") } - dbconfigs.RegisterFlags(dbconfigFlags) + dbconfigs.RegisterFlags(dbconfigs.Dba) flag.Parse() tabletAddr = netutil.JoinHostPort("localhost", int32(*port)) diff --git a/go/cmd/mysqlctld/mysqlctld.go b/go/cmd/mysqlctld/mysqlctld.go index f1aca79fe23..b304f9dcc71 100644 --- a/go/cmd/mysqlctld/mysqlctld.go +++ b/go/cmd/mysqlctld/mysqlctld.go @@ -56,8 +56,7 @@ func main() { defer logutil.Flush() // mysqlctld only starts and stops mysql, only needs dba. - dbconfigFlags := dbconfigs.DbaConfig - dbconfigs.RegisterFlags(dbconfigFlags) + dbconfigs.RegisterFlags(dbconfigs.Dba) servenv.ParseFlags("mysqlctld") // We'll register this OnTerm handler before mysqld starts, so we get notified @@ -75,7 +74,7 @@ func main() { log.Infof("mycnf file (%s) doesn't exist, initializing", mycnfFile) var err error - mysqld, err = mysqlctl.CreateMysqld(uint32(*tabletUID), *mysqlSocket, int32(*mysqlPort), dbconfigFlags) + mysqld, err = mysqlctl.CreateMysqld(uint32(*tabletUID), *mysqlSocket, int32(*mysqlPort)) if err != nil { log.Errorf("failed to initialize mysql config: %v", err) exit.Return(1) @@ -91,7 +90,7 @@ func main() { log.Infof("mycnf file (%s) already exists, starting without init", mycnfFile) var err error - mysqld, err = mysqlctl.OpenMysqld(uint32(*tabletUID), dbconfigFlags) + mysqld, err = mysqlctl.OpenMysqld(uint32(*tabletUID)) if err != nil { log.Errorf("failed to find mysql config: %v", err) exit.Return(1) diff --git a/go/cmd/vtcombo/main.go b/go/cmd/vtcombo/main.go index 30eca302d16..9171f123638 100644 --- a/go/cmd/vtcombo/main.go +++ b/go/cmd/vtcombo/main.go @@ -63,9 +63,7 @@ func main() { defer exit.Recover() // flag parsing - dbconfigFlags := dbconfigs.AppConfig | dbconfigs.AllPrivsConfig | dbconfigs.DbaConfig | - dbconfigs.FilteredConfig | dbconfigs.ReplConfig - dbconfigs.RegisterFlags(dbconfigFlags) + dbconfigs.RegisterFlags(dbconfigs.All...) mysqlctl.RegisterFlags() servenv.ParseFlags("vtcombo") @@ -102,15 +100,15 @@ func main() { log.Errorf("mycnf read failed: %v", err) exit.Return(1) } - dbcfgs, err := dbconfigs.Init(mycnf.SocketFile, dbconfigFlags) + dbcfgs, err := dbconfigs.Init(mycnf.SocketFile) if err != nil { log.Warning(err) } - mysqld := mysqlctl.NewMysqld(mycnf, dbcfgs, dbconfigFlags) + mysqld := mysqlctl.NewMysqld(mycnf, dbcfgs) servenv.OnClose(mysqld.Close) // tablets configuration and init - if err := vtcombo.InitTabletMap(ts, tpb, mysqld, *dbcfgs, *schemaDir, mycnf); err != nil { + if err := vtcombo.InitTabletMap(ts, tpb, mysqld, dbcfgs, *schemaDir, mycnf); err != nil { log.Errorf("initTabletMapProto failed: %v", err) exit.Return(1) } diff --git a/go/cmd/vtqueryserver/vtqueryserver.go b/go/cmd/vtqueryserver/vtqueryserver.go index 6a189096016..2dcb629a60d 100644 --- a/go/cmd/vtqueryserver/vtqueryserver.go +++ b/go/cmd/vtqueryserver/vtqueryserver.go @@ -28,16 +28,19 @@ import ( ) var ( + dbName string mysqlSocketFile = flag.String("mysql-socket-file", "", "path to unix socket file to connect to mysql") ) func init() { servenv.RegisterDefaultFlags() + // TODO(demmer): remove once migrated to using db_name + flag.StringVar(&dbName, "db-config-app-dbname", "", "db connection dbname") + flag.StringVar(&dbName, "db_name", "", "db connection dbname") } func main() { - dbconfigFlags := dbconfigs.AppConfig | dbconfigs.AppDebugConfig - dbconfigs.RegisterFlags(dbconfigFlags) + dbconfigs.RegisterFlags(dbconfigs.App, dbconfigs.AppDebug) flag.Parse() if *servenv.Version { @@ -57,10 +60,12 @@ func main() { servenv.Init() - dbcfgs, err := dbconfigs.Init(*mysqlSocketFile, dbconfigFlags) + dbcfgs, err := dbconfigs.Init(*mysqlSocketFile) if err != nil { log.Fatal(err) } + // DB name must be explicitly set. + dbcfgs.DBName.Set(dbName) err = vtqueryserver.Init(dbcfgs) if err != nil { diff --git a/go/cmd/vttablet/vttablet.go b/go/cmd/vttablet/vttablet.go index a3130b70002..e96f7984604 100644 --- a/go/cmd/vttablet/vttablet.go +++ b/go/cmd/vttablet/vttablet.go @@ -47,9 +47,7 @@ func init() { } func main() { - dbconfigFlags := dbconfigs.AppConfig | dbconfigs.AppDebugConfig | dbconfigs.AllPrivsConfig | dbconfigs.DbaConfig | - dbconfigs.FilteredConfig | dbconfigs.ReplConfig - dbconfigs.RegisterFlags(dbconfigFlags) + dbconfigs.RegisterFlags(dbconfigs.All...) mysqlctl.RegisterFlags() servenv.ParseFlags("vttablet") @@ -75,7 +73,7 @@ func main() { log.Exitf("mycnf read failed: %v", err) } - dbcfgs, err := dbconfigs.Init(mycnf.SocketFile, dbconfigFlags) + dbcfgs, err := dbconfigs.Init(mycnf.SocketFile) if err != nil { log.Warning(err) } @@ -117,7 +115,7 @@ func main() { // Create mysqld and register the health reporter (needs to be done // before initializing the agent, so the initial health check // done by the agent has the right reporter) - mysqld := mysqlctl.NewMysqld(mycnf, dbcfgs, dbconfigFlags) + mysqld := mysqlctl.NewMysqld(mycnf, dbcfgs) servenv.OnClose(mysqld.Close) // Depends on both query and updateStream. @@ -125,7 +123,7 @@ func main() { if servenv.GRPCPort != nil { gRPCPort = int32(*servenv.GRPCPort) } - agent, err = tabletmanager.NewActionAgent(context.Background(), ts, mysqld, qsc, tabletAlias, *dbcfgs, mycnf, int32(*servenv.Port), gRPCPort) + agent, err = tabletmanager.NewActionAgent(context.Background(), ts, mysqld, qsc, tabletAlias, dbcfgs, mycnf, int32(*servenv.Port), gRPCPort) if err != nil { log.Exitf("NewActionAgent() failed: %v", err) } diff --git a/go/vt/binlog/binlogplayer/dbclient.go b/go/vt/binlog/binlogplayer/dbclient.go index fb3652d1110..4ceb692e078 100644 --- a/go/vt/binlog/binlogplayer/dbclient.go +++ b/go/vt/binlog/binlogplayer/dbclient.go @@ -53,7 +53,7 @@ func (dc *DBClient) Connect() error { return err } ctx := context.Background() - dc.dbConn, err = mysql.Connect(ctx, ¶ms) + dc.dbConn, err = mysql.Connect(ctx, params) if err != nil { return fmt.Errorf("error in connecting to mysql db, err %v", err) } diff --git a/go/vt/binlog/slave_connection.go b/go/vt/binlog/slave_connection.go index f3bd4bf1008..2c8db9bdf85 100644 --- a/go/vt/binlog/slave_connection.go +++ b/go/vt/binlog/slave_connection.go @@ -76,7 +76,7 @@ func connectForReplication(cp *mysql.ConnParams) (*mysql.Conn, error) { } ctx := context.Background() - conn, err := mysql.Connect(ctx, ¶ms) + conn, err := mysql.Connect(ctx, params) if err != nil { return nil, err } diff --git a/go/vt/dbconfigs/credentials.go b/go/vt/dbconfigs/credentials.go index 63116542761..ab73e9d5695 100644 --- a/go/vt/dbconfigs/credentials.go +++ b/go/vt/dbconfigs/credentials.go @@ -110,7 +110,7 @@ func (fcs *FileCredentialsServer) GetUserAndPassword(user string) (string, strin // WithCredentials returns a copy of the provided ConnParams that we can use // to connect, after going through the CredentialsServer. -func WithCredentials(cp *mysql.ConnParams) (mysql.ConnParams, error) { +func WithCredentials(cp *mysql.ConnParams) (*mysql.ConnParams, error) { result := *cp user, passwd, err := GetCredentialsServer().GetUserAndPassword(cp.Uname) switch err { @@ -121,7 +121,7 @@ func WithCredentials(cp *mysql.ConnParams) (mysql.ConnParams, error) { // we just use what we have, and will fail later anyway err = nil } - return result, err + return &result, err } func init() { diff --git a/go/vt/dbconfigs/dbconfigs.go b/go/vt/dbconfigs/dbconfigs.go index f3cfda72ad5..33e4d6c49a9 100644 --- a/go/vt/dbconfigs/dbconfigs.go +++ b/go/vt/dbconfigs/dbconfigs.go @@ -14,8 +14,10 @@ See the License for the specific language governing permissions and limitations under the License. */ -// Package dbconfigs is reusable by vt tools to load -// the db configs file. +// Package dbconfigs provides the registration for command line options +// to collect db connection parameters. Once registered and collected, +// it provides variables and functions to build connection parameters +// for connecting to the database. package dbconfigs import ( @@ -24,191 +26,256 @@ import ( "fmt" "vitess.io/vitess/go/mysql" + "vitess.io/vitess/go/sync2" "vitess.io/vitess/go/vt/log" ) -// We keep a global singleton for the db configs, and that's the one -// the flags will change -var dbConfigs = DBConfigs{ - SidecarDBName: "_vt", +var ( + dbConfigs = DBConfigs{userConfigs: make(map[string]*userConfig)} + baseConfig = mysql.ConnParams{} +) + +// DBConfigs stores all the data needed to build various connection +// parameters for the db. It stores credentials for app, appdebug, +// allprivs, dba, filtered and repl users. +// It contains other connection parameters like socket, charset, etc. +// It also stores the default db name, which it can combine with the +// rest of the data to build db-sepcific connection parameters. +// It also supplies the SidecarDBName. This is currently hardcoded +// to "_vt", but will soon become customizable. +// The life-cycle of this package is as follows: +// App must call RegisterFlags to request the types of connections +// it wants support for. This must be done before involing flags.Parse. +// After flag parsing, app invokes the Init function, which will return +// a DBConfigs object. +// The app must store the DBConfigs object internally, and use it to +// build connection parameters as needed. +// The DBName is initially empty and may later be set or changed by the app. +type DBConfigs struct { + userConfigs map[string]*userConfig + DBName sync2.AtomicString + SidecarDBName sync2.AtomicString } -// DBConfigFlag describes which flags we need -type DBConfigFlag int +type userConfig struct { + useSSL bool + param mysql.ConnParams +} // config flags const ( - EmptyConfig DBConfigFlag = 0 - AppConfig DBConfigFlag = 1 << iota - AppDebugConfig + App = "app" + AppDebug = "appdebug" // AllPrivs user should have more privileges than App (should include possibility to do // schema changes and write to internal Vitess tables), but it shouldn't have SUPER // privilege like Dba has. - AllPrivsConfig - DbaConfig - FilteredConfig - ReplConfig + AllPrivs = "allprivs" + Dba = "dba" + Filtered = "filtered" + Repl = "repl" ) -// redactedPassword is used in redacted configs so it's not in logs. -const redactedPassword = "****" - -// The flags will change the global singleton -func registerConnFlags(connParams *mysql.ConnParams, name string) { - flag.StringVar(&connParams.Host, "db-config-"+name+"-host", "", "db "+name+" connection host") - flag.IntVar(&connParams.Port, "db-config-"+name+"-port", 0, "db "+name+" connection port") - flag.StringVar(&connParams.Uname, "db-config-"+name+"-uname", "", "db "+name+" connection uname") - flag.StringVar(&connParams.Pass, "db-config-"+name+"-pass", "", "db "+name+" connection pass") - flag.StringVar(&connParams.DbName, "db-config-"+name+"-dbname", "", "db "+name+" connection dbname") - flag.StringVar(&connParams.UnixSocket, "db-config-"+name+"-unixsocket", "", "db "+name+" connection unix socket") - flag.StringVar(&connParams.Charset, "db-config-"+name+"-charset", "", "db "+name+" connection charset") - flag.Uint64Var(&connParams.Flags, "db-config-"+name+"-flags", 0, "db "+name+" connection flags") - flag.StringVar(&connParams.SslCa, "db-config-"+name+"-ssl-ca", "", "db "+name+" connection ssl ca") - flag.StringVar(&connParams.SslCaPath, "db-config-"+name+"-ssl-ca-path", "", "db "+name+" connection ssl ca path") - flag.StringVar(&connParams.SslCert, "db-config-"+name+"-ssl-cert", "", "db "+name+" connection ssl certificate") - flag.StringVar(&connParams.SslKey, "db-config-"+name+"-ssl-key", "", "db "+name+" connection ssl key") -} +// All can be used to register all flags: RegisterFlags(All...) +var All = []string{App, AppDebug, AllPrivs, Dba, Filtered, Repl} // RegisterFlags registers the flags for the given DBConfigFlag. // For instance, vttablet will register client, dba and repl. // Returns all registered flags. -func RegisterFlags(flags DBConfigFlag) DBConfigFlag { - if flags == EmptyConfig { - panic("No DB config is provided.") - } - registeredFlags := EmptyConfig - if AppConfig&flags != 0 { - registerConnFlags(&dbConfigs.App, "app") - registeredFlags |= AppConfig +func RegisterFlags(userKeys ...string) { + registerBaseFlags() + for _, userKey := range userKeys { + uc := &userConfig{} + dbConfigs.userConfigs[userKey] = uc + registerPerUserFlags(uc, userKey) } - if AppDebugConfig&flags != 0 { - registerConnFlags(&dbConfigs.AppDebug, "appdebug") - registeredFlags |= AppDebugConfig - } - if AllPrivsConfig&flags != 0 { - registerConnFlags(&dbConfigs.AllPrivs, "allprivs") - registeredFlags |= AllPrivsConfig - } - if DbaConfig&flags != 0 { - registerConnFlags(&dbConfigs.Dba, "dba") - registeredFlags |= DbaConfig - } - if FilteredConfig&flags != 0 { - registerConnFlags(&dbConfigs.Filtered, "filtered") - registeredFlags |= FilteredConfig - } - if ReplConfig&flags != 0 { - registerConnFlags(&dbConfigs.Repl, "repl") - registeredFlags |= ReplConfig - } - return registeredFlags } -// initConnParams may overwrite the socket file, -// and refresh the password to check that works. -func initConnParams(cp *mysql.ConnParams, socketFile string) error { - // Always try to connect with the socket if provided. - if socketFile != "" { - cp.UnixSocket = socketFile - } +func registerBaseFlags() { + flag.StringVar(&baseConfig.UnixSocket, "db_socket", "", "The unix socket to connect on. If this is specifed, host and port will not be used.") + flag.StringVar(&baseConfig.Host, "db_host", "", "The host name for the tcp connection.") + flag.IntVar(&baseConfig.Port, "db_port", 0, "tcp port") + flag.StringVar(&baseConfig.Charset, "db_charset", "utf8", "Character set. Only utf8 or latin1 based character sets are supported.") + flag.Uint64Var(&baseConfig.Flags, "db_flags", 0, "Flag values as defined by MySQL.") + flag.StringVar(&baseConfig.SslCa, "db_ssl_ca", "", "connection ssl ca") + flag.StringVar(&baseConfig.SslCaPath, "db_ssl_ca_path", "", "connection ssl ca path") + flag.StringVar(&baseConfig.SslCert, "db_ssl_cert", "", "connection ssl certificate") + flag.StringVar(&baseConfig.SslKey, "db_ssl_key", "", "connection ssl key") +} - // See if the CredentialsServer is working. We do not use the - // result for anything, this is just a check. - _, err := WithCredentials(cp) - return err -} - -// DBConfigs is all we need for a smart tablet server: -// - App access with db name for serving app queries -// - AllPrivs access for administrative actions (like schema changes) -// that should be done without SUPER privilege -// - Dba access for any dba-type operation (db creation, replication, ...) -// - Filtered access for filtered replication -// - Replication access to change master -// - SidecarDBName for storing operational metadata -type DBConfigs struct { - App mysql.ConnParams - AppDebug mysql.ConnParams - AllPrivs mysql.ConnParams - Dba mysql.ConnParams - Filtered mysql.ConnParams - Repl mysql.ConnParams - SidecarDBName string +// The flags will change the global singleton +// TODO(sougou): deprecate the legacy flags. +func registerPerUserFlags(dbc *userConfig, userKey string) { + newUserFlag := "db_" + userKey + "_user" + flag.StringVar(&dbc.param.Uname, "db-config-"+userKey+"-uname", "vt_"+userKey, "deprecated: use "+newUserFlag) + flag.StringVar(&dbc.param.Uname, newUserFlag, "vt_"+userKey, "db "+userKey+" user userKey") + + newPasswordFlag := "db_" + userKey + "_password" + flag.StringVar(&dbc.param.Pass, "db-config-"+userKey+"-pass", "", "db "+userKey+" deprecated: use "+newPasswordFlag) + flag.StringVar(&dbc.param.Pass, newPasswordFlag, "", "db "+userKey+" password") + + flag.BoolVar(&dbc.useSSL, "db_"+userKey+"_use_ssl", true, "Set this flag to false to make the "+userKey+" connection to not use ssl") + + flag.StringVar(&dbc.param.Host, "db-config-"+userKey+"-host", "", "deprecated: use db_host") + flag.IntVar(&dbc.param.Port, "db-config-"+userKey+"-port", 0, "deprecated: use db_port") + flag.StringVar(&dbc.param.UnixSocket, "db-config-"+userKey+"-unixsocket", "", "deprecated: use db_socket") + flag.StringVar(&dbc.param.Charset, "db-config-"+userKey+"-charset", "utf8", "deprecated: use db_charset") + flag.Uint64Var(&dbc.param.Flags, "db-config-"+userKey+"-flags", 0, "deprecated: use db_flags") + flag.StringVar(&dbc.param.SslCa, "db-config-"+userKey+"-ssl-ca", "", "deprecated: use db_ssl_ca") + flag.StringVar(&dbc.param.SslCaPath, "db-config-"+userKey+"-ssl-ca-path", "", "deprecated: use db_ssl_ca_path") + flag.StringVar(&dbc.param.SslCert, "db-config-"+userKey+"-ssl-cert", "", "deprecated: use db_ssl_cert") + flag.StringVar(&dbc.param.SslKey, "db-config-"+userKey+"-ssl-key", "", "deprecated: use db_ssl_key") } -func (dbcfgs *DBConfigs) String() string { - if dbcfgs.App.Pass != redactedPassword { - panic("Cannot log a non-redacted DBConfig") - } - if dbcfgs.AppDebug.Pass != redactedPassword { - panic("Cannot log a non-redacted DBConfig") - } - data, err := json.MarshalIndent(dbcfgs, "", " ") - if err != nil { - return err.Error() - } - return string(data) +// AppWithDB returns connection parameters for app with dbname set. +func (dbcfgs *DBConfigs) AppWithDB() *mysql.ConnParams { + return dbcfgs.makeParams(App, true) +} + +// AppDebugWithDB returns connection parameters for appdebug with dbname set. +func (dbcfgs *DBConfigs) AppDebugWithDB() *mysql.ConnParams { + return dbcfgs.makeParams(AppDebug, true) } -// Redact will remove the password, so the object can be logged -func (dbcfgs *DBConfigs) Redact() { - dbcfgs.App.Pass = redactedPassword - dbcfgs.AppDebug.Pass = redactedPassword - dbcfgs.AllPrivs.Pass = redactedPassword - dbcfgs.Dba.Pass = redactedPassword - dbcfgs.Filtered.Pass = redactedPassword - dbcfgs.Repl.Pass = redactedPassword +// AllPrivsWithDB returns connection parameters for appdebug with dbname set. +func (dbcfgs *DBConfigs) AllPrivsWithDB() *mysql.ConnParams { + return dbcfgs.makeParams(AllPrivs, true) +} + +// Dba returns connection parameters for dba with no dbname set. +func (dbcfgs *DBConfigs) Dba() *mysql.ConnParams { + return dbcfgs.makeParams(Dba, false) +} + +// DbaWithDB returns connection parameters for appdebug with dbname set. +func (dbcfgs *DBConfigs) DbaWithDB() *mysql.ConnParams { + return dbcfgs.makeParams(Dba, true) +} + +// FilteredWithDB returns connection parameters for appdebug with dbname set. +func (dbcfgs *DBConfigs) FilteredWithDB() *mysql.ConnParams { + return dbcfgs.makeParams(Filtered, true) +} + +// Repl returns connection parameters for appdebug with no dbname set. +func (dbcfgs *DBConfigs) Repl() *mysql.ConnParams { + return dbcfgs.makeParams(Repl, false) +} + +// AppWithDB returns connection parameters for app with dbname set. +func (dbcfgs *DBConfigs) makeParams(userKey string, withDB bool) *mysql.ConnParams { + orig := dbcfgs.userConfigs[userKey] + if orig == nil { + return &mysql.ConnParams{} + } + result := orig.param + if withDB { + result.DbName = dbcfgs.DBName.Get() + } + return &result } // IsZero returns true if DBConfigs was uninitialized. func (dbcfgs *DBConfigs) IsZero() bool { - return dbcfgs.App.Uname == "" + return len(dbcfgs.userConfigs) == 0 } -// Init will initialize app, allprivs, dba, filtered and repl configs. -func Init(socketFile string, flags DBConfigFlag) (*DBConfigs, error) { - if flags == EmptyConfig { - panic("No DB config is provided.") +func (dbcfgs *DBConfigs) String() string { + out := struct { + Conn mysql.ConnParams + Users map[string]string + }{ + Users: make(map[string]string), } - if AppConfig&flags != 0 { - if err := initConnParams(&dbConfigs.App, socketFile); err != nil { - return nil, fmt.Errorf("app dbconfig cannot be initialized: %v", err) - } + if conn := dbcfgs.userConfigs[App]; conn != nil { + out.Conn = conn.param + } else if conn := dbcfgs.userConfigs[Dba]; conn != nil { + out.Conn = conn.param } - if AppDebugConfig&flags != 0 { - if err := initConnParams(&dbConfigs.AppDebug, socketFile); err != nil { - return nil, fmt.Errorf("appdebug dbconfig cannot be initialized: %v", err) - } + out.Conn.Pass = "****" + for k, uc := range dbcfgs.userConfigs { + out.Users[k] = uc.param.Uname } - if AllPrivsConfig&flags != 0 { - if err := initConnParams(&dbConfigs.AllPrivs, socketFile); err != nil { - return nil, fmt.Errorf("allprivs dbconfig cannot be initialized: %v", err) - } + data, err := json.MarshalIndent(out, "", " ") + if err != nil { + return err.Error() } - if DbaConfig&flags != 0 { - if err := initConnParams(&dbConfigs.Dba, socketFile); err != nil { - return nil, fmt.Errorf("dba dbconfig cannot be initialized: %v", err) - } + return string(data) +} + +// Copy returns a copy of the DBConfig. +func (dbcfgs *DBConfigs) Copy() *DBConfigs { + result := &DBConfigs{userConfigs: make(map[string]*userConfig)} + for k, u := range dbcfgs.userConfigs { + newu := *u + result.userConfigs[k] = &newu } - if FilteredConfig&flags != 0 { - if err := initConnParams(&dbConfigs.Filtered, socketFile); err != nil { - return nil, fmt.Errorf("filtered dbconfig cannot be initialized: %v", err) + result.DBName.Set(dbcfgs.DBName.Get()) + result.SidecarDBName.Set(dbcfgs.SidecarDBName.Get()) + return result +} + +// Init will initialize all the necessary connection parameters. +// Precedence is as follows: if baseConfig command line options are +// set, they supersede all other settings. +// If baseConfig is not set, the next priority is with per-user connection +// parameters. This is only for legacy support. +// If no per-user parameters are supplied, then the defaultSocketFile +// is used to initialize the per-user conn params. +func Init(defaultSocketFile string) (*DBConfigs, error) { + // This is to support legacy behavior: use supplied socket value + // if conn parameters are not specified. + // TODO(sougou): deprecate. + for _, uc := range dbConfigs.userConfigs { + if uc.param.UnixSocket == "" && uc.param.Host == "" { + uc.param.UnixSocket = defaultSocketFile } } - if ReplConfig&flags != 0 { - if err := initConnParams(&dbConfigs.Repl, socketFile); err != nil { - return nil, fmt.Errorf("repl dbconfig cannot be initialized: %v", err) + + // The new base configs, if set, supersede legacy settings. + if baseConfig.Host != "" || baseConfig.UnixSocket != "" { + for _, uc := range dbConfigs.userConfigs { + uc.param.Host = baseConfig.Host + uc.param.Port = baseConfig.Port + uc.param.UnixSocket = baseConfig.UnixSocket + uc.param.Charset = baseConfig.Charset + uc.param.Flags = baseConfig.Flags + if uc.useSSL { + uc.param.SslCa = baseConfig.SslCa + uc.param.SslCaPath = baseConfig.SslCaPath + uc.param.SslCert = baseConfig.SslCert + uc.param.SslKey = baseConfig.SslKey + } } } - // the Dba connection is not linked to a specific database - // (allows us to create them) - if dbConfigs.Dba.DbName != "" { - log.Warningf("dba dbname is set to '%v', ignoring the value", dbConfigs.Dba.DbName) - dbConfigs.Dba.DbName = "" + + // See if the CredentialsServer is working. We do not use the + // result for anything, this is just a check. + for _, uc := range dbConfigs.userConfigs { + if _, err := WithCredentials(&uc.param); err != nil { + return nil, fmt.Errorf("dbconfig cannot be initialized: %v", err) + } + // Check for only one. + break } + dbConfigs.SidecarDBName.Set("_vt") - toLog := dbConfigs - toLog.Redact() - log.Infof("DBConfigs: %v\n", toLog.String()) + log.Infof("DBConfigs: %v\n", dbConfigs.String()) return &dbConfigs, nil } + +// NewTestDBConfigs returns a DBConfigs meant for testing. +func NewTestDBConfigs(genParams, appDebugParams mysql.ConnParams, dbName string) *DBConfigs { + dbcfgs := &DBConfigs{ + userConfigs: map[string]*userConfig{ + App: {param: genParams}, + AppDebug: {param: appDebugParams}, + AllPrivs: {param: genParams}, + Dba: {param: genParams}, + Filtered: {param: genParams}, + Repl: {param: genParams}, + }, + } + dbcfgs.DBName.Set(dbName) + dbcfgs.SidecarDBName.Set("_vt") + return dbcfgs +} diff --git a/go/vt/dbconfigs/dbconfigs_test.go b/go/vt/dbconfigs/dbconfigs_test.go index 83fef537923..0f6e87fd4c5 100644 --- a/go/vt/dbconfigs/dbconfigs_test.go +++ b/go/vt/dbconfigs/dbconfigs_test.go @@ -16,40 +16,204 @@ limitations under the License. package dbconfigs -import "testing" +import ( + "reflect" + "testing" -func TestRegisterFlagsWithoutFlags(t *testing.T) { + "vitess.io/vitess/go/mysql" +) + +func TestRegisterFlagsWithSomeFlags(t *testing.T) { + savedDBConfig := dbConfigs + savedBaseConfig := baseConfig defer func() { - if r := recover(); r == nil { - t.Error("RegisterFlags should panic with empty db flags") + dbConfigs = savedDBConfig + baseConfig = savedBaseConfig + }() + + dbConfigs = DBConfigs{userConfigs: make(map[string]*userConfig)} + RegisterFlags(Dba, Repl) + for k := range dbConfigs.userConfigs { + if k != Dba && k != Repl { + t.Errorf("dbConfigs.params: %v, want dba or repl", k) } + } +} + +func TestInit(t *testing.T) { + savedDBConfig := dbConfigs + savedBaseConfig := baseConfig + defer func() { + dbConfigs = savedDBConfig + baseConfig = savedBaseConfig }() - dbConfigs = DBConfigs{} - RegisterFlags(EmptyConfig) + + dbConfigs = DBConfigs{ + userConfigs: map[string]*userConfig{ + App: {param: mysql.ConnParams{UnixSocket: "socket"}}, + AppDebug: {}, + Dba: {param: mysql.ConnParams{Host: "host"}}, + }, + } + dbc, err := Init("default") + if err != nil { + t.Fatal(err) + } + if got, want := dbc.userConfigs[App].param.UnixSocket, "socket"; got != want { + t.Errorf("dbc.app.UnixSocket: %v, want %v", got, want) + } + if got, want := dbc.userConfigs[Dba].param.Host, "host"; got != want { + t.Errorf("dbc.app.Host: %v, want %v", got, want) + } + if got, want := dbc.userConfigs[AppDebug].param.UnixSocket, "default"; got != want { + t.Errorf("dbc.app.UnixSocket: %v, want %v", got, want) + } + + baseConfig = mysql.ConnParams{ + Host: "a", + Port: 1, + Uname: "b", + Pass: "c", + DbName: "d", + UnixSocket: "e", + Charset: "f", + Flags: 2, + SslCa: "g", + SslCaPath: "h", + SslCert: "i", + SslKey: "j", + } + dbConfigs = DBConfigs{ + userConfigs: map[string]*userConfig{ + App: { + param: mysql.ConnParams{ + Uname: "app", + Pass: "apppass", + UnixSocket: "socket", + }, + }, + AppDebug: { + useSSL: true, + }, + Dba: { + useSSL: true, + param: mysql.ConnParams{ + Uname: "dba", + Pass: "dbapass", + Host: "host", + }, + }, + }, + } + dbc, err = Init("default") + if err != nil { + t.Fatal(err) + } + want := &DBConfigs{ + userConfigs: map[string]*userConfig{ + App: { + param: mysql.ConnParams{ + Host: "a", + Port: 1, + Uname: "app", + Pass: "apppass", + UnixSocket: "e", + Charset: "f", + Flags: 2, + }, + }, + AppDebug: { + useSSL: true, + param: mysql.ConnParams{ + Host: "a", + Port: 1, + UnixSocket: "e", + Charset: "f", + Flags: 2, + SslCa: "g", + SslCaPath: "h", + SslCert: "i", + SslKey: "j", + }, + }, + Dba: { + useSSL: true, + param: mysql.ConnParams{ + Host: "a", + Port: 1, + Uname: "dba", + Pass: "dbapass", + UnixSocket: "e", + Charset: "f", + Flags: 2, + SslCa: "g", + SslCaPath: "h", + SslCert: "i", + SslKey: "j", + }, + }, + }, + } + // Compare individually, otherwise the errors are not readable. + if !reflect.DeepEqual(dbc.userConfigs[App].param, want.userConfigs[App].param) { + t.Errorf("dbc: \n%#v, want \n%#v", dbc.userConfigs[App].param, want.userConfigs[App].param) + } + if !reflect.DeepEqual(dbc.userConfigs[AppDebug].param, want.userConfigs[AppDebug].param) { + t.Errorf("dbc: \n%#v, want \n%#v", dbc.userConfigs[AppDebug].param, want.userConfigs[AppDebug].param) + } + if !reflect.DeepEqual(dbc.userConfigs[Dba].param, want.userConfigs[Dba].param) { + t.Errorf("dbc: \n%#v, want \n%#v", dbc.userConfigs[Dba].param, want.userConfigs[Dba].param) + } } -func TestRegisterFlagsWithSomeFlags(t *testing.T) { - dbConfigs = DBConfigs{} - registeredFlags := RegisterFlags(DbaConfig | ReplConfig) - if registeredFlags&AppConfig != 0 { - t.Error("App connection params should not be registered.") +func TestAccessors(t *testing.T) { + dbc := &DBConfigs{ + userConfigs: map[string]*userConfig{ + App: {}, + AppDebug: {}, + AllPrivs: {}, + Dba: {}, + Filtered: {}, + Repl: {}, + }, + } + dbc.DBName.Set("db") + if got, want := dbc.AppWithDB().DbName, "db"; got != want { + t.Errorf("dbc.AppWithDB().DbName: %v, want %v", got, want) + } + if got, want := dbc.AllPrivsWithDB().DbName, "db"; got != want { + t.Errorf("dbc.AllPrivsWithDB().DbName: %v, want %v", got, want) } - if registeredFlags&DbaConfig == 0 { - t.Error("Dba connection params should be registered.") + if got, want := dbc.AppDebugWithDB().DbName, "db"; got != want { + t.Errorf("dbc.AppDebugWithDB().DbName: %v, want %v", got, want) } - if registeredFlags&FilteredConfig != 0 { - t.Error("Filtered connection params should not be registered.") + if got, want := dbc.Dba().DbName, ""; got != want { + t.Errorf("dbc.Dba().DbName: %v, want %v", got, want) } - if registeredFlags&ReplConfig == 0 { - t.Error("Repl connection params should be registered.") + if got, want := dbc.DbaWithDB().DbName, "db"; got != want { + t.Errorf("dbc.DbaWithDB().DbName: %v, want %v", got, want) + } + if got, want := dbc.FilteredWithDB().DbName, "db"; got != want { + t.Errorf("dbc.FilteredWithDB().DbName: %v, want %v", got, want) + } + if got, want := dbc.Repl().DbName, ""; got != want { + t.Errorf("dbc.Repl().DbName: %v, want %v", got, want) } } -func TestInitWithEmptyFlags(t *testing.T) { - defer func() { - if r := recover(); r == nil { - t.Error("Init should panic with empty db flags") - } - }() - Init("", EmptyConfig) +func TestCopy(t *testing.T) { + want := &DBConfigs{ + userConfigs: map[string]*userConfig{ + App: {param: mysql.ConnParams{UnixSocket: "aa"}}, + AppDebug: {}, + Repl: {}, + }, + } + want.DBName.Set("db") + want.SidecarDBName.Set("_vt") + + got := want.Copy() + if !reflect.DeepEqual(got, want) { + t.Errorf("DBConfig: %v, want %v", got, want) + } } diff --git a/go/vt/dbconnpool/connection.go b/go/vt/dbconnpool/connection.go index b522b414f04..921c752e5cb 100644 --- a/go/vt/dbconnpool/connection.go +++ b/go/vt/dbconnpool/connection.go @@ -124,7 +124,7 @@ func NewDBConnection(info *mysql.ConnParams, mysqlStats *stats.Timings) (*DBConn return nil, err } ctx := context.Background() - c, err := mysql.Connect(ctx, ¶ms) + c, err := mysql.Connect(ctx, params) if err != nil { mysqlStats.Record("ConnectError", start) } diff --git a/go/vt/mysqlctl/cmd.go b/go/vt/mysqlctl/cmd.go index 35b4b0c5b3b..ab5684d0b81 100644 --- a/go/vt/mysqlctl/cmd.go +++ b/go/vt/mysqlctl/cmd.go @@ -29,7 +29,7 @@ import ( // CreateMysqld returns a Mysqld object to use for working with a MySQL // installation that hasn't been set up yet. This will generate a new my.cnf // from scratch and use that to call NewMysqld(). -func CreateMysqld(tabletUID uint32, mysqlSocket string, mysqlPort int32, dbconfigFlags dbconfigs.DBConfigFlag) (*Mysqld, error) { +func CreateMysqld(tabletUID uint32, mysqlSocket string, mysqlPort int32) (*Mysqld, error) { mycnf := NewMycnf(tabletUID, mysqlPort) // Choose a random MySQL server-id, since this is a fresh data dir. // We don't want to use the tablet UID as the MySQL server-id, @@ -47,28 +47,28 @@ func CreateMysqld(tabletUID uint32, mysqlSocket string, mysqlPort int32, dbconfi mycnf.SocketFile = mysqlSocket } - dbcfgs, err := dbconfigs.Init(mycnf.SocketFile, dbconfigFlags) + dbcfgs, err := dbconfigs.Init(mycnf.SocketFile) if err != nil { return nil, fmt.Errorf("couldn't Init dbconfigs: %v", err) } - return NewMysqld(mycnf, dbcfgs, dbconfigFlags), nil + return NewMysqld(mycnf, dbcfgs), nil } // OpenMysqld returns a Mysqld object to use for working with a MySQL // installation that already exists. This will look for an existing my.cnf file // and use that to call NewMysqld(). -func OpenMysqld(tabletUID uint32, dbconfigFlags dbconfigs.DBConfigFlag) (*Mysqld, error) { +func OpenMysqld(tabletUID uint32) (*Mysqld, error) { // We pass a port of 0, this will be read and overwritten from the path on disk mycnf, err := ReadMycnf(NewMycnf(tabletUID, 0)) if err != nil { return nil, fmt.Errorf("couldn't read my.cnf file: %v", err) } - dbcfgs, err := dbconfigs.Init(mycnf.SocketFile, dbconfigFlags) + dbcfgs, err := dbconfigs.Init(mycnf.SocketFile) if err != nil { return nil, fmt.Errorf("couldn't Init dbconfigs: %v", err) } - return NewMysqld(mycnf, dbcfgs, dbconfigFlags), nil + return NewMysqld(mycnf, dbcfgs), nil } diff --git a/go/vt/mysqlctl/mysqld.go b/go/vt/mysqlctl/mysqld.go index 29fc68485cc..900f8f9ee2c 100644 --- a/go/vt/mysqlctl/mysqld.go +++ b/go/vt/mysqlctl/mysqld.go @@ -94,7 +94,7 @@ type Mysqld struct { // NewMysqld creates a Mysqld object based on the provided configuration // and connection parameters. -func NewMysqld(config *Mycnf, dbcfgs *dbconfigs.DBConfigs, dbconfigsFlags dbconfigs.DBConfigFlag) *Mysqld { +func NewMysqld(config *Mycnf, dbcfgs *dbconfigs.DBConfigs) *Mysqld { result := &Mysqld{ config: config, dbcfgs: dbcfgs, @@ -102,16 +102,12 @@ func NewMysqld(config *Mycnf, dbcfgs *dbconfigs.DBConfigs, dbconfigsFlags dbconf } // Create and open the connection pool for dba access. - if dbconfigs.DbaConfig&dbconfigsFlags != 0 { - result.dbaPool = dbconnpool.NewConnectionPool("DbaConnPool", *dbaPoolSize, *dbaIdleTimeout) - result.dbaPool.Open(&dbcfgs.Dba, dbaMysqlStats) - } + result.dbaPool = dbconnpool.NewConnectionPool("DbaConnPool", *dbaPoolSize, *dbaIdleTimeout) + result.dbaPool.Open(dbcfgs.Dba(), dbaMysqlStats) // Create and open the connection pool for app access. - if dbconfigs.AppConfig&dbconfigsFlags != 0 { - result.appPool = dbconnpool.NewConnectionPool("AppConnPool", *appPoolSize, *appIdleTimeout) - result.appPool.Open(&dbcfgs.App, appMysqlStats) - } + result.appPool = dbconnpool.NewConnectionPool("AppConnPool", *appPoolSize, *appIdleTimeout) + result.appPool.Open(dbcfgs.AppWithDB(), appMysqlStats) return result } @@ -160,11 +156,11 @@ func (mysqld *Mysqld) RunMysqlUpgrade() error { // privileges' right in the middle, and then subsequent // commands fail if we don't use valid credentials. So let's // use dba credentials. - params, err := dbconfigs.WithCredentials(&mysqld.dbcfgs.Dba) + params, err := dbconfigs.WithCredentials(mysqld.dbcfgs.Dba()) if err != nil { return err } - cnf, err := mysqld.defaultsExtraFile(¶ms) + cnf, err := mysqld.defaultsExtraFile(params) if err != nil { return err } @@ -302,7 +298,7 @@ func (mysqld *Mysqld) startNoWait(ctx context.Context, mysqldArgs ...string) err // will use the dba credentials to try to connect. Use wait() with // different credentials if needed. func (mysqld *Mysqld) Wait(ctx context.Context) error { - params, err := dbconfigs.WithCredentials(&mysqld.dbcfgs.Dba) + params, err := dbconfigs.WithCredentials(mysqld.dbcfgs.Dba()) if err != nil { return err } @@ -311,7 +307,7 @@ func (mysqld *Mysqld) Wait(ctx context.Context) error { } // wait is the internal version of Wait, that takes credentials. -func (mysqld *Mysqld) wait(ctx context.Context, params mysql.ConnParams) error { +func (mysqld *Mysqld) wait(ctx context.Context, params *mysql.ConnParams) error { log.Infof("Waiting for mysqld socket file (%v) to be ready...", mysqld.config.SocketFile) for { @@ -324,7 +320,7 @@ func (mysqld *Mysqld) wait(ctx context.Context, params mysql.ConnParams) error { _, statErr := os.Stat(mysqld.config.SocketFile) if statErr == nil { // Make sure the socket file isn't stale. - conn, connErr := mysql.Connect(ctx, ¶ms) + conn, connErr := mysql.Connect(ctx, params) if connErr == nil { conn.Close() return nil @@ -333,7 +329,7 @@ func (mysqld *Mysqld) wait(ctx context.Context, params mysql.ConnParams) error { } else if !os.IsNotExist(statErr) { return fmt.Errorf("can't stat mysqld socket file: %v", statErr) } - time.Sleep(100 * time.Millisecond) + time.Sleep(1000 * time.Millisecond) } } @@ -392,11 +388,11 @@ func (mysqld *Mysqld) Shutdown(ctx context.Context, waitForMysqld bool) error { if err != nil { return err } - params, err := dbconfigs.WithCredentials(&mysqld.dbcfgs.Dba) + params, err := dbconfigs.WithCredentials(mysqld.dbcfgs.Dba()) if err != nil { return err } - cnf, err := mysqld.defaultsExtraFile(¶ms) + cnf, err := mysqld.defaultsExtraFile(params) if err != nil { return err } @@ -528,7 +524,7 @@ func (mysqld *Mysqld) Init(ctx context.Context, initDBSQLFile string) error { // Wait for mysqld to be ready, using root credentials, as no // user is created yet. - params := mysql.ConnParams{ + params := &mysql.ConnParams{ Uname: "root", Charset: "utf8", UnixSocket: mysqld.config.SocketFile, @@ -544,7 +540,7 @@ func (mysqld *Mysqld) Init(ctx context.Context, initDBSQLFile string) error { return fmt.Errorf("can't open init_db_sql_file (%v): %v", initDBSQLFile, err) } defer sqlFile.Close() - if err := mysqld.executeMysqlScript(¶ms, sqlFile); err != nil { + if err := mysqld.executeMysqlScript(params, sqlFile); err != nil { return fmt.Errorf("can't run init_db_sql_file (%v): %v", initDBSQLFile, err) } @@ -911,12 +907,12 @@ func (mysqld *Mysqld) GetAppConnection(ctx context.Context) (*dbconnpool.PooledD // GetDbaConnection creates a new DBConnection. func (mysqld *Mysqld) GetDbaConnection() (*dbconnpool.DBConnection, error) { - return dbconnpool.NewDBConnection(&mysqld.dbcfgs.Dba, dbaMysqlStats) + return dbconnpool.NewDBConnection(mysqld.dbcfgs.Dba(), dbaMysqlStats) } // GetAllPrivsConnection creates a new DBConnection. func (mysqld *Mysqld) GetAllPrivsConnection() (*dbconnpool.DBConnection, error) { - return dbconnpool.NewDBConnection(&mysqld.dbcfgs.AllPrivs, allprivsMysqlStats) + return dbconnpool.NewDBConnection(mysqld.dbcfgs.AllPrivsWithDB(), allprivsMysqlStats) } // Close will close this instance of Mysqld. It will wait for all dba diff --git a/go/vt/mysqlctl/replication.go b/go/vt/mysqlctl/replication.go index 10f52df698c..11e64c2dfa1 100644 --- a/go/vt/mysqlctl/replication.go +++ b/go/vt/mysqlctl/replication.go @@ -216,7 +216,7 @@ func (mysqld *Mysqld) SetSlavePosition(ctx context.Context, pos mysql.Position) // SetMaster makes the provided host / port the master. It optionally // stops replication before, and starts it after. func (mysqld *Mysqld) SetMaster(ctx context.Context, masterHost string, masterPort int, slaveStopBefore bool, slaveStartAfter bool) error { - params, err := dbconfigs.WithCredentials(&mysqld.dbcfgs.Repl) + params, err := dbconfigs.WithCredentials(mysqld.dbcfgs.Repl()) if err != nil { return err } @@ -230,7 +230,7 @@ func (mysqld *Mysqld) SetMaster(ctx context.Context, masterHost string, masterPo if slaveStopBefore { cmds = append(cmds, conn.StopSlaveCommand()) } - smc := conn.SetMasterCommand(¶ms, masterHost, masterPort, int(masterConnectRetry.Seconds())) + smc := conn.SetMasterCommand(params, masterHost, masterPort, int(masterConnectRetry.Seconds())) cmds = append(cmds, smc) if slaveStartAfter { cmds = append(cmds, conn.StartSlaveCommand()) diff --git a/go/vt/mysqlctl/schema.go b/go/vt/mysqlctl/schema.go index 5d65ec239ef..e9d9518a420 100644 --- a/go/vt/mysqlctl/schema.go +++ b/go/vt/mysqlctl/schema.go @@ -37,12 +37,12 @@ var autoIncr = regexp.MustCompile(" AUTO_INCREMENT=\\d+") // executeSchemaCommands executes some SQL commands, using the mysql // command line tool. It uses the dba connection parameters, with credentials. func (mysqld *Mysqld) executeSchemaCommands(sql string) error { - params, err := dbconfigs.WithCredentials(&mysqld.dbcfgs.Dba) + params, err := dbconfigs.WithCredentials(mysqld.dbcfgs.Dba()) if err != nil { return err } - return mysqld.executeMysqlScript(¶ms, strings.NewReader(sql)) + return mysqld.executeMysqlScript(params, strings.NewReader(sql)) } // GetSchema returns the schema for database for tables listed in diff --git a/go/vt/vtcombo/tablet_map.go b/go/vt/vtcombo/tablet_map.go index 9fc9cfa36b4..93f0a3fa9e6 100644 --- a/go/vt/vtcombo/tablet_map.go +++ b/go/vt/vtcombo/tablet_map.go @@ -73,7 +73,7 @@ var tabletMap map[uint32]*tablet // CreateTablet creates an individual tablet, with its agent, and adds // it to the map. If it's a master tablet, it also issues a TER. -func CreateTablet(ctx context.Context, ts *topo.Server, cell string, uid uint32, keyspace, shard, dbname string, tabletType topodatapb.TabletType, mysqld mysqlctl.MysqlDaemon, dbcfgs dbconfigs.DBConfigs) error { +func CreateTablet(ctx context.Context, ts *topo.Server, cell string, uid uint32, keyspace, shard, dbname string, tabletType topodatapb.TabletType, mysqld mysqlctl.MysqlDaemon, dbcfgs *dbconfigs.DBConfigs) error { alias := &topodatapb.TabletAlias{ Cell: cell, Uid: uid, @@ -106,7 +106,7 @@ func CreateTablet(ctx context.Context, ts *topo.Server, cell string, uid uint32, // InitTabletMap creates the action agents and associated data structures // for all tablets, based on the vttest proto parameter. -func InitTabletMap(ts *topo.Server, tpb *vttestpb.VTTestTopology, mysqld mysqlctl.MysqlDaemon, dbcfgs dbconfigs.DBConfigs, schemaDir string, mycnf *mysqlctl.Mycnf) error { +func InitTabletMap(ts *topo.Server, tpb *vttestpb.VTTestTopology, mysqld mysqlctl.MysqlDaemon, dbcfgs *dbconfigs.DBConfigs, schemaDir string, mycnf *mysqlctl.Mycnf) error { tabletMap = make(map[uint32]*tablet) ctx := context.Background() @@ -165,9 +165,9 @@ func InitTabletMap(ts *topo.Server, tpb *vttestpb.VTTestTopology, mysqld mysqlct if dbname == "" { dbname = fmt.Sprintf("vt_%v_%v", keyspace, shard) } - dbcfgs.App.DbName = dbname - // Override SidecarDBName because there will be one for each db. - dbcfgs.SidecarDBName = "_" + dbname + // Copy dbcfgs and override SidecarDBName because there will be one for each db. + copydbcfgs := dbcfgs.Copy() + copydbcfgs.SidecarDBName.Set("_" + dbname) replicas := int(kpb.ReplicaCount) if replicas == 0 { @@ -183,7 +183,7 @@ func InitTabletMap(ts *topo.Server, tpb *vttestpb.VTTestTopology, mysqld mysqlct replicas-- // create the master - if err := CreateTablet(ctx, ts, cell, uid, keyspace, shard, dbname, topodatapb.TabletType_MASTER, mysqld, dbcfgs); err != nil { + if err := CreateTablet(ctx, ts, cell, uid, keyspace, shard, dbname, topodatapb.TabletType_MASTER, mysqld, copydbcfgs); err != nil { return err } uid++ @@ -191,7 +191,7 @@ func InitTabletMap(ts *topo.Server, tpb *vttestpb.VTTestTopology, mysqld mysqlct for i := 0; i < replicas; i++ { // create a replica slave - if err := CreateTablet(ctx, ts, cell, uid, keyspace, shard, dbname, topodatapb.TabletType_REPLICA, mysqld, dbcfgs); err != nil { + if err := CreateTablet(ctx, ts, cell, uid, keyspace, shard, dbname, topodatapb.TabletType_REPLICA, mysqld, copydbcfgs); err != nil { return err } uid++ @@ -199,7 +199,7 @@ func InitTabletMap(ts *topo.Server, tpb *vttestpb.VTTestTopology, mysqld mysqlct for i := 0; i < rdonlys; i++ { // create a rdonly slave - if err := CreateTablet(ctx, ts, cell, uid, keyspace, shard, dbname, topodatapb.TabletType_RDONLY, mysqld, dbcfgs); err != nil { + if err := CreateTablet(ctx, ts, cell, uid, keyspace, shard, dbname, topodatapb.TabletType_RDONLY, mysqld, copydbcfgs); err != nil { return err } uid++ diff --git a/go/vt/vtexplain/vtexplain_vttablet.go b/go/vt/vtexplain/vtexplain_vttablet.go index 6b03e0c87fb..dae8b3929fe 100644 --- a/go/vt/vtexplain/vtexplain_vttablet.go +++ b/go/vt/vtexplain/vtexplain_vttablet.go @@ -93,10 +93,7 @@ func newTablet(opts *Options, t *topodatapb.Tablet) *explainTablet { }, ) - dbcfgs := dbconfigs.DBConfigs{ - App: *db.ConnParams(), - SidecarDBName: "_vt", - } + dbcfgs := dbconfigs.NewTestDBConfigs(*db.ConnParams(), *db.ConnParams(), "") cnf := mysqlctl.NewMycnf(22222, 6802) cnf.ServerID = 33333 diff --git a/go/vt/vtqueryserver/endtoend_test.go b/go/vt/vtqueryserver/endtoend_test.go index 4e216659253..dc4fb82e426 100644 --- a/go/vt/vtqueryserver/endtoend_test.go +++ b/go/vt/vtqueryserver/endtoend_test.go @@ -105,10 +105,8 @@ func TestMain(m *testing.M) { defer func() { tabletenv.Config = tabletenv.DefaultQsConfig }() // Initialize the query service on top of the vttest MySQL database. - dbcfgs := dbconfigs.DBConfigs{ - App: mysqlConnParams, - } - queryServer, err = initProxy(&dbcfgs) + dbcfgs := dbconfigs.NewTestDBConfigs(mysqlConnParams, mysqlConnParams, cluster.DbName()) + queryServer, err = initProxy(dbcfgs) if err != nil { fmt.Fprintf(os.Stderr, "could not start proxy: %v\n", err) return 1 diff --git a/go/vt/vtqueryserver/vtqueryserver.go b/go/vt/vtqueryserver/vtqueryserver.go index 587046a1255..722a3ce49ad 100644 --- a/go/vt/vtqueryserver/vtqueryserver.go +++ b/go/vt/vtqueryserver/vtqueryserver.go @@ -58,7 +58,7 @@ func initProxy(dbcfgs *dbconfigs.DBConfigs) (*tabletserver.TabletServer, error) qs.SetAllowUnsafeDMLs(*allowUnsafeDMLs) mysqlProxy = mysqlproxy.NewProxy(&target, qs, *normalizeQueries) - err := qs.StartService(target, *dbcfgs) + err := qs.StartService(target, dbcfgs) if err != nil { return nil, err } diff --git a/go/vt/vttablet/endtoend/framework/server.go b/go/vt/vttablet/endtoend/framework/server.go index 9a719273e80..ce6c7bd59ef 100644 --- a/go/vt/vttablet/endtoend/framework/server.go +++ b/go/vt/vttablet/endtoend/framework/server.go @@ -49,7 +49,7 @@ var ( // StartServer starts the server and initializes // all the global variables. This function should only be called // once at the beginning of the test. -func StartServer(connParams, connAppDebugParams mysql.ConnParams) error { +func StartServer(connParams, connAppDebugParams mysql.ConnParams, dbName string) error { // Setup a fake vtgate server. protocol := "resolveTest" *vtgateconn.VtgateProtocol = protocol @@ -59,11 +59,7 @@ func StartServer(connParams, connAppDebugParams mysql.ConnParams) error { }, nil }) - dbcfgs := dbconfigs.DBConfigs{ - App: connParams, - AppDebug: connAppDebugParams, - SidecarDBName: "_vt", - } + dbcfgs := dbconfigs.NewTestDBConfigs(connParams, connAppDebugParams, dbName) config := tabletenv.DefaultQsConfig config.EnableAutoCommit = true diff --git a/go/vt/vttablet/endtoend/main_test.go b/go/vt/vttablet/endtoend/main_test.go index f01ef804df2..67f03b8e597 100644 --- a/go/vt/vttablet/endtoend/main_test.go +++ b/go/vt/vttablet/endtoend/main_test.go @@ -79,7 +79,7 @@ func TestMain(m *testing.M) { connParams = cluster.MySQLConnParams() connAppDebugParams = cluster.MySQLAppDebugConnParams() - err := framework.StartServer(connParams, connAppDebugParams) + err := framework.StartServer(connParams, connAppDebugParams, cluster.DbName()) if err != nil { fmt.Fprintf(os.Stderr, "%v", err) return 1 diff --git a/go/vt/vttablet/heartbeat/reader.go b/go/vt/vttablet/heartbeat/reader.go index 07852e51df2..b75eb9ada80 100644 --- a/go/vt/vttablet/heartbeat/reader.go +++ b/go/vt/vttablet/heartbeat/reader.go @@ -48,7 +48,7 @@ const ( // table against the current time at read time. This value is reported in metrics and // also to the healthchecks. type Reader struct { - dbconfigs dbconfigs.DBConfigs + dbconfigs *dbconfigs.DBConfigs enabled bool interval time.Duration @@ -84,7 +84,7 @@ func NewReader(checker connpool.MySQLChecker, config tabletenv.TabletConfig) *Re } // InitDBConfig must be called before Init. -func (r *Reader) InitDBConfig(dbcfgs dbconfigs.DBConfigs) { +func (r *Reader) InitDBConfig(dbcfgs *dbconfigs.DBConfigs) { r.dbconfigs = dbcfgs } @@ -94,7 +94,7 @@ func (r *Reader) Init(target querypb.Target) { if !r.enabled { return } - r.dbName = sqlescape.EscapeID(r.dbconfigs.SidecarDBName) + r.dbName = sqlescape.EscapeID(r.dbconfigs.SidecarDBName.Get()) r.keyspaceShard = fmt.Sprintf("%s:%s", target.Keyspace, target.Shard) } @@ -111,7 +111,7 @@ func (r *Reader) Open() { } log.Info("Beginning heartbeat reads") - r.pool.Open(&r.dbconfigs.App, &r.dbconfigs.Dba, &r.dbconfigs.AppDebug) + r.pool.Open(r.dbconfigs.AppWithDB(), r.dbconfigs.DbaWithDB(), r.dbconfigs.AppDebugWithDB()) r.ticks.Start(func() { r.readHeartbeat() }) r.isOpen = true } diff --git a/go/vt/vttablet/heartbeat/reader_test.go b/go/vt/vttablet/heartbeat/reader_test.go index be26bc5af35..ede891a810b 100644 --- a/go/vt/vttablet/heartbeat/reader_test.go +++ b/go/vt/vttablet/heartbeat/reader_test.go @@ -106,17 +106,13 @@ func newReader(db *fakesqldb.DB, nowFunc func() time.Time) *Reader { config := tabletenv.DefaultQsConfig config.HeartbeatEnable = true config.PoolNamePrefix = fmt.Sprintf("Pool-%d-", randID) - dbc := dbconfigs.DBConfigs{ - App: *db.ConnParams(), - Dba: *db.ConnParams(), - SidecarDBName: "_vt", - } + dbc := dbconfigs.NewTestDBConfigs(*db.ConnParams(), *db.ConnParams(), "") tr := NewReader(&fakeMysqlChecker{}, config) - tr.dbName = sqlescape.EscapeID(dbc.SidecarDBName) + tr.dbName = sqlescape.EscapeID(dbc.SidecarDBName.Get()) tr.keyspaceShard = "test:0" tr.now = nowFunc - tr.pool.Open(&dbc.App, &dbc.Dba, &dbc.AppDebug) + tr.pool.Open(dbc.AppWithDB(), dbc.DbaWithDB(), dbc.AppDebugWithDB()) return tr } diff --git a/go/vt/vttablet/heartbeat/writer.go b/go/vt/vttablet/heartbeat/writer.go index bb7b3f2a2ca..5503aad3599 100644 --- a/go/vt/vttablet/heartbeat/writer.go +++ b/go/vt/vttablet/heartbeat/writer.go @@ -56,7 +56,7 @@ const ( // Writer runs on master tablets and writes heartbeats to the _vt.heartbeat // table at a regular interval, defined by heartbeat_interval. type Writer struct { - dbconfigs dbconfigs.DBConfigs + dbconfigs *dbconfigs.DBConfigs enabled bool interval time.Duration @@ -89,7 +89,7 @@ func NewWriter(checker connpool.MySQLChecker, alias topodatapb.TabletAlias, conf } // InitDBConfig must be called before Init. -func (w *Writer) InitDBConfig(dbcfgs dbconfigs.DBConfigs) { +func (w *Writer) InitDBConfig(dbcfgs *dbconfigs.DBConfigs) { w.dbconfigs = dbcfgs } @@ -102,9 +102,9 @@ func (w *Writer) Init(target querypb.Target) error { w.mu.Lock() defer w.mu.Unlock() log.Info("Initializing heartbeat table.") - w.dbName = sqlescape.EscapeID(w.dbconfigs.SidecarDBName) + w.dbName = sqlescape.EscapeID(w.dbconfigs.SidecarDBName.Get()) w.keyspaceShard = fmt.Sprintf("%s:%s", target.Keyspace, target.Shard) - err := w.initializeTables(&w.dbconfigs.Dba) + err := w.initializeTables(w.dbconfigs.DbaWithDB()) if err != nil { w.recordError(err) return err @@ -127,7 +127,7 @@ func (w *Writer) Open() { return } log.Info("Beginning heartbeat writes") - w.pool.Open(&w.dbconfigs.App, &w.dbconfigs.Dba, &w.dbconfigs.AppDebug) + w.pool.Open(w.dbconfigs.AppWithDB(), w.dbconfigs.DbaWithDB(), w.dbconfigs.AppDebugWithDB()) w.ticks.Start(func() { w.writeHeartbeat() }) w.isOpen = true } diff --git a/go/vt/vttablet/heartbeat/writer_test.go b/go/vt/vttablet/heartbeat/writer_test.go index a1b053b5509..f0f6b6acae1 100644 --- a/go/vt/vttablet/heartbeat/writer_test.go +++ b/go/vt/vttablet/heartbeat/writer_test.go @@ -111,19 +111,15 @@ func newTestWriter(db *fakesqldb.DB, nowFunc func() time.Time) *Writer { config.HeartbeatEnable = true config.PoolNamePrefix = fmt.Sprintf("Pool-%d-", randID) - dbc := dbconfigs.DBConfigs{ - App: *db.ConnParams(), - Dba: *db.ConnParams(), - SidecarDBName: "_vt", - } + dbc := dbconfigs.NewTestDBConfigs(*db.ConnParams(), *db.ConnParams(), "") tw := NewWriter(&fakeMysqlChecker{}, topodatapb.TabletAlias{Cell: "test", Uid: 1111}, config) - tw.dbName = sqlescape.EscapeID(dbc.SidecarDBName) + tw.dbName = sqlescape.EscapeID(dbc.SidecarDBName.Get()) tw.keyspaceShard = "test:0" tw.now = nowFunc - tw.pool.Open(&dbc.App, &dbc.Dba, &dbc.AppDebug) + tw.pool.Open(dbc.AppWithDB(), dbc.DbaWithDB(), dbc.AppDebugWithDB()) return tw } diff --git a/go/vt/vttablet/tabletmanager/action_agent.go b/go/vt/vttablet/tabletmanager/action_agent.go index 173a234534a..79e6dae5600 100644 --- a/go/vt/vttablet/tabletmanager/action_agent.go +++ b/go/vt/vttablet/tabletmanager/action_agent.go @@ -88,7 +88,7 @@ type ActionAgent struct { TopoServer *topo.Server TabletAlias *topodatapb.TabletAlias MysqlDaemon mysqlctl.MysqlDaemon - DBConfigs dbconfigs.DBConfigs + DBConfigs *dbconfigs.DBConfigs BinlogPlayerMap *BinlogPlayerMap // exportStats is set only for production tablet. @@ -213,7 +213,7 @@ func NewActionAgent( mysqld mysqlctl.MysqlDaemon, queryServiceControl tabletserver.Controller, tabletAlias *topodatapb.TabletAlias, - dbcfgs dbconfigs.DBConfigs, + dbcfgs *dbconfigs.DBConfigs, mycnf *mysqlctl.Mycnf, port, gRPCPort int32, ) (agent *ActionAgent, err error) { @@ -248,7 +248,7 @@ func NewActionAgent( // Start the binlog player services, not playing at start. agent.BinlogPlayerMap = NewBinlogPlayerMap(ts, mysqld, func() binlogplayer.VtClient { - return binlogplayer.NewDbClient(&agent.DBConfigs.Filtered) + return binlogplayer.NewDbClient(agent.DBConfigs.FilteredWithDB()) }) // Stop all binlog players upon entering lameduck. servenv.OnTerm(agent.BinlogPlayerMap.StopAllPlayersAndReset) @@ -256,9 +256,9 @@ func NewActionAgent( var mysqlHost string var mysqlPort int32 - if dbcfgs.App.Host != "" { - mysqlHost = dbcfgs.App.Host - mysqlPort = int32(dbcfgs.App.Port) + if appConfig := dbcfgs.AppWithDB(); appConfig.Host != "" { + mysqlHost = appConfig.Host + mysqlPort = int32(appConfig.Port) } else { // Assume unix socket was specified and try to figure out the mysql port // by other means. @@ -337,7 +337,7 @@ func NewTestActionAgent(batchCtx context.Context, ts *topo.Server, tabletAlias * TopoServer: ts, TabletAlias: tabletAlias, MysqlDaemon: mysqlDaemon, - DBConfigs: dbconfigs.DBConfigs{}, + DBConfigs: &dbconfigs.DBConfigs{}, BinlogPlayerMap: nil, History: history.New(historyLength), _healthy: fmt.Errorf("healthcheck not run yet"), @@ -366,7 +366,7 @@ func NewTestActionAgent(batchCtx context.Context, ts *topo.Server, tabletAlias * // NewComboActionAgent creates an agent tailored specifically to run // within the vtcombo binary. It cannot be called concurrently, // as it changes the flags. -func NewComboActionAgent(batchCtx context.Context, ts *topo.Server, tabletAlias *topodatapb.TabletAlias, vtPort, grpcPort int32, queryServiceControl tabletserver.Controller, dbcfgs dbconfigs.DBConfigs, mysqlDaemon mysqlctl.MysqlDaemon, keyspace, shard, dbname, tabletType string) *ActionAgent { +func NewComboActionAgent(batchCtx context.Context, ts *topo.Server, tabletAlias *topodatapb.TabletAlias, vtPort, grpcPort int32, queryServiceControl tabletserver.Controller, dbcfgs *dbconfigs.DBConfigs, mysqlDaemon mysqlctl.MysqlDaemon, keyspace, shard, dbname, tabletType string) *ActionAgent { agent := &ActionAgent{ QueryServiceControl: queryServiceControl, UpdateStream: binlog.NewUpdateStreamControlMock(), @@ -588,23 +588,14 @@ func (agent *ActionAgent) Start(ctx context.Context, mysqlHost string, mysqlPort // Get and fix the dbname if necessary, only for real instances. if !agent.DBConfigs.IsZero() { dbname := topoproto.TabletDbName(agent.initialTablet) - - // Update our DB config to match the info we have in the tablet - if agent.DBConfigs.App.DbName == "" { - agent.DBConfigs.App.DbName = dbname - } - if agent.DBConfigs.Filtered.DbName == "" { - agent.DBConfigs.Filtered.DbName = dbname - } + agent.DBConfigs.DBName.Set(dbname) } // Create and register the RPC services from UpdateStream. // (it needs the dbname, so it has to be delayed up to here, // but it has to be before updateState below that may use it) if initUpdateStream { - cp := agent.DBConfigs.Dba - cp.DbName = agent.DBConfigs.App.DbName - us := binlog.NewUpdateStream(agent.TopoServer, agent.initialTablet.Keyspace, agent.TabletAlias.Cell, &cp, agent.QueryServiceControl.SchemaEngine()) + us := binlog.NewUpdateStream(agent.TopoServer, agent.initialTablet.Keyspace, agent.TabletAlias.Cell, agent.DBConfigs.DbaWithDB(), agent.QueryServiceControl.SchemaEngine()) agent.UpdateStream = us servenv.OnRun(func() { us.RegisterService() diff --git a/go/vt/vttablet/tabletmanager/init_tablet_test.go b/go/vt/vttablet/tabletmanager/init_tablet_test.go index fcba915013b..0cbac8915ee 100644 --- a/go/vt/vttablet/tabletmanager/init_tablet_test.go +++ b/go/vt/vttablet/tabletmanager/init_tablet_test.go @@ -50,7 +50,7 @@ func TestInitTablet(t *testing.T) { TopoServer: ts, TabletAlias: tabletAlias, MysqlDaemon: mysqlDaemon, - DBConfigs: dbconfigs.DBConfigs{}, + DBConfigs: &dbconfigs.DBConfigs{}, BinlogPlayerMap: nil, batchCtx: ctx, History: history.New(historyLength), diff --git a/go/vt/vttablet/tabletserver/controller.go b/go/vt/vttablet/tabletserver/controller.go index 9e8c82cc358..afffe54b661 100644 --- a/go/vt/vttablet/tabletserver/controller.go +++ b/go/vt/vttablet/tabletserver/controller.go @@ -39,7 +39,7 @@ type Controller interface { AddStatusPart() // InitDBConfig sets up the db config vars. - InitDBConfig(querypb.Target, dbconfigs.DBConfigs) error + InitDBConfig(querypb.Target, *dbconfigs.DBConfigs) error // SetServingType transitions the query service to the required serving type. // Returns true if the state of QueryService or the tablet type changed. diff --git a/go/vt/vttablet/tabletserver/messager/engine.go b/go/vt/vttablet/tabletserver/messager/engine.go index 8d8741262ce..b13e81bc8a6 100644 --- a/go/vt/vttablet/tabletserver/messager/engine.go +++ b/go/vt/vttablet/tabletserver/messager/engine.go @@ -47,7 +47,7 @@ type TabletService interface { // Engine is the engine for handling messages. type Engine struct { - dbconfigs dbconfigs.DBConfigs + dbconfigs *dbconfigs.DBConfigs mu sync.Mutex isOpen bool @@ -76,7 +76,7 @@ func NewEngine(tsv TabletService, se *schema.Engine, config tabletenv.TabletConf } // InitDBConfig must be called before Open. -func (me *Engine) InitDBConfig(dbcfgs dbconfigs.DBConfigs) { +func (me *Engine) InitDBConfig(dbcfgs *dbconfigs.DBConfigs) { me.dbconfigs = dbcfgs } @@ -85,7 +85,7 @@ func (me *Engine) Open() error { if me.isOpen { return nil } - me.conns.Open(&me.dbconfigs.App, &me.dbconfigs.Dba, &me.dbconfigs.AppDebug) + me.conns.Open(me.dbconfigs.AppWithDB(), me.dbconfigs.DbaWithDB(), me.dbconfigs.AppDebugWithDB()) me.se.RegisterNotifier("messages", me.schemaChanged) me.isOpen = true return nil diff --git a/go/vt/vttablet/tabletserver/messager/engine_test.go b/go/vt/vttablet/tabletserver/messager/engine_test.go index 44bb13b4df3..9ac4abed76b 100644 --- a/go/vt/vttablet/tabletserver/messager/engine_test.go +++ b/go/vt/vttablet/tabletserver/messager/engine_test.go @@ -261,10 +261,7 @@ func newTestEngine(db *fakesqldb.DB) *Engine { tsv := newFakeTabletServer() se := schema.NewEngine(tsv, config) te := NewEngine(tsv, se, config) - te.InitDBConfig(dbconfigs.DBConfigs{ - App: *db.ConnParams(), - SidecarDBName: "_vt", - }) + te.InitDBConfig(dbconfigs.NewTestDBConfigs(*db.ConnParams(), *db.ConnParams(), "")) te.Open() return te } diff --git a/go/vt/vttablet/tabletserver/messager/message_manager_test.go b/go/vt/vttablet/tabletserver/messager/message_manager_test.go index 2d092b5bac3..08a58ef198b 100644 --- a/go/vt/vttablet/tabletserver/messager/message_manager_test.go +++ b/go/vt/vttablet/tabletserver/messager/message_manager_test.go @@ -785,10 +785,7 @@ func (fts *fakeTabletServer) PurgeMessages(ctx context.Context, target *querypb. func newMMConnPool(db *fakesqldb.DB) *connpool.Pool { pool := connpool.New("", 20, time.Duration(10*time.Minute), newFakeTabletServer()) - dbconfigs := dbconfigs.DBConfigs{ - App: *db.ConnParams(), - SidecarDBName: "_vt", - } - pool.Open(&dbconfigs.App, &dbconfigs.Dba, &dbconfigs.AppDebug) + dbconfigs := dbconfigs.NewTestDBConfigs(*db.ConnParams(), *db.ConnParams(), "") + pool.Open(dbconfigs.AppWithDB(), dbconfigs.DbaWithDB(), dbconfigs.AppDebugWithDB()) return pool } diff --git a/go/vt/vttablet/tabletserver/query_engine.go b/go/vt/vttablet/tabletserver/query_engine.go index 3656c8627a6..dfe4006a5a8 100644 --- a/go/vt/vttablet/tabletserver/query_engine.go +++ b/go/vt/vttablet/tabletserver/query_engine.go @@ -118,7 +118,7 @@ func (ep *TabletPlan) buildAuthorized() { // function is called. type QueryEngine struct { se *schema.Engine - dbconfigs dbconfigs.DBConfigs + dbconfigs *dbconfigs.DBConfigs // mu protects the following fields. mu sync.RWMutex @@ -273,13 +273,13 @@ func NewQueryEngine(checker connpool.MySQLChecker, se *schema.Engine, config tab } // InitDBConfig must be called before Open. -func (qe *QueryEngine) InitDBConfig(dbcfgs dbconfigs.DBConfigs) { +func (qe *QueryEngine) InitDBConfig(dbcfgs *dbconfigs.DBConfigs) { qe.dbconfigs = dbcfgs } // Open must be called before sending requests to QueryEngine. func (qe *QueryEngine) Open() error { - qe.conns.Open(&qe.dbconfigs.App, &qe.dbconfigs.Dba, &qe.dbconfigs.AppDebug) + qe.conns.Open(qe.dbconfigs.AppWithDB(), qe.dbconfigs.DbaWithDB(), qe.dbconfigs.AppDebugWithDB()) conn, err := qe.conns.Get(tabletenv.LocalContext()) if err != nil { @@ -294,7 +294,7 @@ func (qe *QueryEngine) Open() error { return err } - qe.streamConns.Open(&qe.dbconfigs.App, &qe.dbconfigs.Dba, &qe.dbconfigs.AppDebug) + qe.streamConns.Open(qe.dbconfigs.AppWithDB(), qe.dbconfigs.DbaWithDB(), qe.dbconfigs.AppDebugWithDB()) qe.se.RegisterNotifier("qe", qe.schemaChanged) return nil } @@ -428,7 +428,7 @@ func (qe *QueryEngine) ClearQueryPlanCache() { // IsMySQLReachable returns true if we can connect to MySQL. func (qe *QueryEngine) IsMySQLReachable() bool { - conn, err := dbconnpool.NewDBConnection(&qe.dbconfigs.App, tabletenv.MySQLStats) + conn, err := dbconnpool.NewDBConnection(qe.dbconfigs.AppWithDB(), tabletenv.MySQLStats) if err != nil { if mysql.IsConnErr(err) { return false diff --git a/go/vt/vttablet/tabletserver/query_engine_test.go b/go/vt/vttablet/tabletserver/query_engine_test.go index e90f9a8effa..882cbe0dd5f 100644 --- a/go/vt/vttablet/tabletserver/query_engine_test.go +++ b/go/vt/vttablet/tabletserver/query_engine_test.go @@ -54,6 +54,7 @@ func TestStrictTransTables(t *testing.T) { // config.EnforceStrictTransTable is true by default. qe := NewQueryEngine(DummyChecker, schema.NewEngine(DummyChecker, config), config) qe.InitDBConfig(dbcfgs) + qe.se.InitDBConfig(dbcfgs) qe.se.Open() if err := qe.Open(); err != nil { t.Error(err) @@ -291,7 +292,7 @@ func TestStatsURL(t *testing.T) { qe.ServeHTTP(response, request) } -func newTestQueryEngine(queryPlanCacheSize int, idleTimeout time.Duration, strict bool, dbcfgs dbconfigs.DBConfigs) *QueryEngine { +func newTestQueryEngine(queryPlanCacheSize int, idleTimeout time.Duration, strict bool, dbcfgs *dbconfigs.DBConfigs) *QueryEngine { config := tabletenv.DefaultQsConfig config.QueryPlanCacheSize = queryPlanCacheSize config.IdleTimeout = float64(idleTimeout) / 1e9 diff --git a/go/vt/vttablet/tabletserver/queryz_test.go b/go/vt/vttablet/tabletserver/queryz_test.go index eab4c08512f..43cc7d384f3 100644 --- a/go/vt/vttablet/tabletserver/queryz_test.go +++ b/go/vt/vttablet/tabletserver/queryz_test.go @@ -35,7 +35,7 @@ import ( func TestQueryzHandler(t *testing.T) { resp := httptest.NewRecorder() req, _ := http.NewRequest("GET", "/schemaz", nil) - qe := newTestQueryEngine(100, 10*time.Second, true, dbconfigs.DBConfigs{}) + qe := newTestQueryEngine(100, 10*time.Second, true, &dbconfigs.DBConfigs{}) plan1 := &TabletPlan{ Plan: &planbuilder.Plan{ diff --git a/go/vt/vttablet/tabletserver/replication_watcher.go b/go/vt/vttablet/tabletserver/replication_watcher.go index 3c2e6986959..da3bfd9dfaa 100644 --- a/go/vt/vttablet/tabletserver/replication_watcher.go +++ b/go/vt/vttablet/tabletserver/replication_watcher.go @@ -39,7 +39,7 @@ import ( // replication stream. It can tell you the current event token, // and it will trigger schema reloads if a DDL is encountered. type ReplicationWatcher struct { - dbconfigs dbconfigs.DBConfigs + dbconfigs *dbconfigs.DBConfigs // Life cycle management vars isOpen bool @@ -82,7 +82,7 @@ func NewReplicationWatcher(se *schema.Engine, config tabletenv.TabletConfig) *Re } // InitDBConfig must be called before Open. -func (rpw *ReplicationWatcher) InitDBConfig(dbcfgs dbconfigs.DBConfigs) { +func (rpw *ReplicationWatcher) InitDBConfig(dbcfgs *dbconfigs.DBConfigs) { rpw.dbconfigs = dbcfgs } @@ -109,16 +109,14 @@ func (rpw *ReplicationWatcher) Close() { } // Process processes the replication stream. -func (rpw *ReplicationWatcher) Process(ctx context.Context, dbconfigs dbconfigs.DBConfigs) { +func (rpw *ReplicationWatcher) Process(ctx context.Context, dbconfigs *dbconfigs.DBConfigs) { defer func() { tabletenv.LogError() rpw.wg.Done() }() for { log.Infof("Starting a binlog Streamer from current replication position to monitor binlogs") - cp := dbconfigs.Dba - cp.DbName = dbconfigs.App.DbName - streamer := binlog.NewStreamer(&cp, rpw.se, nil /*clientCharset*/, mysql.Position{}, 0 /*timestamp*/, func(eventToken *querypb.EventToken, statements []binlog.FullBinlogStatement) error { + streamer := binlog.NewStreamer(dbconfigs.DbaWithDB(), rpw.se, nil /*clientCharset*/, mysql.Position{}, 0 /*timestamp*/, func(eventToken *querypb.EventToken, statements []binlog.FullBinlogStatement) error { // Save the event token. rpw.mu.Lock() rpw.eventToken = eventToken diff --git a/go/vt/vttablet/tabletserver/schema/engine.go b/go/vt/vttablet/tabletserver/schema/engine.go index 944e640227e..91a588791a7 100644 --- a/go/vt/vttablet/tabletserver/schema/engine.go +++ b/go/vt/vttablet/tabletserver/schema/engine.go @@ -49,7 +49,7 @@ type notifier func(full map[string]*Table, created, altered, dropped []string) // Engine stores the schema info and performs operations that // keep itself up-to-date. type Engine struct { - dbconfigs dbconfigs.DBConfigs + dbconfigs *dbconfigs.DBConfigs // mu protects the following fields. mu sync.Mutex @@ -93,7 +93,7 @@ func NewEngine(checker connpool.MySQLChecker, config tabletenv.TabletConfig) *En } // InitDBConfig must be called before Open. -func (se *Engine) InitDBConfig(dbcfgs dbconfigs.DBConfigs) { +func (se *Engine) InitDBConfig(dbcfgs *dbconfigs.DBConfigs) { se.dbconfigs = dbcfgs } @@ -108,7 +108,7 @@ func (se *Engine) Open() error { start := time.Now() defer log.Infof("Time taken to load the schema: %v", time.Now().Sub(start)) ctx := tabletenv.LocalContext() - dbaParams := &se.dbconfigs.Dba + dbaParams := se.dbconfigs.DbaWithDB() se.conns.Open(dbaParams, dbaParams, dbaParams) conn, err := se.conns.Get(ctx) diff --git a/go/vt/vttablet/tabletserver/schema/engine_test.go b/go/vt/vttablet/tabletserver/schema/engine_test.go index c31aeb18bb0..2d535f3d0d8 100644 --- a/go/vt/vttablet/tabletserver/schema/engine_test.go +++ b/go/vt/vttablet/tabletserver/schema/engine_test.go @@ -433,10 +433,6 @@ func newEngine(queryPlanCacheSize int, reloadTime time.Duration, idleTimeout tim return se } -func newDBConfigs(db *fakesqldb.DB) dbconfigs.DBConfigs { - return dbconfigs.DBConfigs{ - App: *db.ConnParams(), - Dba: *db.ConnParams(), - SidecarDBName: "_vt", - } +func newDBConfigs(db *fakesqldb.DB) *dbconfigs.DBConfigs { + return dbconfigs.NewTestDBConfigs(*db.ConnParams(), *db.ConnParams(), "") } diff --git a/go/vt/vttablet/tabletserver/tabletserver.go b/go/vt/vttablet/tabletserver/tabletserver.go index bc6d5e16c0c..1441cb0f981 100644 --- a/go/vt/vttablet/tabletserver/tabletserver.go +++ b/go/vt/vttablet/tabletserver/tabletserver.go @@ -155,7 +155,7 @@ type TabletServer struct { // The following variables should be initialized only once // before starting the tabletserver. - dbconfigs dbconfigs.DBConfigs + dbconfigs *dbconfigs.DBConfigs // The following variables should only be accessed within // the context of a startRequest-endRequest. @@ -328,7 +328,7 @@ func (tsv *TabletServer) IsServing() bool { // InitDBConfig inititalizes the db config variables for TabletServer. You must call this function before // calling SetServingType. -func (tsv *TabletServer) InitDBConfig(target querypb.Target, dbcfgs dbconfigs.DBConfigs) error { +func (tsv *TabletServer) InitDBConfig(target querypb.Target, dbcfgs *dbconfigs.DBConfigs) error { tsv.mu.Lock() defer tsv.mu.Unlock() if tsv.state != StateNotConnected { @@ -336,13 +336,6 @@ func (tsv *TabletServer) InitDBConfig(target querypb.Target, dbcfgs dbconfigs.DB } tsv.target = target tsv.dbconfigs = dbcfgs - // Massage Dba so that it inherits the - // App values but keeps the credentials. - tsv.dbconfigs.Dba = dbcfgs.App - if n, p := dbcfgs.Dba.Uname, dbcfgs.Dba.Pass; n != "" { - tsv.dbconfigs.Dba.Uname = n - tsv.dbconfigs.Dba.Pass = p - } tsv.se.InitDBConfig(tsv.dbconfigs) tsv.qe.InitDBConfig(tsv.dbconfigs) @@ -356,7 +349,7 @@ func (tsv *TabletServer) InitDBConfig(target querypb.Target, dbcfgs dbconfigs.DB // StartService is a convenience function for InitDBConfig->SetServingType // with serving=true. -func (tsv *TabletServer) StartService(target querypb.Target, dbcfgs dbconfigs.DBConfigs) (err error) { +func (tsv *TabletServer) StartService(target querypb.Target, dbcfgs *dbconfigs.DBConfigs) (err error) { // Save tablet type away to prevent data races tabletType := target.TabletType err = tsv.InitDBConfig(target, dbcfgs) @@ -465,7 +458,7 @@ func (tsv *TabletServer) decideAction(tabletType topodatapb.TabletType, serving } func (tsv *TabletServer) fullStart() (err error) { - c, err := dbconnpool.NewDBConnection(&tsv.dbconfigs.App, tabletenv.MySQLStats) + c, err := dbconnpool.NewDBConnection(tsv.dbconfigs.AppWithDB(), tabletenv.MySQLStats) if err != nil { log.Errorf("error creating db app connection: %v", err) return err @@ -1796,9 +1789,7 @@ func (tsv *TabletServer) UpdateStream(ctx context.Context, target *querypb.Targe } defer tsv.endRequest(false) - cp := tsv.dbconfigs.Dba - cp.DbName = tsv.dbconfigs.App.DbName - s := binlog.NewEventStreamer(&cp, tsv.se, p, timestamp, callback) + s := binlog.NewEventStreamer(tsv.dbconfigs.DbaWithDB(), tsv.se, p, timestamp, callback) // Create a cancelable wrapping context. streamCtx, streamCancel := context.WithCancel(ctx) diff --git a/go/vt/vttablet/tabletserver/testutils_test.go b/go/vt/vttablet/tabletserver/testutils_test.go index 3043f865b45..54259c67c08 100644 --- a/go/vt/vttablet/tabletserver/testutils_test.go +++ b/go/vt/vttablet/tabletserver/testutils_test.go @@ -49,12 +49,8 @@ func (util *testUtils) checkEqual(t *testing.T, expected interface{}, result int } } -func (util *testUtils) newDBConfigs(db *fakesqldb.DB) dbconfigs.DBConfigs { - return dbconfigs.DBConfigs{ - App: *db.ConnParams(), - Dba: *db.ConnParams(), - SidecarDBName: "_vt", - } +func (util *testUtils) newDBConfigs(db *fakesqldb.DB) *dbconfigs.DBConfigs { + return dbconfigs.NewTestDBConfigs(*db.ConnParams(), *db.ConnParams(), "") } func (util *testUtils) newQueryServiceConfig() tabletenv.TabletConfig { diff --git a/go/vt/vttablet/tabletserver/twopc.go b/go/vt/vttablet/tabletserver/twopc.go index f1d67496e8d..b707d41b6c2 100644 --- a/go/vt/vttablet/tabletserver/twopc.go +++ b/go/vt/vttablet/tabletserver/twopc.go @@ -201,8 +201,8 @@ func (tpc *TwoPC) Init(sidecarDBName string, dbaparams *mysql.ConnParams) error } // Open starts the TwoPC service. -func (tpc *TwoPC) Open(dbconfigs dbconfigs.DBConfigs) { - tpc.readPool.Open(&dbconfigs.App, &dbconfigs.Dba, &dbconfigs.AppDebug) +func (tpc *TwoPC) Open(dbconfigs *dbconfigs.DBConfigs) { + tpc.readPool.Open(dbconfigs.AppWithDB(), dbconfigs.DbaWithDB(), dbconfigs.DbaWithDB()) } // Close closes the TwoPC service. diff --git a/go/vt/vttablet/tabletserver/tx_engine.go b/go/vt/vttablet/tabletserver/tx_engine.go index 87d10453843..17f0405fb82 100644 --- a/go/vt/vttablet/tabletserver/tx_engine.go +++ b/go/vt/vttablet/tabletserver/tx_engine.go @@ -37,7 +37,7 @@ import ( // TxEngine handles transactions. type TxEngine struct { - dbconfigs dbconfigs.DBConfigs + dbconfigs *dbconfigs.DBConfigs isOpen, twopcEnabled bool shutdownGracePeriod time.Duration @@ -107,7 +107,7 @@ func NewTxEngine(checker connpool.MySQLChecker, config tabletenv.TabletConfig) * } // InitDBConfig must be called before Init. -func (te *TxEngine) InitDBConfig(dbcfgs dbconfigs.DBConfigs) { +func (te *TxEngine) InitDBConfig(dbcfgs *dbconfigs.DBConfigs) { te.dbconfigs = dbcfgs } @@ -115,7 +115,7 @@ func (te *TxEngine) InitDBConfig(dbcfgs dbconfigs.DBConfigs) { // up the metadata tables. func (te *TxEngine) Init() error { if te.twopcEnabled { - return te.twoPC.Init(te.dbconfigs.SidecarDBName, &te.dbconfigs.Dba) + return te.twoPC.Init(te.dbconfigs.SidecarDBName.Get(), te.dbconfigs.DbaWithDB()) } return nil } @@ -126,7 +126,7 @@ func (te *TxEngine) Open() { if te.isOpen { return } - te.txPool.Open(&te.dbconfigs.App, &te.dbconfigs.Dba, &te.dbconfigs.AppDebug) + te.txPool.Open(te.dbconfigs.AppWithDB(), te.dbconfigs.DbaWithDB(), te.dbconfigs.AppDebugWithDB()) if !te.twopcEnabled { te.isOpen = true return diff --git a/go/vt/vttablet/tabletservermock/controller.go b/go/vt/vttablet/tabletservermock/controller.go index b3e1f309ce0..d5e906240d3 100644 --- a/go/vt/vttablet/tabletservermock/controller.go +++ b/go/vt/vttablet/tabletservermock/controller.go @@ -109,7 +109,7 @@ func (tqsc *Controller) AddStatusPart() { } // InitDBConfig is part of the tabletserver.Controller interface -func (tqsc *Controller) InitDBConfig(target querypb.Target, dbcfgs dbconfigs.DBConfigs) error { +func (tqsc *Controller) InitDBConfig(target querypb.Target, dbcfgs *dbconfigs.DBConfigs) error { tqsc.mu.Lock() defer tqsc.mu.Unlock() diff --git a/go/vt/vttest/mysqlctl.go b/go/vt/vttest/mysqlctl.go index aeeb0af639c..d7e135dbfa4 100644 --- a/go/vt/vttest/mysqlctl.go +++ b/go/vt/vttest/mysqlctl.go @@ -60,8 +60,6 @@ func (ctl *Mysqlctl) Setup() error { "-alsologtostderr", "-tablet_uid", "1", "-mysql_port", fmt.Sprintf("%d", ctl.Port), - "-db-config-dba-charset", "utf8", - "-db-config-dba-uname", "vt_dba", "init", "-init_db_sql_file", ctl.InitFile, ) @@ -86,8 +84,6 @@ func (ctl *Mysqlctl) TearDown() error { "-alsologtostderr", "-tablet_uid", "1", "-mysql_port", fmt.Sprintf("%d", ctl.Port), - "-db-config-dba-charset", "utf8", - "-db-config-dba-uname", "vt_dba", "shutdown", ) diff --git a/go/vt/vttest/vtprocess.go b/go/vt/vttest/vtprocess.go index 9f43ee13393..af5901edabd 100644 --- a/go/vt/vttest/vtprocess.go +++ b/go/vt/vttest/vtprocess.go @@ -208,12 +208,11 @@ func VtcomboProcess(env Environment, args *Config, mysql MySQLManager) *VtProces } vt.ExtraArgs = append(vt.ExtraArgs, []string{ - "-db-config-app-charset", charset, - "-db-config-app-uname", user, - "-db-config-app-pass", pass, - "-db-config-dba-charset", charset, - "-db-config-dba-uname", user, - "-db-config-dba-pass", pass, + "-db_charset", charset, + "-db_app_user", user, + "-db_app_password", pass, + "-db_dba_user", user, + "-db_dba_password", pass, "-proto_topo", proto.CompactTextString(args.Topology), "-mycnf_server_id", "1", "-mycnf_socket_file", socket, @@ -235,18 +234,15 @@ func VtcomboProcess(env Environment, args *Config, mysql MySQLManager) *VtProces if socket != "" { vt.ExtraArgs = append(vt.ExtraArgs, []string{ - "-db-config-app-unixsocket", socket, - "-db-config-dba-unixsocket", socket, + "-db_socket", socket, }...) } else { hostname, p := mysql.Address() port := fmt.Sprintf("%d", p) vt.ExtraArgs = append(vt.ExtraArgs, []string{ - "-db-config-app-host", hostname, - "-db-config-app-port", port, - "-db-config-dba-host", hostname, - "-db-config-dba-port", port, + "-db_host", hostname, + "-db_port", port, }...) } diff --git a/helm/vitess/templates/_vttablet.tpl b/helm/vitess/templates/_vttablet.tpl index 751b110b225..04edcedde9f 100644 --- a/helm/vitess/templates/_vttablet.tpl +++ b/helm/vitess/templates/_vttablet.tpl @@ -402,18 +402,6 @@ spec: -init_tablet_type {{ $tablet.type | quote }} -health_check_interval "5s" -mysqlctl_socket "/vtdataroot/mysqlctl.sock" - -db-config-app-uname "vt_app" - -db-config-app-dbname "vt_{{$keyspace.name}}" - -db-config-app-charset "utf8" - -db-config-dba-uname "vt_dba" - -db-config-dba-dbname "vt_{{$keyspace.name}}" - -db-config-dba-charset "utf8" - -db-config-repl-uname "vt_repl" - -db-config-repl-dbname "vt_{{$keyspace.name}}" - -db-config-repl-charset "utf8" - -db-config-filtered-uname "vt_filtered" - -db-config-filtered-dbname "vt_{{$keyspace.name}}" - -db-config-filtered-charset "utf8" -enable_replication_reporter {{ if $defaultVttablet.enableSemisync }} -enable_semi_sync @@ -485,8 +473,6 @@ spec: -tablet_dir "tabletdata" -tablet_uid "{{$uid}}" -socket_file "/vtdataroot/mysqlctl.sock" - -db-config-dba-uname "vt_dba" - -db-config-dba-charset "utf8" -init_db_sql_file "/vt/config/init_db.sql" END_OF_COMMAND diff --git a/py/vttest/mysql_db_mysqlctl.py b/py/vttest/mysql_db_mysqlctl.py index a48e4851982..50e7cc58d27 100644 --- a/py/vttest/mysql_db_mysqlctl.py +++ b/py/vttest/mysql_db_mysqlctl.py @@ -38,8 +38,6 @@ def setup(self): '-alsologtostderr', '-tablet_uid', '1', '-mysql_port', str(self._port), - '-db-config-dba-charset', 'utf8', - '-db-config-dba-uname', 'vt_dba', 'init', '-init_db_sql_file', os.path.join(os.environ['VTTOP'], 'config/init_db.sql'), @@ -60,8 +58,6 @@ def teardown(self): '-alsologtostderr', '-tablet_uid', '1', '-mysql_port', str(self._port), - '-db-config-dba-charset', 'utf8', - '-db-config-dba-uname', 'vt_dba', 'shutdown', ] result = subprocess.call(cmd) diff --git a/py/vttest/vt_processes.py b/py/vttest/vt_processes.py index 6c6fe94e741..7dccde5ec06 100644 --- a/py/vttest/vt_processes.py +++ b/py/vttest/vt_processes.py @@ -141,12 +141,11 @@ def __init__(self, directory, topology, mysql_db, schema_dir, charset, VtProcess.__init__(self, 'vtcombo-%s' % os.environ['USER'], directory, environment.vtcombo_binary, port_name='vtcombo') self.extraparams = [ - '-db-config-app-charset', charset, - '-db-config-app-uname', mysql_db.username(), - '-db-config-app-pass', mysql_db.password(), - '-db-config-dba-charset', charset, - '-db-config-dba-uname', mysql_db.username(), - '-db-config-dba-pass', mysql_db.password(), + '-db_charset', charset, + '-db_app_user', mysql_db.username(), + '-db_app_password', mysql_db.password(), + '-db_dba_user', mysql_db.username(), + '-db_dba_password', mysql_db.password(), '-proto_topo', text_format.MessageToString(topology, as_one_line=True), '-mycnf_server_id', '1', '-mycnf_socket_file', mysql_db.unix_socket(), @@ -159,15 +158,11 @@ def __init__(self, directory, topology, mysql_db, schema_dir, charset, if web_dir2: self.extraparams.extend(['-web_dir2', web_dir2]) if mysql_db.unix_socket(): - self.extraparams.extend( - ['-db-config-app-unixsocket', mysql_db.unix_socket(), - '-db-config-dba-unixsocket', mysql_db.unix_socket()]) + self.extraparams.extend(['-db_socket', mysql_db.unix_socket()]) else: self.extraparams.extend( - ['-db-config-app-host', mysql_db.hostname(), - '-db-config-app-port', str(mysql_db.port()), - '-db-config-dba-host', mysql_db.hostname(), - '-db-config-dba-port', str(mysql_db.port())]) + ['-db_host', mysql_db.hostname(), + '-db_port', str(mysql_db.port())]) self.vtcombo_mysql_port = environment.get_port('vtcombo_mysql_port') self.extraparams.extend( ['-mysql_auth_server_impl', 'none', diff --git a/test/tablet.py b/test/tablet.py index 5b06a4db2e5..8c08738dfc7 100644 --- a/test/tablet.py +++ b/test/tablet.py @@ -69,34 +69,6 @@ class Tablet(object): default_uid = 62344 seq = 0 tablets_running = 0 - default_db_dba_config = { - 'dba': { - 'uname': 'vt_dba', - 'charset': 'utf8' - }, - } - default_db_config = { - 'app': { - 'uname': 'vt_app', - 'charset': 'utf8' - }, - 'allprivs': { - 'uname': 'vt_allprivs', - 'charset': 'utf8' - }, - 'dba': { - 'uname': 'vt_dba', - 'charset': 'utf8' - }, - 'filtered': { - 'uname': 'vt_filtered', - 'charset': 'utf8' - }, - 'repl': { - 'uname': 'vt_repl', - 'charset': 'utf8' - } - } def __init__(self, tablet_uid=None, port=None, mysql_port=None, cell=None, use_mysqlctld=False, vt_dba_passwd=None): @@ -158,7 +130,6 @@ def mysqlctl(self, cmd, extra_my_cnf=None, with_ports=False, verbose=False, if with_ports: args.extend(['-port', str(self.port), '-mysql_port', str(self.mysql_port)]) - self._add_dbconfigs(self.default_db_dba_config, args) if verbose: args.append('-alsologtostderr') if extra_args: @@ -186,7 +157,6 @@ def mysqlctld(self, cmd, extra_my_cnf=None, verbose=False, extra_args=None): '-tablet_uid', str(self.tablet_uid), '-mysql_port', str(self.mysql_port), '-socket_file', os.path.join(self.tablet_dir, 'mysqlctl.sock')] - self._add_dbconfigs(self.default_db_dba_config, args) if verbose: args.append('-alsologtostderr') if extra_args: @@ -587,8 +557,6 @@ def start_vttablet( args.extend(['-port', '%s' % (port or self.port), '-log_dir', environment.vtlogroot]) - self._add_dbconfigs(self.default_db_config, args, repl_extra_flags) - if topocustomrule_path: args.extend(['-topocustomrule_path', topocustomrule_path]) @@ -688,22 +656,6 @@ def wait_for_mysqlctl_socket(self, timeout=60.0): timeout = utils.wait_step('waiting for socket files: %s' % str(wait_for), timeout, sleep_time=2.0) - def _add_dbconfigs(self, cfg, args, repl_extra_flags=None): - """Helper method to generate and add --db-config-* flags to 'args'.""" - if repl_extra_flags is None: - repl_extra_flags = {} - config = dict(cfg) - if self.keyspace: - if 'app' in config: - config['app']['dbname'] = self.dbname - if 'repl' in config: - config['repl']['dbname'] = self.dbname - if 'repl' in config: - config['repl'].update(repl_extra_flags) - for key1 in config: - for key2 in config[key1]: - args.extend(['-db-config-' + key1 + '-' + key2, config[key1][key2]]) - def get_status(self): return utils.get_status(self.port) diff --git a/test/tabletmanager.py b/test/tabletmanager.py index b5c84d1dbd1..1ce2c27b571 100755 --- a/test/tabletmanager.py +++ b/test/tabletmanager.py @@ -14,9 +14,11 @@ # See the License for the specific language governing permissions and # limitations under the License. +# vim: tabstop=8 expandtab shiftwidth=2 softtabstop=2 import json import logging +import os import time import unittest import urllib @@ -172,6 +174,25 @@ def _test_sanity(self): "insert into vt_select_test (msg) values ('test %s')" % x for x in xrange(4)] + # Test if a vttablet can be pointed at an existing mysql + # We point 62044 at 62344's mysql and try to read from it. + def test_command_line(self): + utils.run_vtctl(['CreateKeyspace', 'test_keyspace']) + tablet_62044.init_tablet('master', 'test_keyspace', '0') + tablet_62344.populate('vt_test_keyspace', self._create_vt_select_test, + self._populate_vt_select_test) + + # mycnf_server_id prevents vttablet from reading the mycnf + extra_args = [ + '-mycnf_server_id', str(tablet_62044.tablet_uid), + '-db_socket', os.path.join(tablet_62344.tablet_dir, 'mysql.sock')] + # supports_backup=False prevents vttablet from trying to restore + tablet_62044.start_vttablet(extra_args=extra_args, supports_backups=False) + qr = tablet_62044.execute('select id, msg from vt_select_test') + self.assertEqual(len(qr['rows']), 4, + 'expected 4 rows in vt_select_test: %s' % str(qr)) + tablet_62044.kill_vttablet() + def test_actions_and_timeouts(self): # Start up a master mysql and vttablet utils.run_vtctl(['CreateKeyspace', 'test_keyspace'])