diff --git a/content/en/docs/concepts/topology-service.md b/content/en/docs/concepts/topology-service.md index c71f84d25..679a698d2 100644 --- a/content/en/docs/concepts/topology-service.md +++ b/content/en/docs/concepts/topology-service.md @@ -17,9 +17,9 @@ A Vitess cluster has one global topology service, and a local topology service i ## Global Topology -The global topology stores Vitess-wide data that does not change frequently. Specifically, it contains data about keyspaces and shards as well as the master tablet alias for each shard. +The global topology service stores Vitess-wide data that does not change frequently. Specifically, it contains data about keyspaces and shards as well as the master tablet alias for each shard. -The global topology is used for some operations, including reparenting and resharding. By design, the global topology server is not used a lot. +The global topology is used for some operations, including reparenting and resharding. By design, the global topology service is not used a lot. In order to survive any single cell going down, the global topology service should have nodes in multiple cells, with enough to maintain quorum in the event of a cell failure. diff --git a/content/en/docs/reference/schema-management.md b/content/en/docs/reference/schema-management.md index 0f7a684b7..e6139030f 100644 --- a/content/en/docs/reference/schema-management.md +++ b/content/en/docs/reference/schema-management.md @@ -10,7 +10,7 @@ Using Vitess requires you to work with two different types of schemas: The workflow for the `VSchema` is as follows: -1. Apply the `VSchema` for each keyspace using the `ApplyVschema` command. This saves the VSchemas in the global topo server. +1. Apply the `VSchema` for each keyspace using the `ApplyVschema` command. This saves the VSchemas in the global topology service. 2. Execute `RebuildVSchemaGraph` for each cell (or all cells). This command propagates a denormalized version of the combined VSchema to all the specified cells. The main purpose for this propagation is to minimize the dependency of each cell from the global topology. The ability to push a change to only specific cells allows you to canary the change to make sure that it's good before deploying it everywhere. This document describes the [`vtctl`](../../reference/vtctl/) commands that you can use to [review](#reviewing-your-schema) or [update](#changing-your-schema) your schema in Vitess. @@ -84,7 +84,7 @@ Vitess' schema modification functionality is designed the following goals in min * Require minimal human interaction. * Minimize errors by testing changes against a temporary database. * Guarantee very little downtime (or no downtime) for most schema updates. -* Do not store permanent schema data in the topology server. +* Do not store permanent schema data in the topology service. Note that, at this time, Vitess only supports [data definition statements](https://dev.mysql.com/doc/refman/5.6/en/sql-syntax-data-definition.html) that create, modify, or delete database tables. For instance, `ApplySchema` does not affect stored procedures or grants. diff --git a/content/en/docs/user-guides/topology-service.md b/content/en/docs/reference/topology-service.md similarity index 91% rename from content/en/docs/user-guides/topology-service.md rename to content/en/docs/reference/topology-service.md index 138e4e307..13b0a5a80 100644 --- a/content/en/docs/user-guides/topology-service.md +++ b/content/en/docs/reference/topology-service.md @@ -1,18 +1,11 @@ --- title: Topology Service -weight: 6 +aliases: ['/docs/user-guides/topology-service/'] --- -This document describes the Topology Service, a key part of the Vitess -architecture. This service is exposed to all Vitess processes, and is used to -store small pieces of configuration data about the Vitess cluster, and provide -cluster-wide locks. It also supports watches, and master election. +This document describes the Topology Service, a key part of the Vitess architecture. This service is exposed to all Vitess processes, and is used to store small pieces of configuration data about the Vitess cluster, and provide cluster-wide locks. It also supports watches, and master election. -Concretely, the Topology Service features are implemented by -a [Lock Server](http://en.wikipedia.org/wiki/Distributed_lock_manager), referred -to as Topology Server in Vitess. We use a plugin implementation and we -support multiple Lock Servers (Zookeeper, etcd, Consul, …) as backends for the -service. +Vitess uses a plugin implementation to support multiple backend technologies for the Topology Service (etcd, ZooKeeper, Consul). Concretely, the Topology Service handles two functions: it is both a [distributed lock manager](http://en.wikipedia.org/wiki/Distributed_lock_manager) and a repository for topology metadata. In earlier versions of Vitess, the Topology Serice was also referred to as the Lock Service. ## Requirements and usage @@ -20,21 +13,21 @@ The Topology Service is used to store information about the Keyspaces, the Shards, the Tablets, the Replication Graph, and the Serving Graph. We store small data structures (a few hundred bytes) per object. -The main contract for the Topology Server is to be very highly available and +The main contract for the Topology Service is to be very highly available and consistent. It is understood it will come at a higher latency cost and very low throughput. -We never use the Topology Server as an RPC or queuing mechanism or as a storage -system for logs. We never depend on the Topology Server being responsive and +We never use the Topology Service as an RPC or queuing mechanism or as a storage +system for logs. We never depend on the Topology Service being responsive and fast to serve every query. -The Topology Server must also support a Watch interface, to signal when certain +The Topology Service must also support a Watch interface, to signal when certain conditions occur on a node. This is used, for instance, to know when the Keyspace topology changes (e.g. for resharding). ### Global vs Local -We differentiate two instances of the Topology Server: the Global instance, and +We differentiate two instances of the Topology Service: the Global instance, and the per-cell Local instance: * The Global instance is used to store global data about the topology that @@ -52,25 +45,22 @@ exception to that is if a reparent needs to be processed, it might not work). If a Local instance goes down, it only affects the local tablets in that instance (and then the cell is usually in bad shape, and should not be used). -Furthermore, the Vitess processes will not use the Global nor the Local Topology -Server to serve individual queries. They only use the Topology Server to get the -topology information at startup and in the background, but never to directly -serve queries. +Vitess will not use the global or local topology service as part of serving individual queries. The Topology Service is only used to get the topology information at startup and in the background. ### Recovery -If a Local Topology Server dies and is not recoverable, it can be wiped out. All +If a Local Topology Service dies and is not recoverable, it can be wiped out. All the tablets in that cell then need to be restarted so they re-initialize their topology records (but they won’t lose any MySQL data). -If the Global Topology Server dies and is not recoverable, this is more of a +If the Global Topology Service dies and is not recoverable, this is more of a problem. All the Keyspace / Shard objects have to be recreated or be restored. Then the cells should recover. ## Global data This section describes the data structures stored in the Global instance of the -topology server. +topology service. ### Keyspace @@ -85,7 +75,7 @@ guarantee only one operation changes the Keyspace data concurrently. ### Shard A Shard contains a subset of the data for a Keyspace. The Shard record in the -Global topology server contains: +Global topology service contains: * the Master tablet alias for this shard (that has the MySQL master). * the sharding key range covered by this Shard inside the Keyspace. @@ -109,7 +99,7 @@ the [VTGate V3](https://github.com/vitessio/vitess/blob/master/doc/VTGateV3Featu ## Local data This section describes the data structures stored in the Local instance (per -cell) of the topology server. +cell) of the topology service. ### Tablets @@ -170,9 +160,9 @@ keyspaces in a single object. It can be rebuilt by running `vtctl RebuildVSchemaGraph`. It is automatically rebuilt when using `vtctl ApplyVSchema` (unless prevented by flags). -## Workflows involving the Topology Server +## Workflows involving the Topology Service -The Topology Server is involved in many Vitess workflows. +The Topology Service is involved in many Vitess workflows. When a Tablet is initialized, we create the Tablet record, and add the Tablet to the Replication Graph. If it is the master for a Shard, we update the global @@ -201,7 +191,7 @@ will change the global Shard records, and the local SrvKeyspace records. A vertical split will change the global Keyspace records, and the local SrvKeyspace records. -## Exploring the data in a Topology Server +## Exploring the data in a Topology Service We store the proto3 serialized binary data for each object. @@ -239,7 +229,7 @@ tab on the left side). It will display the various proto files, decoded. ## Implementations -The Topology Server interfaces are defined in our code in `go/vt/topo/`, +The Topology Service interfaces are defined in our code in `go/vt/topo/`, specific implementations are in `go/vt/topo/`, and we also have a set of unit tests for it in `go/vt/topo/test`. @@ -281,7 +271,7 @@ want to use servers `global_server1,global_server2` in path `/vitess/global`: # -topo_global_root /vitess/global ``` -Then to add a cell whose local topology servers `cell1_server1,cell1_server2` +Then to add a cell whose local topology service `cell1_server1,cell1_server2` will store their data under the directory `/vitess/cell1`: ``` sh @@ -342,7 +332,7 @@ want to use servers `http://global_server1,http://global_server2` in path TOPOLOGY="-topo_implementation etcd2 -topo_global_server_address http://global_server1,http://global_server2 -topo_global_root /vitess/global ``` -Then to add a cell whose local topology servers +Then to add a cell whose local topology service `http://cell1_server1,http://cell1_server2` will store their data under the directory `/vitess/cell1`: @@ -374,7 +364,9 @@ We store the proto3 binary data for each object (as the v3 API allows us to stor binary data). Note that this means that if you want to interact with etcd using the `etcdctl` tool, you will have to tell it to use the v3 API, e.g.: -```ETCDCTL_API=3 etcdctl get / --prefix --keys-only``` +``` +ETCDCTL_API=3 etcdctl get / --prefix --keys-only +``` ### Consul `consul` implementation @@ -396,7 +388,7 @@ want to use servers `global_server:global_port` with node path TOPOLOGY="-topo_implementation consul -topo_global_server_address global_server:global_port -topo_global_root vitess/global ``` -Then to add a cell whose local topology server +Then to add a cell whose local topology service `cell1_server1:cell1_port` will store their data under the directory `vitess/cell1`: @@ -440,7 +432,7 @@ the same server address and, very importantly, a *different* root node path. In that case, just running 3 servers for topology service quorum is probably sufficient. For instance, 3 etcd servers. And use their address for the local cell as well. Let's use a short cell name, like `local`, as the local data in -that topology server will later on be moved to a different topology service, +that topology service will later on be moved to a different topology service, which will have the real cell name. ### Extending to more cells diff --git a/content/en/docs/reference/vtctl.md b/content/en/docs/reference/vtctl.md index dc99518b7..d64ec463d 100644 --- a/content/en/docs/reference/vtctl.md +++ b/content/en/docs/reference/vtctl.md @@ -43,8 +43,8 @@ Registers a local topology service in a new cell by creating the CellInfo with t | Name | Type | Definition | | :-------- | :--------- | :--------- | -| root | string | The root path the topology server is using for that cell. | -| server_address | string | The address the topology server is using for that cell. | +| root | string | The root path the topology service is using for that cell. | +| server_address | string | The address the topology service is using for that cell. | #### Arguments @@ -108,8 +108,8 @@ Updates the content of a CellInfo with the provided parameters. If a value is em | Name | Type | Definition | | :-------- | :--------- | :--------- | -| root | string | The root path the topology server is using for that cell. | -| server_address | string | The address the topology server is using for that cell. | +| root | string | The root path the topology service is using for that cell. | +| server_address | string | The address the topology service is using for that cell. | #### Arguments @@ -428,7 +428,7 @@ Removes the cell from the Cells list for all shards in the keyspace, and the Srv | Name | Type | Definition | | :-------- | :--------- | :--------- | -| force | Boolean | Proceeds even if the cell's topology server cannot be reached. The assumption is that you turned down the entire cell, and just need to update the global topo data. | +| force | Boolean | Proceeds even if the cell's topology service cannot be reached. The assumption is that you turned down the entire cell, and just need to update the global topo data. | | recursive | Boolean | Also delete all tablets in that cell belonging to the specified keyspace. | @@ -1716,7 +1716,7 @@ Removes the cell from the shard's Cells list. | Name | Type | Definition | | :-------- | :--------- | :--------- | -| force | Boolean | Proceeds even if the cell's topology server cannot be reached. The assumption is that you turned down the entire cell, and just need to update the global topo data. | +| force | Boolean | Proceeds even if the cell's topology service cannot be reached. The assumption is that you turned down the entire cell, and just need to update the global topo data. | | recursive | Boolean | Also delete all tablets in that cell belonging to the specified shard. | @@ -1896,7 +1896,7 @@ Deletes the SourceShard record with the provided index. This is meant as an emer ### TabletExternallyReparented -Changes metadata in the topology server to acknowledge a shard master change performed by an external tool. See [Reparenting](../../user-guides/reparenting/#external-reparenting) for more information. +Changes metadata in the topology service to acknowledge a shard master change performed by an external tool. See [Reparenting](../../user-guides/reparenting/#external-reparenting) for more information. #### Example diff --git a/content/en/docs/reference/vttablet-modes.md b/content/en/docs/reference/vttablet-modes.md index 051e0feb5..087f30995 100644 --- a/content/en/docs/reference/vttablet-modes.md +++ b/content/en/docs/reference/vttablet-modes.md @@ -66,7 +66,7 @@ $TOPOLOGY_FLAGS `$alias` needs to be of the form: `-id`, and the cell should match one of the local cells that was created in the topology. The id can be left padded with zeroes: `cell-100` and `cell-000000100` are synonymous. -Example `TOPOLOGY_FLAGS` for a lockserver like zookeeper: +Example `TOPOLOGY_FLAGS` for a Topology Service like zookeeper: `-topo_implementation zk2 -topo_global_server_address localhost:21811,localhost:21812,localhost:21813 -topo_global_root /vitess/global` diff --git a/content/en/docs/resources/roadmap.md b/content/en/docs/resources/roadmap.md index 209767da6..7b3834555 100644 --- a/content/en/docs/resources/roadmap.md +++ b/content/en/docs/resources/roadmap.md @@ -29,5 +29,5 @@ If you have a specific question about the Roadmap, we recommend posting in our [ - Support for Schema Changes - Backfill lookup indexes - Support for Data Migration -- Topo Server: Reduce dependencies on the topo server. i.e. Vitess should be operable normally even if topo server is down for several hours. Topo server should be used only for passive discovery. +- Topology Service: Reduce dependencies on the topology service. i.e. Vitess should be operable normally even if topology service is down for several hours. Topology service should be used only for passive discovery. - Support for PostgreSQL: Vitess should be able to support PostgreSQL for both storing data, and speaking the protocol in VTGate. diff --git a/content/en/docs/user-guides/production-planning.md b/content/en/docs/user-guides/production-planning.md index 10b67c868..a846adcfb 100644 --- a/content/en/docs/user-guides/production-planning.md +++ b/content/en/docs/user-guides/production-planning.md @@ -6,55 +6,27 @@ aliases: ['/docs/launching/production-planning/','/docs/launching/'] ## Provisioning -### Estimating total resources +### General Recommendations -Although Vitess helps you scale indefinitely, the various layers do consume CPU and memory. Currently, the cost of Vitess servers is dominated by the RPC framework which we use: gRPC (gRPC is a relatively young product). So, Vitess servers are expected to get more efficient over time as there are improvements in gRPC as well as the Go runtime. For now, you can use the following rules of thumb to budget resources for Vitess: +Vitess components (excluding the `mysqld` server) tend to be CPU-bound processes. They use disk space for storing their logs, but do not store any intermediate results on disk, and tend not to be disk IO bound. -Every MySQL instance that serves traffic requires one VTTablet, which is in turn expected to consume an equal amount of CPU. So, if MySQL consumes 8 CPUs, VTTablet is likely going to consume another 8. +The memory requirements for VTGate and VTTablet servers will depend on QPS and result size, but a typical rule of thumb is to provision a baseline of 1GB per core. Typically it is 2-4 cores for each VTGate server, and the same number of cores to VTTablet as with `mysqld`. -The memory consumed by VTTablet depends on QPS and result size, but you can start off with the rule of thumb of requesting 1 GB/CPU. +The impact of network latency can be a major factor when migrating from MySQL to Vitess. A simple rule of thumb is to estimate 2ms of round trip latency added to each query. On the application side, if a significant time is spent making database calls, then you may have to run additional threads or workers to compensate for the delay, which may result in additional memory requirements. -As for VTGate, double the total number of CPUs you’ve allocated for VTTablet. That should be approximately how much the VTGate servers are expected to consume. In terms of memory, you should again budget about 1 GB/CPU (needs verification). +### Planning Shard Size -Vitess servers will use disk space for their logs. A smoothly running server should create very little log spam. However, log files can grow big very quickly if there are too many errors. It will be wise to run a log purger daemon if you’re concerned about filling up disk. +Vitess recommends provisioning shard sizes to approximately 250GB. This is not a hard-limit, and is driven primarily by the recovery time should an instance fail. With 250GB a full-recovery from backup is expected within less than 15 minutes. -Vitess servers are also likely to add about 2 ms of round-trip latency per MySQL call. This may result in some hidden costs that may or may not be negligible. On the app side, if a significant time is spent making database calls, then you may have to run additional threads or workers to compensate for the delay, which may result in additional memory requirements. +### Running Multiple Tablets Per Server -The client driver CPU usage may be different from a normal MySQL driver. That may require you to allocate more CPU per app thread. +Vitess encourages running multiple tablets (shards) per physical server. Typically the best way to do this is with Kubernetes, but `mysqlctl` also supports launching and managing multiple tablet servers if required. -On the server side, this could result in longer running transactions, which could weigh down MySQL. +Assuming tablets are kept to the recommended size of 250GB, they will require 2-4 cores for `mysqld` plus 2-4 cores for the VTTablet process. -With the above numbers as starting point, the next step will be to set up benchmarks that generate production representative load. If you cannot afford this luxury, you may have to go into production with some over-provisioning, just in case. +### Topology Service Provisioning -### Mapping topology to hardware - -The different Vitess components have different resource requirements e.g. VTGate requires little disk in comparison to VTTablet. Therefore, the components should be mapped to different machine classes for optimal resource usage. If you’re using a cluster manager (such as Kubernetes), the automatic scheduler will do this for you. Otherwise, you have to allocate physical machines and plan out how you’re going to map servers onto them. - -Machine classes needed: - -#### MySQL + VTTablet - -You’ll need database-class machines that are likely to have SSDs, and enough RAM to fit the MySQL working set in buffer cache. Make sure that there will be sufficient CPU left for VTTablet to run on them. - -The VTTablet provisioning will be dictated by the MySQL instances they run against. However, soon after launch, it’s recommended to shard these instances to a data size of 100-300 GB. This should also typically reduce the per-MySQL CPU usage to around 2-4 CPUs depending on the load pattern. - -#### VTGate - -For VTGate servers, you’ll need a class of machines that would be CPU heavy, but may be light on memory usage, and should require normal hard disks, for binary and logs only. - -It’s advisable to run more instances than there are machines. VTGate servers are happiest when they’re consuming between 2-4 CPUs. So, if your total requirement was 400 CPUs, and your VTGate class machine has 48 cores each, you’ll need about 10 such machines and you’ll be running about 10 VTGate servers per box. - -You may have to add a few more app class machines to absorb any additional CPU and latency overheads. - -## Lock service setup - -The Lock Service should be running, and both the global and local instances should be up. See the [Topology Service](../../user-guides/topology-service) document for more information. - -Each lock service implementation supports a couple configuration command line parameters, they need to be specified for each Vitess process. - -For sizing purposes, the Vitess processes do not access the lock service very much. Each *vtgate* process keeps a few watches on a few local nodes (`VSchema` and `SrvKeyspace`). Each *vttablet* process will keep its own Tablet record up to date, but it usually doesn't change. The *vtctld* process will access it a lot more, but only on demand to display web pages. - -As mentioned previously, if the setup is only in one cell, the global and local instances can be combined. Just use different top-level directories. +By design, Vitess tries to contact the topology service as little as possible. For estimating provisioning, you can typically use the minimum requirements recommended by your preferred Topology Service. ## Production testing @@ -62,8 +34,8 @@ Before running Vitess in production, please make yourself comfortable first with Here is a short list of all the basic workflows Vitess supports: -* [Failover](../../user-guides/upgrading/) / [Reparents](../../user-guides/reparenting/) +* [Reparenting](../../user-guides/reparenting) * [Backup/Restore](../../user-guides/backup-and-restore) * [Schema Management](../../schema-management) -* [Resharding](../../sharding/#resharding) / [Horizontal Resharding Tutorial](../../get-started/local/) +* [Resharding](../../reference/sharding#resharding) / [Horizontal Sharding Tutorial](../../user-guides/horizonal-sharding) * [Upgrading](../../user-guides/upgrading) diff --git a/content/en/docs/user-guides/reparenting.md b/content/en/docs/user-guides/reparenting.md index aaa497679..f033d5af6 100644 --- a/content/en/docs/user-guides/reparenting.md +++ b/content/en/docs/user-guides/reparenting.md @@ -4,14 +4,14 @@ weight: 5 aliases: ['/user-guide/reparenting.html','/user-guide/reparenting/'] --- -**Reparenting** is the process of changing a shard's master tablet from one host to another or changing a slave tablet to have a different master. Reparenting can be initiated manually or it can occur automatically in response to particular database conditions. As examples, you might reparent a shard or tablet during a maintenance exercise or automatically trigger reparenting when a master tablet dies. +**Reparenting** is the process of changing a shard's master tablet from one host to another or changing a replica tablet to have a different master. Reparenting can be initiated manually or it can occur automatically in response to particular database conditions. As examples, you might reparent a shard or tablet during a maintenance exercise or automatically trigger reparenting when a master tablet dies. This document explains the types of reparenting that Vitess supports: -* [Active reparenting](../reparenting/#active-reparenting) occurs when the Vitess toolchain manages the entire reparenting process. -* [External reparenting](../reparenting/#external-reparenting) occurs when another tool handles the reparenting process, and the Vitess toolchain just updates its topology server, replication graph, and serving graph to accurately reflect master-slave relationships. +* [Active reparenting](../reparenting/#active-reparenting) occurs when Vitess manages the entire reparenting process. +* [External reparenting](../reparenting/#external-reparenting) occurs when another tool handles the reparenting process, and Vitess just updates its topology service, replication graph, and serving graph to accurately reflect master-replica relationships. -**Note:** The `InitShardMaster` command defines the initial parenting relationships within a shard. That command makes the specified tablet the master and makes the other tablets in the shard slaves that replicate from that master. +**Note:** The `InitShardMaster` command defines the initial parenting relationships within a shard. That command makes the specified tablet the master and makes the other tablets in the shard replicas that replicate from that master. ## MySQL requirements @@ -33,9 +33,9 @@ You can use the following `vtctl` commands to perform reparenting operations: * `PlannedReparentShard` * `EmergencyReparentShard` -Both commands lock the Shard record in the global topology server. The two commands cannot run in parallel, nor can either command run in parallel with the `InitShardMaster` command. +Both commands lock the Shard record in the global topology service. The two commands cannot run in parallel, nor can either command run in parallel with the `InitShardMaster` command. -The two commands are both dependent on the global topology server being available, and they both insert rows in the topology server's `_vt.reparent_journal` table. As such, you can review your database's reparenting history by inspecting that table. +Both commands are both dependent on the global topology service being available, and they both insert rows in the topology service's `_vt.reparent_journal` table. As such, you can review your database's reparenting history by inspecting that table. ### PlannedReparentShard: Planned reparenting @@ -49,7 +49,7 @@ This command performs the following actions: 4. Instructs the master-elect tablet to wait for replication data and then begin functioning as the new master after that data is fully transferred. 5. Ensures replication is functioning properly via the following steps: - On the master-elect tablet, insert an entry in a test table and then update the global Shard object's MasterAlias record. - - In parallel on each slave, including the old master, set the new master and wait for the test entry to replicate to the slave tablet. (Slave tablets that had not been replicating before the command was called are left in their current state and do not start replication after the reparenting process.) + - In parallel on each replica, including the old master, set the new master and wait for the test entry to replicate to the replica tablet. Replica tablets that had not been replicating before the command was called are left in their current state and do not start replication after the reparenting process. - Start replication on the old master tablet so it catches up to the new master. In this scenario, the old master's tablet type transitions to `spare`. If health checking is enabled on the old master, it will likely rejoin the cluster as a replica on the next health check. To enable health checking, set the `target_tablet_type` parameter when starting a tablet. That parameter indicates what type of tablet that tablet tries to be when healthy. When it is not healthy, the tablet type changes to spare. @@ -58,34 +58,33 @@ In this scenario, the old master's tablet type transitions to `spare`. If health The `EmergencyReparentShard` command is used to force a reparent to a new master when the current master is unavailable. The command assumes that data cannot be retrieved from the current master because it is dead or not working properly. -As such, this command does not rely on the current master at all to replicate data to the new master. Instead, it makes sure that the master-elect is the most advanced in replication within all of the available slaves. +As such, this command does not rely on the current master at all to replicate data to the new master. Instead, it makes sure that the master-elect is the most advanced in replication within all of the available replicas. -**Important**: Before calling this command, you must first identify the slave with the most advanced replication position as that slave must be designated as the new master. You can use the [`vtctl ShardReplicationPositions`](https://vitess.io/reference/vtctl/#shardreplicationpositions) command to determine the current replication positions of a shard's slaves. +**Important**: Before calling this command, you must first identify the replica with the most advanced replication position as that replica must be designated as the new master. You can use the [`vtctl ShardReplicationPositions`](../../reference/vtctl/#shardreplicationpositions) command to determine the current replication positions of a shard's replicas. This command performs the following actions: -1. Determines the current replication position on all of the slave tablets and confirms that the master-elect tablet has the most advanced replication position. +1. Determines the current replication position on all of the replica tablets and confirms that the master-elect tablet has the most advanced replication position. 2. Promotes the master-elect tablet to be the new master. In addition to changing its tablet type to master, the master-elect performs any other changes that might be required for its new state. 3. Ensures replication is functioning properly via the following steps: - On the master-elect tablet, Vitess inserts an entry in a test table and then updates the `MasterAlias` record of the global Shard object. - - In parallel on each slave, excluding the old master, Vitess sets the master and waits for the test entry to replicate to the slave tablet. (Slave tablets that had not been replicating before the command was called are left in their current state and do not start replication after the reparenting process.) + - In parallel on each replica, excluding the old master, Vitess sets the master and waits for the test entry to replicate to the replica tablet. Replica tablets that had not been replicating before the command was called are left in their current state and do not start replication after the reparenting process. ## External Reparenting -External reparenting occurs when another tool handles the process of changing a shard's master tablet. After that occurs, the tool needs to call the [`vtctl TabletExternallyReparented`](https://vitess.io/reference/vtctl/#tabletexternallyreparented) command to ensure that the topology server, replication graph, and serving graph are updated accordingly. +External reparenting occurs when another tool handles the process of changing a shard's master tablet. After that occurs, the tool needs to call the [`vtctl TabletExternallyReparented`](../../reference/vtctl/#tabletexternallyreparented) command to ensure that the topology service, replication graph, and serving graph are updated accordingly. That command performs the following operations: -1. Locks the shard in the global topology server. -2. Reads the `Shard` object from the global topology server. -3. Reads all of the tablets in the replication graph for the shard. Vitess does allow partial reads in this step, which means that Vitess will proceed even if a data center is down as long as the data center containing the new master is available. -4. Ensures that the new master's state is updated correctly and that the new master is not a MySQL slave of another server. It runs the MySQL `show slave status` command, ultimately aiming to confirm that the MySQL `reset slave` command already executed on the tablet. -5. Updates, for each slave, the topology server record and replication graph to reflect the new master. If the old master does not return successfully in this step, Vitess changes its tablet type to spare to ensure that it does not interfere with ongoing operations. -6. Updates the `Shard` object to specify the new master. +1. Reads the Tablet from the local topology service. +2. Reads the Shard object from the global topology service. +3. If the Tablet type is not already `MASTER`, sets the tablet type to `MASTER`. +4. The Shard record is updated asynchronously (if needed) with the current master alias. +5. Any other tablets that still have their tablet type to `MASTER` will demote themselves to `REPLICA`. The `TabletExternallyReparented` command fails in the following cases: -* The global topology server is not available for locking and modification. In that case, the operation fails completely. +* The global topology service is not available for locking and modification. In that case, the operation fails completely. Active reparenting might be a dangerous practice in any system that depends on external reparents. You can disable active reparents by starting `vtctld` with the `--disable_active_reparents` flag set to true. (You cannot set the flag after `vtctld` is started.) diff --git a/content/en/docs/user-guides/server-configuration.md b/content/en/docs/user-guides/server-configuration.md index d792b8724..ac69acf2e 100644 --- a/content/en/docs/user-guides/server-configuration.md +++ b/content/en/docs/user-guides/server-configuration.md @@ -62,12 +62,12 @@ See the [Transport Security Model](../../user-guides/transport-security-model) d ### Topology Service configuration -Vttablet, vtgate, vtctld need the right command line parameters to find the Topology Server. First the `topo_implementation` flag needs to be set to one of zk2, etcd2, or consul. Then they're all configured as follows: +Vttablet, vtgate and vtctld need the right command line parameters to find the topology service. First the `topo_implementation` flag needs to be set to one of zk2, etcd2, or consul. Then they're all configured as follows: -* The `topo_global_server_address` contains the server address / addresses of the global topology server. +* The `topo_global_server_address` contains the server address / addresses of the global topology service. * The `topo_global_root` contains the directory / path to use. -Note that the local cell for the tablet must exist and be configured properly in the Topology Service for vttablet to start. Local cells are configured inside the topo server, by using the vtctl AddCellInfo command. See the Topology Service documentation for more information. +Note that the local cell for the tablet must exist and be configured properly in the Topology Service for vttablet to start. Local cells are configured inside the topology service, by using the `vtctl AddCellInfo` command. See the Topology Service documentation for more information. ## VTTablet @@ -443,7 +443,7 @@ For VTGate, here’s a list of possible variables to alert on: * Error rate * Error/query rate * Error/query/tablet-type rate -* VTGate serving graph is stale by x minutes (lock server is down) +* VTGate serving graph is stale by x minutes (topology service is down) * Qps/core * Latency