From 4647e3311e61deca60d6b1b9126933b2348d9fa7 Mon Sep 17 00:00:00 2001 From: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> Date: Sun, 21 Mar 2021 16:06:33 +0200 Subject: [PATCH 1/8] strategy comparison chart: vitess functionality Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> --- .../schema-changes/ddl-strategies.md | 21 ++++++++++++++++++- 1 file changed, 20 insertions(+), 1 deletion(-) diff --git a/content/en/docs/user-guides/schema-changes/ddl-strategies.md b/content/en/docs/user-guides/schema-changes/ddl-strategies.md index 43432f91f..cb6630d92 100644 --- a/content/en/docs/user-guides/schema-changes/ddl-strategies.md +++ b/content/en/docs/user-guides/schema-changes/ddl-strategies.md @@ -144,8 +144,27 @@ There are pros and cons to using any of the strategies. Some notable differences #### MySQL compatibility -- `pt-online-schema-change` supports foreign keys. Neither `gh-ost` nor `VReplication` support foreign keys. +- `pt-online-schema-change` partially supports foreign keys. Neither `gh-ost` nor `VReplication` support foreign keys. #### External MySQL compatibility * If you run on Aurora or RDS you will need to use the `online` strategy. This is because both `pt-online-schema-change` and `gh-ost` try to create a new user and then attempt to grant the new user SUPER privileges. + +## Vitess functionality comparison + +| Strategy | Managed | Online | Trackable | Declarative | Revertible | +|----------|---------|--------|-----------|-------------|---------------------| +| `direct` | No | MySQL* | No | No | No | +| `pt-osc` | Yes | Yes* | Yes | Yes | `CREATE,DROP` | +| `gh-ost` | Yes | Yes* | Yes+ | Yes | `CREATE,DROP` | +| `online` | Yes | Yes* | Yes | Yes | `CREATE,DROP,ALTER` | + +- **Managed**: whether Vitess schedules and operates the migration +- **Online**: + - MySQL supports limited online ("In place") DDL and instant DDL. See [support chart](https://dev.mysql.com/doc/refman/8.0/en/innodb-online-ddl-operations.html). + - `gh-ost`, `online` do not support foreign keys + - `pt-osc` has support for foreign keys (may apply collateral blocking operations) +- **Trackable**: able to determine migration state (`ready`, `running`, `complete` etc) + - `gh-ost` also makes available _progress %_ and _ETA seconds_ +- **Declarative**: support `-declarative` flag +- **Revertible**: online `online` strategy supports revertible `ALTER` statements (or `ALTER`s implied by `-declarative` migrations). All managed strategies supports revertible `CREATE` and `ALTER`. From 033d6c9746ca9929fd4ef28c79908da2f8ed1d5a Mon Sep 17 00:00:00 2001 From: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> Date: Sun, 21 Mar 2021 19:40:17 +0200 Subject: [PATCH 2/8] adding docs for revertible migrations Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> --- .../schema-changes/revertible-migrations.md | 221 ++++++++++++++++++ 1 file changed, 221 insertions(+) create mode 100644 content/en/docs/user-guides/schema-changes/revertible-migrations.md diff --git a/content/en/docs/user-guides/schema-changes/revertible-migrations.md b/content/en/docs/user-guides/schema-changes/revertible-migrations.md new file mode 100644 index 000000000..d01d18368 --- /dev/null +++ b/content/en/docs/user-guides/schema-changes/revertible-migrations.md @@ -0,0 +1,221 @@ +--- +title: Revertible migrations +weight: 2 +aliases: ['/docs/user-guides/revertible-migrations/'] +--- + +Vitess offers _revert_ for online schema migrations: the user may regret a table migration after completion, and roll back the tabe's schema to previous state _without loss of data_. + +Revertible migrations supported for: + +- `CREATE TABLE` statements: the _revert_ is to _uncreate_ the table +- `DROP TABLE` statements: the _revert_ is to _reinstate_ the table, populated with data from time of `DROP` +- `ALTER TABLE` statements: supported in `online` strategy, the _revert_ is to reapply previous table schema, without losing any data added/modified since migration completion. +- Another `revert` migration. It is possible to revert a _revert_, revert the revert of a _revert_, and so forth. + +## Behavior and limitations + +- A revert is a migration of its own, with a migration UUID, similarly to "normal" migrations. +- Migrations are only for revertible for `24h` since completion. +- It's only possible to revert the last successful migration on a given table. Illustrated following. + - In the future it may be possible to revert down the stack of completed migrations. +- `ALTER` migrations are revertible only in `online` strategy. +- If a DDL is a noop, then so is its revert: + - If a table `t` exists, and an online DDL is `CREATE TABLE IF NOT EXISTS t (...)`, then the DDL does nothing, and its revert will do nothing. + - If a table `t` does not exist, and an online DDL is `DROP TABLE IF EXISTS t`, then likewise the DDL does nothing, and its revert does nothing. +- Some `ALTER` reverts are not guaranteed to succeed. Examples: + - An `ALTER` which modifies column `i` from `int` to `bigint`, followed by an `INSERT` that places a value larger than max `int`, cannot be reverted, because Vitess cannot place that new value in the old schema. + - An `ALTER` which removes a `UNIQUE KEY`, followed by an `INSERT` that populates a duplicate value on some column, may not be reverted if that duplicate violates the removed `UNIQUE` constraint. + + Vitess cannot know ahead of time whether a _revert_ is possible or not. + + +## Usage + +Consider the following annotated flow: +```sql +mysql> set @@ddl_strategy='online'; + +mysql> create table t(id int primary key); ++--------------------------------------+ +| uuid | ++--------------------------------------+ +| 3837e739_8a60_11eb_badd_f875a4d24e90 | ++--------------------------------------+ +-- Wait until migration is complete + +mysql> alter table t add column ts timestamp not null default current_timestamp; ++--------------------------------------+ +| uuid | ++--------------------------------------+ +| 6bc591b2_8a60_11eb_badd_f875a4d24e90 | ++--------------------------------------+ +-- Wait until migration is complete + +mysql> show create table t \G +*************************** 1. row *************************** + Table: t +Create Table: CREATE TABLE `t` ( + `id` int(11) NOT NULL, + `ts` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP, + PRIMARY KEY (`id`) +) ENGINE=InnoDB DEFAULT CHARSET=utf8 + +-- it is now possible to revert 6bc591b2_8a60_11eb_badd_f875a4d24e90, because it was the last successful migration on table t. +-- it is not possible to revert 3837e739_8a60_11eb_badd_f875a4d24e90, because while it was successful, it is not the last successful migration to run on table t t. + +mysql> revert vitess_migration '6bc591b2_8a60_11eb_badd_f875a4d24e90'; ++--------------------------------------+ +| uuid | ++--------------------------------------+ +| ead67f31_8a60_11eb_badd_f875a4d24e90 | ++--------------------------------------+ +-- Wait until migration is complete + +mysql> show create table t \G +*************************** 1. row *************************** + Table: t +Create Table: CREATE TABLE `t` ( + `id` int(11) NOT NULL, + PRIMARY KEY (`id`) +) ENGINE=InnoDB DEFAULT CHARSET=utf8 + +-- It is now possible to revert ead67f31_8a60_11eb_badd_f875a4d24e90 as it is the last successful migration to run on table t. Reverting ead67f31_8a60_11eb_badd_f875a4d24e90 affectively means restoring the changes made by 6bc591b2_8a60_11eb_badd_f875a4d24e90 + +mysql> revert vitess_migration 'ead67f31_8a60_11eb_badd_f875a4d24e90'; ++--------------------------------------+ +| uuid | ++--------------------------------------+ +| 3b99f686_8a61_11eb_badd_f875a4d24e90 | ++--------------------------------------+ +-- Wait until migration is complete + +mysql> show create table t \G +*************************** 1. row *************************** + Table: t +Create Table: CREATE TABLE `t` ( + `id` int(11) NOT NULL, + `ts` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP, + PRIMARY KEY (`id`) +) ENGINE=InnoDB DEFAULT CHARSET=utf8 + +-- Let's try an invalid migration: ++--------------------------------------+ +| uuid | ++--------------------------------------+ +| 7fbdf1c7_8a61_11eb_badd_f875a4d24e90 | ++--------------------------------------+ +-- This will fail because column `id` already exists. + + id: 11 + migration_uuid: 7fbdf1c7_8a61_11eb_badd_f875a4d24e90 + keyspace: commerce + shard: 0 + mysql_schema: vt_commerce + mysql_table: t +migration_statement: alter table t add column id bigint + strategy: online + options: + added_timestamp: 2021-03-21 18:21:36 +requested_timestamp: 2021-03-21 18:21:32 + ready_timestamp: 2021-03-21 18:21:36 + started_timestamp: 2021-03-21 18:21:36 + liveness_timestamp: 2021-03-21 18:21:36 +completed_timestamp: NULL + cleanup_timestamp: NULL + migration_status: failed +... + ddl_action: alter + message: Duplicate column name 'id' (errno 1060) (sqlstate 42S21) during query: ALTER TABLE `_7fbdf1c7_8a61_11eb_badd_f875a4d24e90_20210321182136_vrepl` add column id bigint +... + +-- it is impossible to revert 7fbdf1c7_8a61_11eb_badd_f875a4d24e90 because it failed. + ++--------------------------------------+ +| uuid | ++--------------------------------------+ +| c3dff91a_8a61_11eb_badd_f875a4d24e90 | ++--------------------------------------+ + +mysql> show vitess_migrations like 'c3dff91a_8a61_11eb_badd_f875a4d24e90' \G +*************************** 1. row *************************** + id: 12 + migration_uuid: c3dff91a_8a61_11eb_badd_f875a4d24e90 + keyspace: commerce + shard: 0 + mysql_schema: vt_commerce + mysql_table: +migration_statement: revert 7fbdf1c7_8a61_11eb_badd_f875a4d24e90 + strategy: online + options: + added_timestamp: 2021-03-21 18:23:31 +requested_timestamp: 2021-03-21 18:23:26 + ready_timestamp: 2021-03-21 18:23:36 + started_timestamp: NULL + liveness_timestamp: NULL +completed_timestamp: NULL + cleanup_timestamp: NULL + migration_status: failed +... + ddl_action: revert + message: can only revert a migration in a 'complete' state. Migration 7fbdf1c7_8a61_11eb_badd_f875a4d24e90 is in 'failed' state +... + +mysql> insert into t values (1, now()); + +mysql> select * from t; ++----+---------------------+ +| id | ts | ++----+---------------------+ +| 1 | 2021-03-21 18:26:47 | ++----+---------------------+ + +mysql> drop table t; ++--------------------------------------+ +| uuid | ++--------------------------------------+ +| 69b17887_8a62_11eb_badd_f875a4d24e90 | ++--------------------------------------+ + +mysql> select * from t; +ERROR 1146 (42S02): ... + +mysql> revert vitess_migration '69b17887_8a62_11eb_badd_f875a4d24e90'; ++--------------------------------------+ +| uuid | ++--------------------------------------+ +| 9eb00275_8a62_11eb_badd_f875a4d24e90 | ++--------------------------------------+ + +mysql> select * from t; ++----+---------------------+ +| id | ts | ++----+---------------------+ +| 1 | 2021-03-21 18:26:47 | ++----+---------------------+ +``` + +## REVERT syntax + +Via SQL: + +```sql +REVERT VITESS_MIGRATION ''; +``` + +Via `vtctl`: +```shell +$ vtctlclient OnlineDDL commerce revert 2201058f_f266_11ea_bab4_0242c0a8b007 +``` + +Both operations return a UUID for the revert migration. The user can track the revert migration to find its state. + +## Implementation details + +Revert for `CREATE` and `DROP` are implemented similarly for all online strategies. + +- The revert for a `CREATE` DDL is to rename the table away and into a [table lifecycle](../../../reference/features/table-lifecycle/) name, rather than actually `DROP` it. This keeps th etale safe for a period of time, and makes it possible to reinstate the table, populated with all data, via a 2nd revert. +- The revert for a `DROP` relies on the fact that Online DDL `DROP TABLE` does not, in fact, drop the table, but actually rename it away. Thus, reverting the `DROP` is merely a `RENAME` back into its original place. +- The revert for `ALTER` is only available for `online` strategy, implemented by `VReplication`. VReplication keep track of a DDL migration by writing down the GTID position through the migration flow. In particular, at time of cut-over and when tables are swapped, VReplication notes the _final_ GTID pos for the migration. + When a revert is requested, Vitess computes a new VReplication rule/filter for the new stream. It them copies the _final_ GTID pos from the reverted migration, and instructs VReplication to resume from that point. + As result, a revert for an `ALTER` migration only needs to catch up with the changelog (binary log entries) since the cut-over of the original migration. To elaborate, it does not need to copy table data, and only needs to consider events for the specific table affected by the revert. This makes the revert operation efficient. From 7654175ebd6acf9b2f1366721a413a53dd5d4d15 Mon Sep 17 00:00:00 2001 From: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> Date: Sun, 21 Mar 2021 20:58:34 +0200 Subject: [PATCH 3/8] documenting declarative migrations Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> --- .../schema-changes/declarative-migrations.md | 117 ++++++++++++++++++ .../managed-online-schema-changes.md | 3 +- 2 files changed, 119 insertions(+), 1 deletion(-) create mode 100644 content/en/docs/user-guides/schema-changes/declarative-migrations.md diff --git a/content/en/docs/user-guides/schema-changes/declarative-migrations.md b/content/en/docs/user-guides/schema-changes/declarative-migrations.md new file mode 100644 index 000000000..4c7ac2da7 --- /dev/null +++ b/content/en/docs/user-guides/schema-changes/declarative-migrations.md @@ -0,0 +1,117 @@ +--- +title: Declarative migrations +weight: 2 +aliases: ['/docs/user-guides/declarative-migrations/'] +--- + +Vitess offers _declarative_ online schema migrations: + +- The user may indicate a desired table schema and Vitess will make it so, whether the table exists or not, or +- The user may indicate a table should not exist, and Vitess will make it so. + +Declarative DDLs are expressed via: + +- Complete `CREATE TABLE` statement (make the table in desired state) +- `DROP TABLE` statement (make the table go) + +Altering tables in declarative DDL is done by issuing `CREATE TABLE` statements with the desired state. `ALTER` statements are not allowed. + +Declarative DDLs have the property of being idempotent. For example, a user may submit the same `CREATE TABLE` statement _twice_, one after another. If the 1st is successful, then the 2nd is a noop, and considered as implicitly successful. likewise, two `DROP TABLE` DDLs for same statement will each ensure the table does not exist. If the 1st is successful, then the 2nd has nothing to do and is implicitly successful. + +## Usage + +Add `-declarative` to any of the online DDL strategies. Example: + +```sql + +mysql> set @@ddl_strategy='online -declarative'; + +-- The following migration creates a new table: +mysql> create table decl_table(id int primary key); ++--------------------------------------+ +| uuid | ++--------------------------------------+ +| b06475e5_8a74_11eb_badd_f875a4d24e90 | ++--------------------------------------+ + +-- The next migration will implicitly ALTER the table decl_table into desired state: +mysql> create table decl_table(id int primary key, ts timestamp not null); ++--------------------------------------+ +| uuid | ++--------------------------------------+ +| b7d6e6fb_8a74_11eb_badd_f875a4d24e90 | ++--------------------------------------+ + +-- Next migration does not change table structure, hence is a noop and implicitly successful: +mysql> create table decl_table(id int primary key, ts timestamp not null); ++--------------------------------------+ +| uuid | ++--------------------------------------+ +| 110574b1_8a75_11eb_badd_f875a4d24e90 | ++--------------------------------------+ +``` + +Consider migration `b7d6e6fb_8a74_11eb_badd_f875a4d24e90` above, which results in an `ALTER`. A look into the migration shows: + +```sql +mysql> show vitess_migrations like 'b7d6e6fb_8a74_11eb_badd_f875a4d24e90'\G +*************************** 1. row *************************** + id: 19 + migration_uuid: b7d6e6fb_8a74_11eb_badd_f875a4d24e90 + keyspace: commerce + shard: 0 + mysql_schema: vt_commerce + mysql_table: decl_table +migration_statement: create table decl_table ( + id int primary key, + ts timestamp not null +) + strategy: online + options: -declarative + added_timestamp: 2021-03-21 20:39:08 +requested_timestamp: 2021-03-21 20:39:07 + ready_timestamp: 2021-03-21 20:39:10 + started_timestamp: 2021-03-21 20:39:10 + liveness_timestamp: 2021-03-21 20:39:13 +completed_timestamp: 2021-03-21 20:39:13 + cleanup_timestamp: NULL + migration_status: complete + log_path: + artifacts: _b7d6e6fb_8a74_11eb_badd_f875a4d24e90_20210321203910_vrepl, + retries: 0 + tablet: zone1-0000000100 + tablet_failure: 0 + progress: 100 + migration_context: vtgate:38368dbe-8a60-11eb-badd-f875a4d24e90 + ddl_action: alter + message: ADD COLUMN `ts` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP + eta_seconds: 0 +``` +Note how while the migration statement is `create`, the migration's `ddl_action` ends up being `alter`, and `message` indicates the alter options. + +You may add `-declarative` even if you otherwise supply flags to your favorite strategy. For example, the following is valid: +```sql +set @@ddl_strategy='gh-ost -declarative -max-load=Threads_running=100'; +``` + +Vitess notes down the `-declarative` flag and does not pass it to `gh-ost`, `pt-osc` or `VReplication`. + +## Implementation details + +The user submits a declarative DDL. Tables schedule the migration to execute, but at time of execution, may modify the migration on the fly and end up running a different migration. + +Consider the following types of migrations: + +- A `REVERT` has no special behavior, it acts as a normal revert. +- `ALTER` is rejected (migration will fail) +- `DROP`: silently mark as successful if the table does not exist. Otherwise treat the DDL as a normal `DROP`. +- `CREATE`: either, + - The table does not exist: proceed as normal `CREATE` + - The table exists: evaluate the SQL diff between the existing table schema and the proposed schema. Either: + - There is no diff (exact same schema): silently mark as successful + - There is a diff: rewrite the DDL as an actual `ALTER`, run using relevant strategy. + +Declarative DDLs are [revertible](../revertible-migrations/). Note: + +- A declarative migration which ends up being an `ALTER` is only revertible if executed with `online` strategy. +- A declarative migration which ends up being a noop (and implicitly successful), implies a noop revert. diff --git a/content/en/docs/user-guides/schema-changes/managed-online-schema-changes.md b/content/en/docs/user-guides/schema-changes/managed-online-schema-changes.md index 4f140bdfd..675660efa 100644 --- a/content/en/docs/user-guides/schema-changes/managed-online-schema-changes.md +++ b/content/en/docs/user-guides/schema-changes/managed-online-schema-changes.md @@ -190,7 +190,8 @@ At this time, there are no automated retries. For example, a failover on a shard ## Key Settings - * `retain_online_ddl_tables` - This determines how long vttablet should keep an old migrated table before purging it + * `retain_online_ddl_tables` - This determines how long vttablet should keep an old migrated table before purging it. Unit: duration. Default: 24 hours. Example: `-retain_online_ddl_tables 48h` + * `migration_check_interval` - Interval between checks for submitted migrations. Unit: duration. Default: 1 hour. Example: `-migration_check_interval 30s` ## Tracking migrations From 67c511e5929c1a32eaa3717b1a1578e428d728c8 Mon Sep 17 00:00:00 2001 From: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> Date: Mon, 22 Mar 2021 15:39:00 +0200 Subject: [PATCH 4/8] improved preface for Online DDL. Minor fixes over new docs Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> --- .../schema-changes/ddl-strategies.md | 8 ++-- .../schema-changes/declarative-migrations.md | 2 +- .../managed-online-schema-changes.md | 31 +++++++++++--- .../schema-changes/revertible-migrations.md | 42 ++++++++++--------- 4 files changed, 53 insertions(+), 30 deletions(-) diff --git a/content/en/docs/user-guides/schema-changes/ddl-strategies.md b/content/en/docs/user-guides/schema-changes/ddl-strategies.md index cb6630d92..c4ba524e4 100644 --- a/content/en/docs/user-guides/schema-changes/ddl-strategies.md +++ b/content/en/docs/user-guides/schema-changes/ddl-strategies.md @@ -161,10 +161,10 @@ There are pros and cons to using any of the strategies. Some notable differences - **Managed**: whether Vitess schedules and operates the migration - **Online**: - - MySQL supports limited online ("In place") DDL and instant DDL. See [support chart](https://dev.mysql.com/doc/refman/8.0/en/innodb-online-ddl-operations.html). - - `gh-ost`, `online` do not support foreign keys - - `pt-osc` has support for foreign keys (may apply collateral blocking operations) + - MySQL supports limited online ("In place") DDL and instant DDL. See [support chart](https://dev.mysql.com/doc/refman/8.0/en/innodb-online-ddl-operations.html). + - `gh-ost`, `online` do not support foreign keys + - `pt-osc` has support for foreign keys (may apply collateral blocking operations) - **Trackable**: able to determine migration state (`ready`, `running`, `complete` etc) - `gh-ost` also makes available _progress %_ and _ETA seconds_ - **Declarative**: support `-declarative` flag -- **Revertible**: online `online` strategy supports revertible `ALTER` statements (or `ALTER`s implied by `-declarative` migrations). All managed strategies supports revertible `CREATE` and `ALTER`. +- **Revertible**: `online` strategy supports revertible `ALTER` statements (or `ALTER`s implied by `-declarative` migrations). All managed strategies supports revertible `CREATE` and `ALTER`. diff --git a/content/en/docs/user-guides/schema-changes/declarative-migrations.md b/content/en/docs/user-guides/schema-changes/declarative-migrations.md index 4c7ac2da7..a3bd32d81 100644 --- a/content/en/docs/user-guides/schema-changes/declarative-migrations.md +++ b/content/en/docs/user-guides/schema-changes/declarative-migrations.md @@ -16,7 +16,7 @@ Declarative DDLs are expressed via: Altering tables in declarative DDL is done by issuing `CREATE TABLE` statements with the desired state. `ALTER` statements are not allowed. -Declarative DDLs have the property of being idempotent. For example, a user may submit the same `CREATE TABLE` statement _twice_, one after another. If the 1st is successful, then the 2nd is a noop, and considered as implicitly successful. likewise, two `DROP TABLE` DDLs for same statement will each ensure the table does not exist. If the 1st is successful, then the 2nd has nothing to do and is implicitly successful. +Declarative DDLs have the property of being idempotent. For example, a user may submit the same `CREATE TABLE` statement _twice_, one after another. If the 1st is successful, then the 2nd is a noop, and considered as implicitly successful. Likewise, two `DROP TABLE` DDLs for same statement will each ensure the table does not exist. If the 1st is successful, then the 2nd has nothing to do and is implicitly successful. ## Usage diff --git a/content/en/docs/user-guides/schema-changes/managed-online-schema-changes.md b/content/en/docs/user-guides/schema-changes/managed-online-schema-changes.md index 675660efa..48f884d5d 100644 --- a/content/en/docs/user-guides/schema-changes/managed-online-schema-changes.md +++ b/content/en/docs/user-guides/schema-changes/managed-online-schema-changes.md @@ -6,8 +6,18 @@ aliases: ['/docs/user-guides/managed-online-schema-changes/'] **Note:** this feature is **EXPERIMENTAL**. -Vitess offers managed, online schema migrations (aka Online DDL), transparently to the user. Vitess supports `CREATE`, `ALTER` and `DROP` statements, all non-blocking, auto scheduled and auditable. As general overview: +Vitess offers managed, online schema migrations (aka Online DDL), transparently to the user. Vitess Onine DDL offers: +- Non-blocking migrations +- Migrations are asyncronously auto-scheduled, queued and executed by tablets +- Migration state is trackable +- Migrations are cancellable +- Migrations are retry-able +- Lossless, [revertible migrations](../revertible-migrations/) +- Support for [declarative migrations](../declarative-migrations/) + + +As general overview: - User chooses a [strategy](../ddl-strategies) for online DDL (online DDL is opt in) - User submits one or more schema change queries, using the standard MySQL `CREATE TABLE`, `ALTER TABLE` and `DROP TABLE` syntax. - Vitess responds with a Job ID for each schema change query. @@ -188,11 +198,20 @@ At this time, the user is responsible to track the state of all migrations. VTTa At this time, there are no automated retries. For example, a failover on a shard causes the migration to fail, and Vitess will not try to re-run the migration on the new `primary`. It is the user's responsibility to issue a `retry`. This may change in the future. -## Key Settings +## Configuration + +- `-retain_online_ddl_tables`: (`vttablet`) determines how long vttablet should keep an old migrated table before purging it. Type: duration. Default: 24 hours. + + Example: `vttablet -retain_online_ddl_tables 48h` - * `retain_online_ddl_tables` - This determines how long vttablet should keep an old migrated table before purging it. Unit: duration. Default: 24 hours. Example: `-retain_online_ddl_tables 48h` - * `migration_check_interval` - Interval between checks for submitted migrations. Unit: duration. Default: 1 hour. Example: `-migration_check_interval 30s` +- `-migration_check_interval`: (`vttablet`) interval between checks for submitted migrations. Type: duration. Default: 1 hour. + Example: `vttablet -migration_check_interval 30s` + +- `-enable_online_ddl`: (`vtgate`) whether Online DDL operations are at all possible through `VTGate`. Type: boolean. Default: `true` + + Example: `vtgate -enable_online_ddl=false` to disable access to Online DDL via `VTGate`. + ## Tracking migrations You may track the status of a single migration, of all or recent migrations, or of migrations in a specific state. Examples: @@ -272,7 +291,7 @@ The user may cancel a migration, as follows: The syntax to cancelling a migration is: ``` -vtctlclient OnlineDDL cancel +vtctlclient OnlineDDL cancel ``` Example: @@ -475,7 +494,7 @@ Examples: $ vtctlclient VExec commerce.2201058f_f266_11ea_bab4_0242c0a8b007 "select * from _vt.schema_migrations" -$ vtctlclient VExec commerce.91b5c953-e1e2-11ea-a097-f875a4d24e90 "update _vt.schema_migrations set migration_status='retry' +$ vtctlclient VExec commerce.91b5c953-e1e2-11ea-a097-f875a4d24e90 "update _vt.schema_migrations set migration_status='retry'" $ vtctlclient VExec commerce.91b5c953-e1e2-11ea-a097-f875a4d24e90 "update _vt.schema_migrations set migration_status='retry' where shard='40-80' ``` diff --git a/content/en/docs/user-guides/schema-changes/revertible-migrations.md b/content/en/docs/user-guides/schema-changes/revertible-migrations.md index d01d18368..8f2d92722 100644 --- a/content/en/docs/user-guides/schema-changes/revertible-migrations.md +++ b/content/en/docs/user-guides/schema-changes/revertible-migrations.md @@ -4,7 +4,7 @@ weight: 2 aliases: ['/docs/user-guides/revertible-migrations/'] --- -Vitess offers _revert_ for online schema migrations: the user may regret a table migration after completion, and roll back the tabe's schema to previous state _without loss of data_. +Vitess offers _lossless revert_ for online schema migrations: the user may regret a table migration after completion, and roll back the tabe's schema to previous state _without loss of data_. Revertible migrations supported for: @@ -29,8 +29,22 @@ Revertible migrations supported for: Vitess cannot know ahead of time whether a _revert_ is possible or not. +## REVERT syntax + +Via SQL: + +```sql +REVERT VITESS_MIGRATION '69b17887_8a62_11eb_badd_f875a4d24e90'; +``` + +Via `vtctl`: +```shell +$ vtctlclient OnlineDDL commerce revert 69b17887_8a62_11eb_badd_f875a4d24e90 +``` -## Usage +Both operations return a UUID for the revert migration. The user can track the revert migration to find its state. + +## Usage & walkthrough Consider the following annotated flow: ```sql @@ -62,7 +76,8 @@ Create Table: CREATE TABLE `t` ( ) ENGINE=InnoDB DEFAULT CHARSET=utf8 -- it is now possible to revert 6bc591b2_8a60_11eb_badd_f875a4d24e90, because it was the last successful migration on table t. --- it is not possible to revert 3837e739_8a60_11eb_badd_f875a4d24e90, because while it was successful, it is not the last successful migration to run on table t t. +-- it is not possible to revert 3837e739_8a60_11eb_badd_f875a4d24e90, because while it was successful, it is not the last +-- successful migration to run on table t t. mysql> revert vitess_migration '6bc591b2_8a60_11eb_badd_f875a4d24e90'; +--------------------------------------+ @@ -80,7 +95,8 @@ Create Table: CREATE TABLE `t` ( PRIMARY KEY (`id`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8 --- It is now possible to revert ead67f31_8a60_11eb_badd_f875a4d24e90 as it is the last successful migration to run on table t. Reverting ead67f31_8a60_11eb_badd_f875a4d24e90 affectively means restoring the changes made by 6bc591b2_8a60_11eb_badd_f875a4d24e90 +-- It is now possible to revert ead67f31_8a60_11eb_badd_f875a4d24e90 as it is the last successful migration to run on table t. +-- Reverting ead67f31_8a60_11eb_badd_f875a4d24e90 affectively means restoring the changes made by 6bc591b2_8a60_11eb_badd_f875a4d24e90 mysql> revert vitess_migration 'ead67f31_8a60_11eb_badd_f875a4d24e90'; +--------------------------------------+ @@ -176,6 +192,7 @@ mysql> drop table t; +--------------------------------------+ | 69b17887_8a62_11eb_badd_f875a4d24e90 | +--------------------------------------+ +-- Wait until migration is complete mysql> select * from t; ERROR 1146 (42S02): ... @@ -186,6 +203,8 @@ mysql> revert vitess_migration '69b17887_8a62_11eb_badd_f875a4d24e90'; +--------------------------------------+ | 9eb00275_8a62_11eb_badd_f875a4d24e90 | +--------------------------------------+ +-- Wait until migration is complete +-- `t` was not really dropped, but renamed away. This REVERT reinstates it. mysql> select * from t; +----+---------------------+ @@ -195,21 +214,6 @@ mysql> select * from t; +----+---------------------+ ``` -## REVERT syntax - -Via SQL: - -```sql -REVERT VITESS_MIGRATION ''; -``` - -Via `vtctl`: -```shell -$ vtctlclient OnlineDDL commerce revert 2201058f_f266_11ea_bab4_0242c0a8b007 -``` - -Both operations return a UUID for the revert migration. The user can track the revert migration to find its state. - ## Implementation details Revert for `CREATE` and `DROP` are implemented similarly for all online strategies. From cc6aa8b7ba763b5aeb01c34b8e18f5682fc07ff1 Mon Sep 17 00:00:00 2001 From: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> Date: Thu, 25 Mar 2021 08:38:46 +0200 Subject: [PATCH 5/8] fixed links Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> --- .../docs/user-guides/schema-changes/declarative-migrations.md | 4 ++-- .../docs/user-guides/schema-changes/revertible-migrations.md | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/content/en/docs/user-guides/schema-changes/declarative-migrations.md b/content/en/docs/user-guides/schema-changes/declarative-migrations.md index a3bd32d81..19b94398e 100644 --- a/content/en/docs/user-guides/schema-changes/declarative-migrations.md +++ b/content/en/docs/user-guides/schema-changes/declarative-migrations.md @@ -1,10 +1,10 @@ --- title: Declarative migrations weight: 2 -aliases: ['/docs/user-guides/declarative-migrations/'] +aliases: ['/docs/user-guides/schema-changes/declarative-migrations/'] --- -Vitess offers _declarative_ online schema migrations: +Vitess's [managed schema changes](../managed-online-schema-changes/) offer _declarative_ online schema migrations: - The user may indicate a desired table schema and Vitess will make it so, whether the table exists or not, or - The user may indicate a table should not exist, and Vitess will make it so. diff --git a/content/en/docs/user-guides/schema-changes/revertible-migrations.md b/content/en/docs/user-guides/schema-changes/revertible-migrations.md index 8f2d92722..757e1306b 100644 --- a/content/en/docs/user-guides/schema-changes/revertible-migrations.md +++ b/content/en/docs/user-guides/schema-changes/revertible-migrations.md @@ -1,10 +1,10 @@ --- title: Revertible migrations weight: 2 -aliases: ['/docs/user-guides/revertible-migrations/'] +aliases: ['/docs/user-guides/schema-changes/revertible-migrations/'] --- -Vitess offers _lossless revert_ for online schema migrations: the user may regret a table migration after completion, and roll back the tabe's schema to previous state _without loss of data_. +Vitess's [managed schema changes](../managed-online-schema-changes/) offer _lossless revert_ for online schema migrations: the user may regret a table migration after completion, and roll back the tabe's schema to previous state _without loss of data_. Revertible migrations supported for: From 1f884bd834446c5ac1a5d7386c89dc937da0a135 Mon Sep 17 00:00:00 2001 From: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> Date: Thu, 25 Mar 2021 17:45:13 +0200 Subject: [PATCH 6/8] revised documentation on Online DDL Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> --- .../schema-changes/audit-and-control.md | 649 ++++++++++++++++++ .../schema-changes/ddl-strategies.md | 2 +- .../schema-changes/declarative-migrations.md | 2 +- .../managed-online-schema-changes.md | 397 +---------- .../schema-changes/revertible-migrations.md | 4 +- .../unmanaged-schema-changes.md | 2 +- 6 files changed, 685 insertions(+), 371 deletions(-) create mode 100644 content/en/docs/user-guides/schema-changes/audit-and-control.md diff --git a/content/en/docs/user-guides/schema-changes/audit-and-control.md b/content/en/docs/user-guides/schema-changes/audit-and-control.md new file mode 100644 index 000000000..e47c4671f --- /dev/null +++ b/content/en/docs/user-guides/schema-changes/audit-and-control.md @@ -0,0 +1,649 @@ +--- +title: Applying, auditing, and controlling Online DDL +weight: 4 +aliases: ['/docs/user-guides/managed-online-schema-changes/audit-and-control'] +--- + +Vitess provides two interfaces to interacting with Online DDL: + +- SQL commands, via `VTGate` +- Command line interface, via `vtctl` + +Supported interactions are: + +- [Running migrations](#running-migrations) (submitting Online DDL requests) +- [Tracking migrations](#tracking-migrations) +- [Cancelling a migration](#cancelling-a-migration) +- [Cancelling all pending migrations](#cancelling-all-keyspace-migrations) +- [Retrying a migration](#retrying-a-migration) +- [Reverting a migration](#reverting-a-migration) + +## Running migrations + +To run a managed schema migration, you should: + +- Formulate your DDLs (`CREATE`, `ALTER`, `DROP`) queries +- Choose a [ddl_strategy](../ddl-strategies) + +When the user submits an online DDL, Vitess responds with a UUID, a job Id used to later track or control the migration. The migration does not start immediately. It is queued at the tablets and executed at some point in the future. + +#### Via VTGate/SQL + +```sql +mysql> set @@ddl_strategy='online'; + +mysql> alter table corder add column ts timestamp not null default current_timestamp; ++--------------------------------------+ +| uuid | ++--------------------------------------+ +| bf4598ab_8d55_11eb_815f_f875a4d24e90 | ++--------------------------------------+ + +mysql> drop table customer; ++--------------------------------------+ +| uuid | ++--------------------------------------+ +| 6848c1a4_8d57_11eb_815f_f875a4d24e90 | ++--------------------------------------+ +``` + +- `@@ddl_strategy` behaves like a MySQL session variable, though is only recognized by `VTGate`. Setting `@@ddl_strategy` only applies to that same connection and does not affect other connections. The strategy applies to all migrations executed in that session. You may subsequently set `@@ddl_strategy` to different value. +- If you run `vtgate` without `-ddl_strategy`, then `@@ddl_strategy` defaults to `'direct'`, which implies schema migrations are synchronous. You will need to `set @@ddl_strategy='gh-ost'` to run followup `ALTER TABLE` statements via `gh-ost`. +- If you run `vtgate -ddl_strategy "gh-ost"`, then `@@ddl_strategy` defaults to `'gh-ost'` in each new session. Any `ALTER TABLE` will run via `gh-ost`. You may `set @@ddl_strategy='pt-osc'` to make migrations run through `pt-online-schema-change`, or `set @@ddl_strategy='direct'` to run migrations synchronously. + +#### Via vtctl/ApplySchema + +You may use `vtctl` or `vtctlclient` (the two are interchangeable for the purpose of this document) to apply schema changes. The `ApplySchema` command supports both synchronous and online schema migrations. To run an online schema migration you will supply the `-ddl_strategy` command line flag: + +```shell +$ vtctlclient ApplySchema -ddl_strategy "online" -sql "ALTER TABLE demo MODIFY id bigint UNSIGNED" commerce +a2994c92_f1d4_11ea_afa3_f875a4d24e90 +``` + +## Tracking migrations + +You may track the status of a single or of multiple migrations. Since migrations run asycnhronously, it is the user's responsibility to audit the progress and state of submitted migrations. Users are likely to want to know when a migration is complete (or failed) so as to be able to deploy code changes or run other operations. + +Common patterns are: + +- Show state of a specific migration +- Show all `running`, `complete` or `failed` migrations +- Show recent migrations + +#### Via VTGate/SQL + +Examples for a single shard cluster: + +```sql +mysql> show vitess_migrations like 'bf4598ab_8d55_11eb_815f_f875a4d24e90' \G +*************************** 1. row *************************** + id: 23 + migration_uuid: bf4598ab_8d55_11eb_815f_f875a4d24e90 + keyspace: commerce + shard: 0 + mysql_schema: vt_commerce + mysql_table: corder +migration_statement: alter table corder add column ts timestamp not null default current_timestamp() + strategy: online + options: + added_timestamp: 2021-03-25 12:35:01 +requested_timestamp: 2021-03-25 12:34:58 + ready_timestamp: 2021-03-25 12:35:04 + started_timestamp: 2021-03-25 12:35:04 + liveness_timestamp: 2021-03-25 12:35:06 +completed_timestamp: 2021-03-25 12:35:06 + cleanup_timestamp: NULL + migration_status: complete + log_path: + artifacts: _bf4598ab_8d55_11eb_815f_f875a4d24e90_20210325123504_vrepl, + retries: 0 + tablet: zone1-0000000100 + tablet_failure: 0 + progress: 100 + migration_context: vtgate:a8352418-8d55-11eb-815f-f875a4d24e90 + ddl_action: alter + message: + eta_seconds: 0 +``` + +```sql +mysql> show vitess_migrations like 'complete' \G +... +*************************** 21. row *************************** + id: 24 + migration_uuid: 6848c1a4_8d57_11eb_815f_f875a4d24e90 + keyspace: commerce + shard: 0 + mysql_schema: vt_commerce + mysql_table: customer +migration_statement: drop table customer + strategy: online + options: + added_timestamp: 2021-03-25 12:46:53 +requested_timestamp: 2021-03-25 12:46:51 + ready_timestamp: 2021-03-25 12:46:57 + started_timestamp: 2021-03-25 12:46:57 + liveness_timestamp: 2021-03-25 12:46:57 +completed_timestamp: 2021-03-25 12:46:57 + cleanup_timestamp: NULL + migration_status: complete + log_path: + artifacts: _vt_HOLD_6848c1a48d5711eb815ff875a4d24e90_20210326104657, + retries: 0 + tablet: zone1-0000000100 + tablet_failure: 0 + progress: 100 + migration_context: vtgate:a8352418-8d55-11eb-815f-f875a4d24e90 + ddl_action: drop + message: + eta_seconds: 0 +``` + +```sql +mysql> show vitess_migrations where completed_timestamp > now() - interval 1 day; ++----+--------------------------------------+----------+-------+--------------+-------------+---------------------------------------------------------------------------------+----------+---------+---------------------+---------------------+---------------------+---------------------+---------------------+---------------------+-------------------+------------------+----------+-------------------------------------------------------------+---------+------------------+----------------+----------+---------------------------------------------+------------+---------+-------------+ +| id | migration_uuid | keyspace | shard | mysql_schema | mysql_table | migration_statement | strategy | options | added_timestamp | requested_timestamp | ready_timestamp | started_timestamp | liveness_timestamp | completed_timestamp | cleanup_timestamp | migration_status | log_path | artifacts | retries | tablet | tablet_failure | progress | migration_context | ddl_action | message | eta_seconds | ++----+--------------------------------------+----------+-------+--------------+-------------+---------------------------------------------------------------------------------+----------+---------+---------------------+---------------------+---------------------+---------------------+---------------------+---------------------+-------------------+------------------+----------+-------------------------------------------------------------+---------+------------------+----------------+----------+---------------------------------------------+------------+---------+-------------+ +| 23 | bf4598ab_8d55_11eb_815f_f875a4d24e90 | commerce | 0 | vt_commerce | corder | alter table corder add column ts timestamp not null default current_timestamp() | online | | 2021-03-25 12:35:01 | 2021-03-25 12:34:58 | 2021-03-25 12:35:04 | 2021-03-25 12:35:04 | 2021-03-25 12:35:06 | 2021-03-25 12:35:06 | NULL | complete | | _bf4598ab_8d55_11eb_815f_f875a4d24e90_20210325123504_vrepl, | 0 | zone1-0000000100 | 0 | 100 | vtgate:a8352418-8d55-11eb-815f-f875a4d24e90 | alter | | 0 | +| 24 | 6848c1a4_8d57_11eb_815f_f875a4d24e90 | commerce | 0 | vt_commerce | customer | drop table customer | online | | 2021-03-25 12:46:53 | 2021-03-25 12:46:51 | 2021-03-25 12:46:57 | 2021-03-25 12:46:57 | 2021-03-25 12:46:57 | 2021-03-25 12:46:57 | NULL | complete | | _vt_HOLD_6848c1a48d5711eb815ff875a4d24e90_20210326104657, | 0 | zone1-0000000100 | 0 | 100 | vtgate:a8352418-8d55-11eb-815f-f875a4d24e90 | drop | | 0 | +| 25 | 6fd57dd3_8d57_11eb_815f_f875a4d24e90 | commerce | 0 | vt_commerce | customer | revert 6848c1a4_8d57_11eb_815f_f875a4d24e90 | online | | 2021-03-25 12:47:08 | 2021-03-25 12:47:04 | 2021-03-25 12:47:12 | 2021-03-25 12:47:12 | 2021-03-25 12:47:12 | 2021-03-25 12:47:12 | NULL | complete | | _vt_HOLD_6848c1a48d5711eb815ff875a4d24e90_20210326104657, | 0 | zone1-0000000100 | 0 | 100 | vtgate:a8352418-8d55-11eb-815f-f875a4d24e90 | create | | 0 | ++----+--------------------------------------+----------+-------+--------------+-------------+---------------------------------------------------------------------------------+----------+---------+---------------------+---------------------+---------------------+---------------------+---------------------+---------------------+-------------------+------------------+----------+-------------------------------------------------------------+---------+------------------+----------------+----------+---------------------------------------------+------------+---------+-------------+ +``` + +- `show vitess_migrations` shows the entire history of migrations. +- `show vitess_migrations like ...` filters migrations by `migration_uuid`, or `migration_context`, or `migration_status`. +- `show vitess_migrations where ...` lets the user specify arbitrary conditions. +- All commands return results for the keyspace (schema) in use. + +#### Via vtctl/ApplySchema + +Examples for a 4-shard cluster: + +```shell +$ vtctlclient OnlineDDL commerce show ab3ffdd5_f25c_11ea_bab4_0242c0a8b007 ++-----------------+-------+--------------+-------------+------------+--------------------------------------+----------+---------------------+---------------------+------------------+ +| Tablet | shard | mysql_schema | mysql_table | ddl_action | migration_uuid | strategy | started_timestamp | completed_timestamp | migration_status | ++-----------------+-------+--------------+-------------+------------+--------------------------------------+----------+---------------------+---------------------+------------------+ +| test-0000000201 | 40-80 | vt_commerce | demo | alter | ab3ffdd5_f25c_11ea_bab4_0242c0a8b007 | online | 2020-09-09 05:24:33 | 2020-09-09 05:24:34 | complete | +| test-0000000301 | 80-c0 | vt_commerce | demo | alter | ab3ffdd5_f25c_11ea_bab4_0242c0a8b007 | online | 2020-09-09 05:25:13 | 2020-09-09 05:25:14 | complete | +| test-0000000401 | c0- | vt_commerce | demo | alter | ab3ffdd5_f25c_11ea_bab4_0242c0a8b007 | online | 2020-09-09 05:25:13 | 2020-09-09 05:25:14 | complete | +| test-0000000101 | -40 | vt_commerce | demo | alter | ab3ffdd5_f25c_11ea_bab4_0242c0a8b007 | online | 2020-09-09 05:25:13 | 2020-09-09 05:25:14 | complete | ++-----------------+-------+--------------+-------------+------------+--------------------------------------+----------+---------------------+---------------------+------------------+ + +$ vtctlclient OnlineDDL commerce show 8a797518_f25c_11ea_bab4_0242c0a8b007 ++-----------------+-------+--------------+-------------+------------+--------------------------------------+----------+---------------------+---------------------+------------------+ +| Tablet | shard | mysql_schema | mysql_table | ddl_action | migration_uuid | strategy | started_timestamp | completed_timestamp | migration_status | ++-----------------+-------+--------------+-------------+------------+--------------------------------------+----------+---------------------+---------------------+------------------+ +| test-0000000401 | c0- | vt_commerce | demo | alter | 8a797518_f25c_11ea_bab4_0242c0a8b007 | online | 2020-09-09 05:23:32 | | running | +| test-0000000201 | 40-80 | vt_commerce | demo | alter | 8a797518_f25c_11ea_bab4_0242c0a8b007 | online | 2020-09-09 05:23:32 | 2020-09-09 05:23:33 | complete | +| test-0000000301 | 80-c0 | vt_commerce | demo | alter | 8a797518_f25c_11ea_bab4_0242c0a8b007 | online | 2020-09-09 05:23:32 | | running | +| test-0000000101 | -40 | vt_commerce | demo | alter | 8a797518_f25c_11ea_bab4_0242c0a8b007 | online | 2020-09-09 05:23:32 | | running | ++-----------------+-------+--------------+-------------+------------+--------------------------------------+----------+---------------------+---------------------+------------------+ + +$ vtctlclient OnlineDDL commerce show 8a797518_f25c_11ea_bab4_0242c0a8b007 ++-----------------+-------+--------------+-------------+------------+--------------------------------------+----------+---------------------+---------------------+------------------+ +| Tablet | shard | mysql_schema | mysql_table | ddl_action | migration_uuid | strategy | started_timestamp | completed_timestamp | migration_status | ++-----------------+-------+--------------+-------------+------------+--------------------------------------+----------+---------------------+---------------------+------------------+ +| test-0000000401 | c0- | vt_commerce | demo | alter | 8a797518_f25c_11ea_bab4_0242c0a8b007 | online | 2020-09-09 05:23:32 | | failed | +| test-0000000101 | -40 | vt_commerce | demo | alter | 8a797518_f25c_11ea_bab4_0242c0a8b007 | online | 2020-09-09 05:23:32 | | failed | +| test-0000000301 | 80-c0 | vt_commerce | demo | alter | 8a797518_f25c_11ea_bab4_0242c0a8b007 | online | 2020-09-09 05:23:32 | | failed | +| test-0000000201 | 40-80 | vt_commerce | demo | alter | 8a797518_f25c_11ea_bab4_0242c0a8b007 | online | 2020-09-09 05:23:32 | 2020-09-09 05:23:33 | complete | ++-----------------+-------+--------------+-------------+------------+--------------------------------------+----------+---------------------+---------------------+------------------+ + +$ vtctlclient OnlineDDL commerce show recent ++-----------------+-------+--------------+-------------+------------+--------------------------------------+----------+---------------------+---------------------+------------------+ +| Tablet | shard | mysql_schema | mysql_table | ddl_action | migration_uuid | strategy | started_timestamp | completed_timestamp | migration_status | ++-----------------+-------+--------------+-------------+------------+--------------------------------------+----------+---------------------+---------------------+------------------+ +| test-0000000201 | 40-80 | vt_commerce | demo | alter | 63b5db0c_f25c_11ea_bab4_0242c0a8b007 | online | 2020-09-09 05:22:41 | 2020-09-09 05:22:42 | complete | +| test-0000000201 | 40-80 | vt_commerce | demo | alter | 8a797518_f25c_11ea_bab4_0242c0a8b007 | online | 2020-09-09 05:23:32 | 2020-09-09 05:23:33 | complete | +| test-0000000201 | 40-80 | vt_commerce | demo | alter | ab3ffdd5_f25c_11ea_bab4_0242c0a8b007 | online | 2020-09-09 05:24:33 | 2020-09-09 05:24:34 | complete | +| test-0000000301 | 80-c0 | vt_commerce | demo | alter | 63b5db0c_f25c_11ea_bab4_0242c0a8b007 | online | 2020-09-09 05:22:41 | 2020-09-09 05:22:42 | complete | +| test-0000000301 | 80-c0 | vt_commerce | demo | alter | 8a797518_f25c_11ea_bab4_0242c0a8b007 | online | 2020-09-09 05:23:32 | | failed | +| test-0000000301 | 80-c0 | vt_commerce | demo | alter | ab3ffdd5_f25c_11ea_bab4_0242c0a8b007 | online | 2020-09-09 05:25:13 | 2020-09-09 05:25:14 | complete | +| test-0000000401 | c0- | vt_commerce | demo | alter | 63b5db0c_f25c_11ea_bab4_0242c0a8b007 | online | 2020-09-09 05:22:41 | 2020-09-09 05:22:42 | complete | +| test-0000000401 | c0- | vt_commerce | demo | alter | 8a797518_f25c_11ea_bab4_0242c0a8b007 | online | 2020-09-09 05:23:32 | | failed | +| test-0000000401 | c0- | vt_commerce | demo | alter | ab3ffdd5_f25c_11ea_bab4_0242c0a8b007 | online | 2020-09-09 05:25:13 | 2020-09-09 05:25:14 | complete | +| test-0000000101 | -40 | vt_commerce | demo | alter | 63b5db0c_f25c_11ea_bab4_0242c0a8b007 | online | 2020-09-09 05:22:41 | 2020-09-09 05:22:42 | complete | +| test-0000000101 | -40 | vt_commerce | demo | alter | 8a797518_f25c_11ea_bab4_0242c0a8b007 | online | 2020-09-09 05:23:32 | | failed | +| test-0000000101 | -40 | vt_commerce | demo | alter | ab3ffdd5_f25c_11ea_bab4_0242c0a8b007 | online | 2020-09-09 05:25:13 | 2020-09-09 05:25:14 | complete | ++-----------------+-------+--------------+-------------+------------+--------------------------------------+----------+---------------------+---------------------+------------------+ + +$ vtctlclient OnlineDDL commerce show failed ++-----------------+-------+--------------+-------------+------------+--------------------------------------+----------+---------------------+---------------------+------------------+ +| Tablet | shard | mysql_schema | mysql_table | ddl_action | migration_uuid | strategy | started_timestamp | completed_timestamp | migration_status | ++-----------------+-------+--------------+-------------+------------+--------------------------------------+----------+---------------------+---------------------+------------------+ +| test-0000000301 | 80-c0 | vt_commerce | demo | alter | 8a797518_f25c_11ea_bab4_0242c0a8b007 | online | 2020-09-09 05:23:32 | | failed | +| test-0000000401 | c0- | vt_commerce | demo | alter | 8a797518_f25c_11ea_bab4_0242c0a8b007 | online | 2020-09-09 05:23:32 | | failed | +| test-0000000101 | -40 | vt_commerce | demo | alter | 8a797518_f25c_11ea_bab4_0242c0a8b007 | online | 2020-09-09 05:23:32 | | failed | ++-----------------+-------+--------------+-------------+------------+--------------------------------------+----------+---------------------+---------------------+------------------+ +``` + +The syntax for tracking migrations is: +``` +vtctlclient OnlineDDL show +``` + +## Cancelling a migration + +The user may cancel a migration, as follows: + +- If the migration hasn't started yet (it is `queued` or `ready`), then it is removed from queue and will not be executed. +- If the migration is `running`, then it is forcibly interrupted. The migration is expected to transition to `failed` state. +- In all other cases, cancelling a migration has no effect. + +#### Via VTGate/SQL + +Examples for a single shard cluster: + + +```sql + id: 28 + migration_uuid: aa89f255_8d68_11eb_815f_f875a4d24e90 + keyspace: commerce + shard: 0 + mysql_schema: vt_commerce + mysql_table: corder +migration_statement: alter table corder add column handler_id int not null + strategy: gh-ost + options: + added_timestamp: 2021-03-25 14:50:27 +requested_timestamp: 2021-03-25 14:50:24 + ready_timestamp: 2021-03-25 14:50:31 + started_timestamp: 2021-03-25 14:50:32 + liveness_timestamp: 2021-03-25 14:50:32 +completed_timestamp: NULL + cleanup_timestamp: NULL + migration_status: running +... + +mysql> alter vitess_migration 'aa89f255_8d68_11eb_815f_f875a4d24e90' cancel; +Query OK, 1 row affected (0.01 sec) + +mysql> show vitess_migrations like 'aa89f255_8d68_11eb_815f_f875a4d24e90' \G +*************************** 1. row *************************** + id: 28 + migration_uuid: aa89f255_8d68_11eb_815f_f875a4d24e90 + keyspace: commerce + shard: 0 + mysql_schema: vt_commerce + mysql_table: corder +migration_statement: alter table corder add column handler_id int not null + strategy: gh-ost + options: --throttle-flag-file=/tmp/throttle.flag + added_timestamp: 2021-03-25 14:50:27 +requested_timestamp: 2021-03-25 14:50:24 + ready_timestamp: 2021-03-25 14:50:31 + started_timestamp: 2021-03-25 14:50:32 + liveness_timestamp: 2021-03-25 14:50:32 +completed_timestamp: NULL + cleanup_timestamp: NULL + migration_status: failed +... +``` + +- `alter vitess_migration ... cancel` takes exactly one migration's UUID. +- `alter vitess_migration ... cancel` responds with number of affected migrations. + +#### Via vtctl/ApplySchema + +Examples for a 4-shard cluster: + +``` +vtctlclient OnlineDDL cancel +``` + +Example: + +```shell +$ vtctlclient OnlineDDL commerce show 2201058f_f266_11ea_bab4_0242c0a8b007 ++-----------------+-------+--------------+-------------+------------+--------------------------------------+----------+---------------------+---------------------+------------------+ +| Tablet | shard | mysql_schema | mysql_table | ddl_action | migration_uuid | strategy | started_timestamp | completed_timestamp | migration_status | ++-----------------+-------+--------------+-------------+------------+--------------------------------------+----------+---------------------+---------------------+------------------+ +| test-0000000301 | 80-c0 | vt_commerce | demo | alter | 2201058f_f266_11ea_bab4_0242c0a8b007 | online | 2020-09-09 06:32:31 | | running | +| test-0000000101 | -40 | vt_commerce | demo | alter | 2201058f_f266_11ea_bab4_0242c0a8b007 | online | 2020-09-09 06:32:31 | | running | +| test-0000000401 | c0- | vt_commerce | demo | alter | 2201058f_f266_11ea_bab4_0242c0a8b007 | online | 2020-09-09 06:32:31 | | running | +| test-0000000201 | 40-80 | vt_commerce | demo | alter | 2201058f_f266_11ea_bab4_0242c0a8b007 | online | 2020-09-09 06:32:31 | | running | ++-----------------+-------+--------------+-------------+------------+--------------------------------------+----------+---------------------+---------------------+------------------+ + +$ vtctlclient OnlineDDL commerce cancel 2201058f_f266_11ea_bab4_0242c0a8b007 ++-----------------+--------------+ +| Tablet | RowsAffected | ++-----------------+--------------+ +| test-0000000401 | 1 | +| test-0000000101 | 1 | +| test-0000000201 | 1 | +| test-0000000301 | 1 | ++-----------------+--------------+ + +$ vtctlclient OnlineDDL commerce show 2201058f_f266_11ea_bab4_0242c0a8b007 ++-----------------+-------+--------------+-------------+------------+--------------------------------------+----------+---------------------+---------------------+------------------+ +| Tablet | shard | mysql_schema | mysql_table | ddl_action | migration_uuid | strategy | started_timestamp | completed_timestamp | migration_status | ++-----------------+-------+--------------+-------------+------------+--------------------------------------+----------+---------------------+---------------------+------------------+ +| test-0000000401 | c0- | vt_commerce | demo | alter | 2201058f_f266_11ea_bab4_0242c0a8b007 | online | 2020-09-09 06:32:31 | | failed | +| test-0000000301 | 80-c0 | vt_commerce | demo | alter | 2201058f_f266_11ea_bab4_0242c0a8b007 | online | 2020-09-09 06:32:31 | | failed | +| test-0000000201 | 40-80 | vt_commerce | demo | alter | 2201058f_f266_11ea_bab4_0242c0a8b007 | online | 2020-09-09 06:32:31 | | failed | +| test-0000000101 | -40 | vt_commerce | demo | alter | 2201058f_f266_11ea_bab4_0242c0a8b007 | online | 2020-09-09 06:32:31 | | failed | ++-----------------+-------+--------------+-------------+------------+--------------------------------------+----------+---------------------+---------------------+------------------+ +``` + + +## Cancelling all keyspace migrations + +The user may cancel all migrations in a keyspace. A migration is cancellable if it is in `queued`, `ready` or `running` states, as described previously. It is a high impact operation and should be used with care. + +#### Via VTGate/SQL + +Examples for a single shard cluster: + +```sql +mysql> alter vitess_migration cancel all; +Query OK, 1 row affected (0.02 sec) +``` + +#### Via vtctl/ApplySchema + +Examples for a 4-shard cluster: + +``` +vtctlclient OnlineDDL cancel-all +``` + +Example: + +```shell +$ vtctlclient OnlineDDL commerce show all ++------------------+-------+--------------+-------------+--------------------------------------+----------+-------------------+---------------------+------------------+ +| Tablet | shard | mysql_schema | mysql_table | migration_uuid | strategy | started_timestamp | completed_timestamp | migration_status | ++------------------+-------+--------------+-------------+--------------------------------------+----------+-------------------+---------------------+------------------+ +| zone1-0000000100 | 0 | vt_commerce | corder | 2c581994_353a_11eb_8b72_f875a4d24e90 | online | | | queued | +| zone1-0000000100 | 0 | vt_commerce | corder | 2c6420c9_353a_11eb_8b72_f875a4d24e90 | online | | | queued | +| zone1-0000000100 | 0 | vt_commerce | corder | 2c7040df_353a_11eb_8b72_f875a4d24e90 | online | | | queued | +| zone1-0000000100 | 0 | vt_commerce | corder | 2c7c0572_353a_11eb_8b72_f875a4d24e90 | online | | | queued | +| zone1-0000000100 | 0 | vt_commerce | corder | 2c87f7cd_353a_11eb_8b72_f875a4d24e90 | online | | | queued | ++------------------+-------+--------------+-------------+--------------------------------------+----------+-------------------+---------------------+------------------+ + +$ vtctlclient OnlineDDL commerce cancel-all ++------------------+--------------+ +| Tablet | RowsAffected | ++------------------+--------------+ +| zone1-0000000100 | 5 | ++------------------+--------------+ + + vtctlclient OnlineDDL commerce show all ++------------------+-------+--------------+-------------+--------------------------------------+----------+-------------------+---------------------+------------------+ +| Tablet | shard | mysql_schema | mysql_table | migration_uuid | strategy | started_timestamp | completed_timestamp | migration_status | ++------------------+-------+--------------+-------------+--------------------------------------+----------+-------------------+---------------------+------------------+ +| zone1-0000000100 | 0 | vt_commerce | corder | 2c581994_353a_11eb_8b72_f875a4d24e90 | online | | | cancelled | +| zone1-0000000100 | 0 | vt_commerce | corder | 2c6420c9_353a_11eb_8b72_f875a4d24e90 | online | | | cancelled | +| zone1-0000000100 | 0 | vt_commerce | corder | 2c7040df_353a_11eb_8b72_f875a4d24e90 | online | | | cancelled | +| zone1-0000000100 | 0 | vt_commerce | corder | 2c7c0572_353a_11eb_8b72_f875a4d24e90 | online | | | cancelled | +| zone1-0000000100 | 0 | vt_commerce | corder | 2c87f7cd_353a_11eb_8b72_f875a4d24e90 | online | | | cancelled | ++------------------+-------+--------------+-------------+--------------------------------------+----------+-------------------+---------------------+------------------+ +``` + +## Retrying a migration + +The user may retry running a migration. If the migration is in `failed` or in `cancelled` state, Vitess will re-run the migration, with exact same arguments as previously intended. If the migration is in any other state, `retry` does nothing. + +It is not possible to retry a migration with different options. e.g. if the user initially runs `ALTER TABLE demo MODIFY id BIGINT` with `@@ddl_strategy='gh-ost --max-load Threads_running=200'` and the migration fails, retrying it will use exact same options. It is not possible to retry with `@@ddl_strategy='gh-ost --max-load Threads_running=500'`. + +#### Via VTGate/SQL + +Examples for a single shard cluster: + +```sql +*************************** 1. row *************************** + id: 28 + migration_uuid: aa89f255_8d68_11eb_815f_f875a4d24e90 + keyspace: commerce + shard: 0 + mysql_schema: vt_commerce + mysql_table: corder +migration_statement: alter table corder add column handler_id int not null + strategy: gh-ost + options: --throttle-flag-file=/tmp/throttle.flag + added_timestamp: 2021-03-25 14:50:27 +requested_timestamp: 2021-03-25 14:50:24 + ready_timestamp: 2021-03-25 14:56:22 + started_timestamp: 2021-03-25 14:56:22 + liveness_timestamp: 2021-03-25 14:56:22 +completed_timestamp: NULL + cleanup_timestamp: NULL + migration_status: failed +... + +mysql> alter vitess_migration 'aa89f255_8d68_11eb_815f_f875a4d24e90' retry; +Query OK, 1 row affected (0.00 sec) + +mysql> show vitess_migrations like 'aa89f255_8d68_11eb_815f_f875a4d24e90' \G +*************************** 1. row *************************** + id: 28 + migration_uuid: aa89f255_8d68_11eb_815f_f875a4d24e90 + keyspace: commerce + shard: 0 + mysql_schema: vt_commerce + mysql_table: corder +migration_statement: alter table corder add column handler_id int not null + strategy: gh-ost + options: --throttle-flag-file=/tmp/throttle.flag + added_timestamp: 2021-03-25 14:50:27 +requested_timestamp: 2021-03-25 14:50:24 + ready_timestamp: 2021-03-25 14:56:42 + started_timestamp: 2021-03-25 14:56:42 + liveness_timestamp: 2021-03-25 14:56:42 +completed_timestamp: NULL + cleanup_timestamp: NULL + migration_status: running +... +``` +- `alter vitess_migration ... cancel` takes exactly one migration's UUID. +- `alter vitess_migration ... cancel` responds with number of affected migrations. + +#### Via vtctl/ApplySchema + +Examples for a 4-shard cluster: + + +```shell +$ vtctlclient OnlineDDL commerce show 2201058f_f266_11ea_bab4_0242c0a8b007 ++-----------------+-------+--------------+-------------+------------+--------------------------------------+----------+---------------------+---------------------+------------------+ +| Tablet | shard | mysql_schema | mysql_table | ddl_action | migration_uuid | strategy | started_timestamp | completed_timestamp | migration_status | ++-----------------+-------+--------------+-------------+------------+--------------------------------------+----------+---------------------+---------------------+------------------+ +| test-0000000401 | c0- | vt_commerce | demo | alter | 2201058f_f266_11ea_bab4_0242c0a8b007 | online | 2020-09-09 06:32:31 | | failed | +| test-0000000301 | 80-c0 | vt_commerce | demo | alter | 2201058f_f266_11ea_bab4_0242c0a8b007 | online | 2020-09-09 06:32:31 | | failed | +| test-0000000201 | 40-80 | vt_commerce | demo | alter | 2201058f_f266_11ea_bab4_0242c0a8b007 | online | 2020-09-09 06:32:31 | | failed | +| test-0000000101 | -40 | vt_commerce | demo | alter | 2201058f_f266_11ea_bab4_0242c0a8b007 | online | 2020-09-09 06:32:31 | | failed | ++-----------------+-------+--------------+-------------+------------+--------------------------------------+----------+---------------------+---------------------+------------------+ + +$ vtctlclient OnlineDDL commerce retry 2201058f_f266_11ea_bab4_0242c0a8b007 ++-----------------+--------------+ +| Tablet | RowsAffected | ++-----------------+--------------+ +| test-0000000101 | 1 | +| test-0000000201 | 1 | +| test-0000000301 | 1 | +| test-0000000401 | 1 | ++-----------------+--------------+ + +$ vtctlclient OnlineDDL commerce show 2201058f_f266_11ea_bab4_0242c0a8b007 ++-----------------+-------+--------------+-------------+------------+--------------------------------------+----------+-------------------+---------------------+------------------+ +| Tablet | shard | mysql_schema | mysql_table | ddl_action | migration_uuid | strategy | started_timestamp | completed_timestamp | migration_status | ++-----------------+-------+--------------+-------------+------------+--------------------------------------+----------+-------------------+---------------------+------------------+ +| test-0000000201 | 40-80 | vt_commerce | demo | alter | 2201058f_f266_11ea_bab4_0242c0a8b007 | online | | | queued | +| test-0000000101 | -40 | vt_commerce | demo | alter | 2201058f_f266_11ea_bab4_0242c0a8b007 | online | | | queued | +| test-0000000301 | 80-c0 | vt_commerce | demo | alter | 2201058f_f266_11ea_bab4_0242c0a8b007 | online | | | queued | +| test-0000000401 | c0- | vt_commerce | demo | alter | 2201058f_f266_11ea_bab4_0242c0a8b007 | online | | | queued | ++-----------------+-------+--------------+-------------+------------+--------------------------------------+----------+-------------------+---------------------+------------------+ + +$ vtctlclient OnlineDDL commerce show 2201058f_f266_11ea_bab4_0242c0a8b007 ++-----------------+-------+--------------+-------------+------------+--------------------------------------+----------+---------------------+---------------------+------------------+ +| Tablet | shard | mysql_schema | mysql_table | ddl_action | migration_uuid | strategy | started_timestamp | completed_timestamp | migration_status | ++-----------------+-------+--------------+-------------+------------+--------------------------------------+----------+---------------------+---------------------+------------------+ +| test-0000000101 | -40 | vt_commerce | demo | alter | 2201058f_f266_11ea_bab4_0242c0a8b007 | online | 2020-09-09 06:37:33 | | running | +| test-0000000401 | c0- | vt_commerce | demo | alter | 2201058f_f266_11ea_bab4_0242c0a8b007 | online | 2020-09-09 06:37:33 | | running | +| test-0000000201 | 40-80 | vt_commerce | demo | alter | 2201058f_f266_11ea_bab4_0242c0a8b007 | online | 2020-09-09 06:37:33 | | running | +| test-0000000301 | 80-c0 | vt_commerce | demo | alter | 2201058f_f266_11ea_bab4_0242c0a8b007 | online | 2020-09-09 06:37:33 | | running | ++-----------------+-------+--------------+-------------+------------+--------------------------------------+----------+---------------------+---------------------+------------------+ +``` + +## Reverting a migration + +Vitess offers _lossless revert_ for online schema migrations: the user may regret a table migration after completion, and roll back the table's schema to previous state _without loss of data_. See [Revertible Migrations](../revertible-migrations/). + +### Via VTGate/SQL + +Examples for a single shard cluster: + +```sql +mysql> show create table corder\G + +Create Table: CREATE TABLE `corder` ( + `order_id` bigint(20) NOT NULL AUTO_INCREMENT, + `customer_id` bigint(20) DEFAULT NULL, + `sku` varbinary(128) DEFAULT NULL, + `price` bigint(20) DEFAULT NULL, + `ts` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP, + PRIMARY KEY (`order_id`) +) ENGINE=InnoDB DEFAULT CHARSET=utf8 +1 row in set (0.01 sec) + +mysql> alter table corder drop column ts, add key customer_idx(customer_id); ++--------------------------------------+ +| uuid | ++--------------------------------------+ +| 1a689113_8d77_11eb_815f_f875a4d24e90 | ++--------------------------------------+ + +mysql> show create table corder\G + +Create Table: CREATE TABLE `corder` ( + `order_id` bigint(20) NOT NULL AUTO_INCREMENT, + `customer_id` bigint(20) DEFAULT NULL, + `sku` varbinary(128) DEFAULT NULL, + `price` bigint(20) DEFAULT NULL, + PRIMARY KEY (`order_id`), + KEY `customer_idx` (`customer_id`) +) ENGINE=InnoDB DEFAULT CHARSET=utf8 +1 row in set (0.00 sec) + +mysql> revert vitess_migration '1a689113_8d77_11eb_815f_f875a4d24e90'; ++--------------------------------------+ +| uuid | ++--------------------------------------+ +| a02e6612_8d79_11eb_815f_f875a4d24e90 | ++--------------------------------------+ + +mysql> show create table corder\G + +Create Table: CREATE TABLE `corder` ( + `order_id` bigint(20) NOT NULL AUTO_INCREMENT, + `customer_id` bigint(20) DEFAULT NULL, + `sku` varbinary(128) DEFAULT NULL, + `price` bigint(20) DEFAULT NULL, + `ts` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP, + PRIMARY KEY (`order_id`) +) ENGINE=InnoDB DEFAULT CHARSET=utf8 +``` + +- A `revert` is its own migration, hence has its own UUID + +### Via vtctl/ApplySchema + +``` +$ vtctlclient OnlineDDL commerce revert 2201058f_f266_11ea_bab4_0242c0a8b007 +``` + +## VExec commands for greater control and visibility + +`vtctlclient OnlineDDL` command should provide with most needs. However, Vitess gives the user greater control through the `VExec` command and via SQL queries. + +For schema migrations, Vitess allows operations on the virtual table `_vt.schema_migrations`. Queries on this virtual table scatter to the underlying tablets and gather or manipulate data on their own, private backend tables (which incidentally are called by the same name). `VExec` only allows specific types of queries on that table. + +- `SELECT`: you may SELECT any column, or `SELECT *`. `vtctlclient OnlineDDL show` commands only present with a subset of columns, and so running ` VExec` `SELECT` provides greater visibility. Some columns that are not shown are: + - `log_path`: tablet server and path where migration logs are. + - `artifacts`: tables created by the migration. This can be used to determine which tables need cleanup. + - `alter`: the exact `alter` statement used by the migration + - `options`: any options passed by the user (e.g. `--max-load=Threads_running=200`) + - Various timestamps indicating the migration progress + Aggregate functions do not work as expected and should be avoided. `LIMIT` and `OFFSET` are not supported. +- `UPDATE`: you may directly update the status of a migration. You may only change status into `cancel` or `retry`, which Vitess interprets similarly to a `vtctlclient OnlineDDL cancel/retry` command. However, you get greater control as you may filter on a specific `shard`. +- `DELETE`: unsupported +- `INSERT`: unsupported, used internally only to advertise new migration requests to the tablets. + +The syntax to run `VExec` queries is: + +``` +vtctlclient VExec . "" +``` + +Examples: + +```shell + +$ vtctlclient VExec commerce.2201058f_f266_11ea_bab4_0242c0a8b007 "select * from _vt.schema_migrations" + +$ vtctlclient VExec commerce.91b5c953-e1e2-11ea-a097-f875a4d24e90 "update _vt.schema_migrations set migration_status='retry'" + +$ vtctlclient VExec commerce.91b5c953-e1e2-11ea-a097-f875a4d24e90 "update _vt.schema_migrations set migration_status='retry' where shard='40-80' +``` + +```shell +$ vtctlclient VExec commerce.2201058f_f266_11ea_bab4_0242c0a8b007 "select shard, mysql_table, migration_uuid, started_timestamp, completed_timestamp, migration_status from _vt.schema_migrations" ++-----------------+-------+-------------+--------------------------------------+---------------------+---------------------+------------------+ +| Tablet | shard | mysql_table | migration_uuid | started_timestamp | completed_timestamp | migration_status | ++-----------------+-------+-------------+--------------------------------------+---------------------+---------------------+------------------+ +| test-0000000301 | 80-c0 | demo | 2201058f_f266_11ea_bab4_0242c0a8b007 | 2020-09-09 06:37:33 | | failed | +| test-0000000101 | -40 | demo | 2201058f_f266_11ea_bab4_0242c0a8b007 | 2020-09-09 06:37:33 | | failed | +| test-0000000201 | 40-80 | demo | 2201058f_f266_11ea_bab4_0242c0a8b007 | 2020-09-09 08:31:47 | | failed | +| test-0000000401 | c0- | demo | 2201058f_f266_11ea_bab4_0242c0a8b007 | 2020-09-09 06:37:33 | | failed | ++-----------------+-------+-------------+--------------------------------------+---------------------+---------------------+------------------+ + +$ vtctlclient VExec commerce.2201058f_f266_11ea_bab4_0242c0a8b007 "update _vt.schema_migrations set migration_status='retry' where migration_uuid='2201058f_f266_11ea_bab4_0242c0a8b007' and shard='40-80'" ++-----------------+--------------+ +| Tablet | RowsAffected | ++-----------------+--------------+ +| test-0000000201 | 1 | ++-----------------+--------------+ + +$ vtctlclient VExec commerce.2201058f_f266_11ea_bab4_0242c0a8b007 "select shard, mysql_table, migration_uuid, started_timestamp, completed_timestamp, migration_status from _vt.schema_migrations" ++-----------------+-------+-------------+--------------------------------------+---------------------+---------------------+------------------+ +| Tablet | shard | mysql_table | migration_uuid | started_timestamp | completed_timestamp | migration_status | ++-----------------+-------+-------------+--------------------------------------+---------------------+---------------------+------------------+ +| test-0000000301 | 80-c0 | demo | 2201058f_f266_11ea_bab4_0242c0a8b007 | 2020-09-09 06:37:33 | | failed | +| test-0000000201 | 40-80 | demo | 2201058f_f266_11ea_bab4_0242c0a8b007 | 2020-09-09 08:34:59 | | running | +| test-0000000101 | -40 | demo | 2201058f_f266_11ea_bab4_0242c0a8b007 | 2020-09-09 06:37:33 | | failed | +| test-0000000401 | c0- | demo | 2201058f_f266_11ea_bab4_0242c0a8b007 | 2020-09-09 06:37:33 | | failed | ++-----------------+-------+-------------+--------------------------------------+---------------------+---------------------+------------------+ + +$ vtctlclient VExec commerce.2201058f_f266_11ea_bab4_0242c0a8b007 "update _vt.schema_migrations set migration_status='cancel' where migration_uuid='2201058f_f266_11ea_bab4_0242c0a8b007' and shard='40-80'" ++-----------------+--------------+ +| Tablet | RowsAffected | ++-----------------+--------------+ +| test-0000000201 | 1 | ++-----------------+--------------+ + +$ vtctlclient VExec commerce.2201058f_f266_11ea_bab4_0242c0a8b007 "select shard, mysql_table, migration_uuid, started_timestamp, completed_timestamp, migration_status from _vt.schema_migrations" ++-----------------+-------+-------------+--------------------------------------+---------------------+---------------------+------------------+ +| Tablet | shard | mysql_table | migration_uuid | started_timestamp | completed_timestamp | migration_status | ++-----------------+-------+-------------+--------------------------------------+---------------------+---------------------+------------------+ +| test-0000000401 | c0- | demo | 2201058f_f266_11ea_bab4_0242c0a8b007 | 2020-09-09 06:37:33 | | failed | +| test-0000000101 | -40 | demo | 2201058f_f266_11ea_bab4_0242c0a8b007 | 2020-09-09 06:37:33 | | failed | +| test-0000000201 | 40-80 | demo | 2201058f_f266_11ea_bab4_0242c0a8b007 | 2020-09-09 08:34:59 | | failed | +| test-0000000301 | 80-c0 | demo | 2201058f_f266_11ea_bab4_0242c0a8b007 | 2020-09-09 06:37:33 | | failed | ++-----------------+-------+-------------+--------------------------------------+---------------------+---------------------+------------------+ + +$ vtctlclient VExec commerce.2201058f_f266_11ea_bab4_0242c0a8b007 "update _vt.schema_migrations set migration_status='cancel' where migration_uuid='2201058f_f266_11ea_bab4_0242c0a8b007' and shard='40-80'" + + +$ vtctlclient VExec commerce.2201058f_f266_11ea_bab4_0242c0a8b007 "select shard, log_path from _vt.schema_migrations" ++-----------------+-------+-----------------------------------------------------------------------------+ +| Tablet | shard | log_path | ++-----------------+-------+-----------------------------------------------------------------------------+ +| test-0000000201 | 40-80 | 11ac2af6e63e:/tmp/online-ddl-2201058f_f266_11ea_bab4_0242c0a8b007-657478384 | +| test-0000000101 | -40 | e779a82d35d7:/tmp/online-ddl-2201058f_f266_11ea_bab4_0242c0a8b007-901629215 | +| test-0000000401 | c0- | 5aad1249ab91:/tmp/online-ddl-2201058f_f266_11ea_bab4_0242c0a8b007-039568897 | +| test-0000000301 | 80-c0 | 5e7c662679d3:/tmp/online-ddl-2201058f_f266_11ea_bab4_0242c0a8b007-532703073 | ++-----------------+-------+-----------------------------------------------------------------------------+ +``` + + diff --git a/content/en/docs/user-guides/schema-changes/ddl-strategies.md b/content/en/docs/user-guides/schema-changes/ddl-strategies.md index c4ba524e4..473d9e631 100644 --- a/content/en/docs/user-guides/schema-changes/ddl-strategies.md +++ b/content/en/docs/user-guides/schema-changes/ddl-strategies.md @@ -1,6 +1,6 @@ --- title: Online DDL strategies -weight: 2 +weight: 3 aliases: ['/docs/user-guides/schema-changes/ddl-strategies/'] --- diff --git a/content/en/docs/user-guides/schema-changes/declarative-migrations.md b/content/en/docs/user-guides/schema-changes/declarative-migrations.md index 19b94398e..30b8016ae 100644 --- a/content/en/docs/user-guides/schema-changes/declarative-migrations.md +++ b/content/en/docs/user-guides/schema-changes/declarative-migrations.md @@ -1,6 +1,6 @@ --- title: Declarative migrations -weight: 2 +weight: 5 aliases: ['/docs/user-guides/schema-changes/declarative-migrations/'] --- diff --git a/content/en/docs/user-guides/schema-changes/managed-online-schema-changes.md b/content/en/docs/user-guides/schema-changes/managed-online-schema-changes.md index 48f884d5d..e560b1440 100644 --- a/content/en/docs/user-guides/schema-changes/managed-online-schema-changes.md +++ b/content/en/docs/user-guides/schema-changes/managed-online-schema-changes.md @@ -33,9 +33,11 @@ As general overview: The standard MySQL syntax for `CREATE`, `ALTER` and `DROP` is supported. + + ### ALTER TABLE -Use the standard MySQL `ALTER TABLE` syntax to run online DDL. Whether your schema migration runs synchronously (the default MySQL behavior) or asynchronously (aka online), is determined by the value of command line flags or a session variable. +Use the standard MySQL `ALTER TABLE` syntax to run online DDL. Whether your schema migration runs synchronously (the default MySQL behavior) or asynchronously (aka online), is determined by `ddl_strategy`. We assume we have a keyspace (schema) called `commerce`, with a table called `demo`, that has the following definition: @@ -88,84 +90,51 @@ You will set either `@@ddl_strategy` session variable, or `-ddl_strategy` comman `CREATE` and `DROP` statements run in the same way for `"online"`, `"gh-ost"` and `"pt-osc"` strategies, and we consider them all to be _online_. -## ApplySchema - -You may use `vtctl` or `vtctlclient` (the two are interchangeable for the purpose of this document) to apply schema changes. The `ApplySchema` command supports both synchronous and online schema migrations. To run an online schema migration you will supply the `-ddl_strategy` command line flag: - -```shell -$ vtctlclient ApplySchema -ddl_strategy "online" -sql "ALTER TABLE demo MODIFY id bigint UNSIGNED" commerce -a2994c92_f1d4_11ea_afa3_f875a4d24e90 -``` - -Notice how `ApplySchema` responds to an online DDL request with a UUID output. When the user indicates online schema change (aka online DDL), `vtctl` registers an online-DDL request with global `topo`. This generates a job ID for tracking. `vtctl` does not try to resolve the shards nor the `primary` tablets. The command returns immediately, without waiting for the migration(s) to start. - -The command prints a job ID for each of the DDL statements. In the above we issue a single DDL statemnt, and `vtctl` responds with a single job ID (`a2994c92_f1d4_11ea_afa3_f875a4d24e90`) - -If we immediately run `SHOW CREATE TABLE`, we are likely to still see the old schema: -```sql -SHOW CREATE TABLE demo; +## Running, tracking and controlling Online DDL -CREATE TABLE `demo` ( - `id` int NOT NULL, - `status` varchar(32) DEFAULT NULL, - PRIMARY KEY (`id`) -) ENGINE=InnoDB -``` - -That's because the migration is to be processed and executed in the future. We discuss how the migration jobs get scheduled and executed shortly. We will use the job ID for tracking. - -`ApplySchema` will have Vitess run some validations: - -```shell -$ vtctlclient ApplySchema -ddl_strategy "gh-ost" -sql "ALTER TABLE demo add column status int" commerce -E0908 16:17:07.651284 3739130 main.go:67] remote error: rpc error: code = Unknown desc = schema change failed, ExecuteResult: { - "FailedShards": null, - "SuccessShards": null, - "CurSQLIndex": 0, - "Sqls": [ - "ALTER TABLE demo add column status int" - ], - "ExecutorErr": "rpc error: code = Unknown desc = TabletManager.PreflightSchema on zone1-0000000100 error: /usr/bin/mysql: exit status 1, output: ERROR 1060 (42S21) at line 3: Duplicate column name 'status'\n: /usr/bin/mysql: exit status 1, output: ERROR 1060 (42S21) at line 3: Duplicate column name 'status'\n", - "TotalTimeSpent": 144283260 -} -``` +Vitess provides two interfaces to interacting with Online DDL: -Vitess was able to determine that the migration is invalid because a column named `status` already exists. `vtctld` generates no job ID, and does not persist any migration request. +- SQL commands, via `VTGate` +- Command line interface, via `vtctl` -## VTGate +Supported interactions are: -When connected to `VTGate`, your schema migration strategy is determined by: +- [Running migrations](../audit-and-control/#running-migrations) (submitting Online DDL requests) +- [Tracking migrations](../audit-and-control/#tracking-migrations) +- [Cancelling a migration](../audit-and-control/#cancelling-a-migration) +- [Cancelling all pending migrations](../audit-and-control/#cancelling-all-keyspace-migrations) +- [Retrying a migration](../audit-and-control/#retrying-a-migration) +- [Reverting a migration](../audit-and-control/#reverting-a-migration) -- `-ddl_strategy` command line flag, or: -- `@@ddl_strategy` session variable. +See [Audit and Control](../audit-and-control/) for a detailed breakdown. As quick examples: -As a simple example, you may control the execution of a migration as follows: +#### Executing an Online DDL via VTGate/SQL -```shell -$ mysql -h 127.0.0.1 -P 15306 commerce -Welcome to the MySQL monitor. Commands end with ; or \g. +```sql +mysql> set @@ddl_strategy='online'; -mysql> SET @@ddl_strategy='pt-osc'; -Query OK, 0 rows affected (0.00 sec) +mysql> alter table corder add column ts timestamp not null default current_timestamp; ++--------------------------------------+ +| uuid | ++--------------------------------------+ +| bf4598ab_8d55_11eb_815f_f875a4d24e90 | ++--------------------------------------+ -mysql> ALTER TABLE demo ADD COLUMN sample INT; +mysql> drop table customer; +--------------------------------------+ | uuid | +--------------------------------------+ -| fa2fb689_f1d5_11ea_859e_f875a4d24e90 | +| 6848c1a4_8d57_11eb_815f_f875a4d24e90 | +--------------------------------------+ -1 row in set (0.00 sec) ``` -Just like in the `ApplySchema` example, `VTGate` identifies that this is an online schema change request, and persists it in global `topo`, returning a job ID for tracking. Migration does not start immediately. +#### Executing an Online DDL via vtctl/ApplySchema -`@@ddl_strategy` behaves like a MySQL session variable, though is only recognized by `VTGate`. Setting `@@ddl_strategy` only applies to that same connection and does not affect other connections. You may subsequently set `@@ddl_strategy` to different value. - -The initial value for `@@ddl_strategy` is derived from the `vtgate -ddl_strategy` command line flag. Examples: +```shell +$ vtctlclient ApplySchema -ddl_strategy "online" -sql "ALTER TABLE demo MODIFY id bigint UNSIGNED" commerce +a2994c92_f1d4_11ea_afa3_f875a4d24e90 +``` -- If you run `vtgate` without `-ddl_strategy`, then `@@ddl_strategy` defaults to `'direct'`, which implies schema migrations are synchronous. You will need to `set @@ddl_strategy='gh-ost'` to run followup `ALTER TABLE` statements via `gh-ost`. -- If you run `vtgate -ddl_strategy "gh-ost"`, then `@@ddl_strategy` defaults to `'gh-ost'` in each new session. Any `ALTER TABLE` will run via `gh-ost`. You may `set @@ddl_strategy='pt-osc'` to make migrations run through `pt-online-schema-change`, or `set @@ddl_strategy='direct'` to run migrations synchronously. - ## Migration flow and states We highlight how Vitess manages migrations internally, and explain what states a migration goes through. @@ -212,216 +181,6 @@ At this time, there are no automated retries. For example, a failover on a shard Example: `vtgate -enable_online_ddl=false` to disable access to Online DDL via `VTGate`. -## Tracking migrations - -You may track the status of a single migration, of all or recent migrations, or of migrations in a specific state. Examples: - -```shell --$ vtctlclient OnlineDDL commerce show ab3ffdd5_f25c_11ea_bab4_0242c0a8b007 -+-----------------+-------+--------------+-------------+------------+--------------------------------------+----------+---------------------+---------------------+------------------+ -| Tablet | shard | mysql_schema | mysql_table | ddl_action | migration_uuid | strategy | started_timestamp | completed_timestamp | migration_status | -+-----------------+-------+--------------+-------------+------------+--------------------------------------+----------+---------------------+---------------------+------------------+ -| test-0000000201 | 40-80 | vt_commerce | demo | alter | ab3ffdd5_f25c_11ea_bab4_0242c0a8b007 | online | 2020-09-09 05:24:33 | 2020-09-09 05:24:34 | complete | -| test-0000000301 | 80-c0 | vt_commerce | demo | alter | ab3ffdd5_f25c_11ea_bab4_0242c0a8b007 | online | 2020-09-09 05:25:13 | 2020-09-09 05:25:14 | complete | -| test-0000000401 | c0- | vt_commerce | demo | alter | ab3ffdd5_f25c_11ea_bab4_0242c0a8b007 | online | 2020-09-09 05:25:13 | 2020-09-09 05:25:14 | complete | -| test-0000000101 | -40 | vt_commerce | demo | alter | ab3ffdd5_f25c_11ea_bab4_0242c0a8b007 | online | 2020-09-09 05:25:13 | 2020-09-09 05:25:14 | complete | -+-----------------+-------+--------------+-------------+------------+--------------------------------------+----------+---------------------+---------------------+------------------+ - -$ vtctlclient OnlineDDL commerce show 8a797518_f25c_11ea_bab4_0242c0a8b007 -+-----------------+-------+--------------+-------------+------------+--------------------------------------+----------+---------------------+---------------------+------------------+ -| Tablet | shard | mysql_schema | mysql_table | ddl_action | migration_uuid | strategy | started_timestamp | completed_timestamp | migration_status | -+-----------------+-------+--------------+-------------+------------+--------------------------------------+----------+---------------------+---------------------+------------------+ -| test-0000000401 | c0- | vt_commerce | demo | alter | 8a797518_f25c_11ea_bab4_0242c0a8b007 | online | 2020-09-09 05:23:32 | | running | -| test-0000000201 | 40-80 | vt_commerce | demo | alter | 8a797518_f25c_11ea_bab4_0242c0a8b007 | online | 2020-09-09 05:23:32 | 2020-09-09 05:23:33 | complete | -| test-0000000301 | 80-c0 | vt_commerce | demo | alter | 8a797518_f25c_11ea_bab4_0242c0a8b007 | online | 2020-09-09 05:23:32 | | running | -| test-0000000101 | -40 | vt_commerce | demo | alter | 8a797518_f25c_11ea_bab4_0242c0a8b007 | online | 2020-09-09 05:23:32 | | running | -+-----------------+-------+--------------+-------------+------------+--------------------------------------+----------+---------------------+---------------------+------------------+ - -$ vtctlclient OnlineDDL commerce show 8a797518_f25c_11ea_bab4_0242c0a8b007 -+-----------------+-------+--------------+-------------+------------+--------------------------------------+----------+---------------------+---------------------+------------------+ -| Tablet | shard | mysql_schema | mysql_table | ddl_action | migration_uuid | strategy | started_timestamp | completed_timestamp | migration_status | -+-----------------+-------+--------------+-------------+------------+--------------------------------------+----------+---------------------+---------------------+------------------+ -| test-0000000401 | c0- | vt_commerce | demo | alter | 8a797518_f25c_11ea_bab4_0242c0a8b007 | online | 2020-09-09 05:23:32 | | failed | -| test-0000000101 | -40 | vt_commerce | demo | alter | 8a797518_f25c_11ea_bab4_0242c0a8b007 | online | 2020-09-09 05:23:32 | | failed | -| test-0000000301 | 80-c0 | vt_commerce | demo | alter | 8a797518_f25c_11ea_bab4_0242c0a8b007 | online | 2020-09-09 05:23:32 | | failed | -| test-0000000201 | 40-80 | vt_commerce | demo | alter | 8a797518_f25c_11ea_bab4_0242c0a8b007 | online | 2020-09-09 05:23:32 | 2020-09-09 05:23:33 | complete | -+-----------------+-------+--------------+-------------+------------+--------------------------------------+----------+---------------------+---------------------+------------------+ - -$ vtctlclient OnlineDDL commerce show recent -+-----------------+-------+--------------+-------------+------------+--------------------------------------+----------+---------------------+---------------------+------------------+ -| Tablet | shard | mysql_schema | mysql_table | ddl_action | migration_uuid | strategy | started_timestamp | completed_timestamp | migration_status | -+-----------------+-------+--------------+-------------+------------+--------------------------------------+----------+---------------------+---------------------+------------------+ -| test-0000000201 | 40-80 | vt_commerce | demo | alter | 63b5db0c_f25c_11ea_bab4_0242c0a8b007 | online | 2020-09-09 05:22:41 | 2020-09-09 05:22:42 | complete | -| test-0000000201 | 40-80 | vt_commerce | demo | alter | 8a797518_f25c_11ea_bab4_0242c0a8b007 | online | 2020-09-09 05:23:32 | 2020-09-09 05:23:33 | complete | -| test-0000000201 | 40-80 | vt_commerce | demo | alter | ab3ffdd5_f25c_11ea_bab4_0242c0a8b007 | online | 2020-09-09 05:24:33 | 2020-09-09 05:24:34 | complete | -| test-0000000301 | 80-c0 | vt_commerce | demo | alter | 63b5db0c_f25c_11ea_bab4_0242c0a8b007 | online | 2020-09-09 05:22:41 | 2020-09-09 05:22:42 | complete | -| test-0000000301 | 80-c0 | vt_commerce | demo | alter | 8a797518_f25c_11ea_bab4_0242c0a8b007 | online | 2020-09-09 05:23:32 | | failed | -| test-0000000301 | 80-c0 | vt_commerce | demo | alter | ab3ffdd5_f25c_11ea_bab4_0242c0a8b007 | online | 2020-09-09 05:25:13 | 2020-09-09 05:25:14 | complete | -| test-0000000401 | c0- | vt_commerce | demo | alter | 63b5db0c_f25c_11ea_bab4_0242c0a8b007 | online | 2020-09-09 05:22:41 | 2020-09-09 05:22:42 | complete | -| test-0000000401 | c0- | vt_commerce | demo | alter | 8a797518_f25c_11ea_bab4_0242c0a8b007 | online | 2020-09-09 05:23:32 | | failed | -| test-0000000401 | c0- | vt_commerce | demo | alter | ab3ffdd5_f25c_11ea_bab4_0242c0a8b007 | online | 2020-09-09 05:25:13 | 2020-09-09 05:25:14 | complete | -| test-0000000101 | -40 | vt_commerce | demo | alter | 63b5db0c_f25c_11ea_bab4_0242c0a8b007 | online | 2020-09-09 05:22:41 | 2020-09-09 05:22:42 | complete | -| test-0000000101 | -40 | vt_commerce | demo | alter | 8a797518_f25c_11ea_bab4_0242c0a8b007 | online | 2020-09-09 05:23:32 | | failed | -| test-0000000101 | -40 | vt_commerce | demo | alter | ab3ffdd5_f25c_11ea_bab4_0242c0a8b007 | online | 2020-09-09 05:25:13 | 2020-09-09 05:25:14 | complete | -+-----------------+-------+--------------+-------------+------------+--------------------------------------+----------+---------------------+---------------------+------------------+ - -$ vtctlclient OnlineDDL commerce show failed -+-----------------+-------+--------------+-------------+------------+--------------------------------------+----------+---------------------+---------------------+------------------+ -| Tablet | shard | mysql_schema | mysql_table | ddl_action | migration_uuid | strategy | started_timestamp | completed_timestamp | migration_status | -+-----------------+-------+--------------+-------------+------------+--------------------------------------+----------+---------------------+---------------------+------------------+ -| test-0000000301 | 80-c0 | vt_commerce | demo | alter | 8a797518_f25c_11ea_bab4_0242c0a8b007 | online | 2020-09-09 05:23:32 | | failed | -| test-0000000401 | c0- | vt_commerce | demo | alter | 8a797518_f25c_11ea_bab4_0242c0a8b007 | online | 2020-09-09 05:23:32 | | failed | -| test-0000000101 | -40 | vt_commerce | demo | alter | 8a797518_f25c_11ea_bab4_0242c0a8b007 | online | 2020-09-09 05:23:32 | | failed | -+-----------------+-------+--------------+-------------+------------+--------------------------------------+----------+---------------------+---------------------+------------------+ -``` - -The syntax for tracking migrations is: -``` -vtctlclient OnlineDDL show -``` - -## Cancelling a migration - -The user may cancel a migration, as follows: - -- If the migration hasn't started yet (it is `queued` or `ready`), then it is removed from queue and will not be executed. -- If the migration is `running`, then it is forcibly interrupted. The migration is expected to transition to `failed` state. -- In all other cases, cancelling a migration has no effect. - -The syntax to cancelling a migration is: - -``` -vtctlclient OnlineDDL cancel -``` - -Example: - -```shell -$ vtctlclient OnlineDDL commerce show 2201058f_f266_11ea_bab4_0242c0a8b007 -+-----------------+-------+--------------+-------------+------------+--------------------------------------+----------+---------------------+---------------------+------------------+ -| Tablet | shard | mysql_schema | mysql_table | ddl_action | migration_uuid | strategy | started_timestamp | completed_timestamp | migration_status | -+-----------------+-------+--------------+-------------+------------+--------------------------------------+----------+---------------------+---------------------+------------------+ -| test-0000000301 | 80-c0 | vt_commerce | demo | alter | 2201058f_f266_11ea_bab4_0242c0a8b007 | online | 2020-09-09 06:32:31 | | running | -| test-0000000101 | -40 | vt_commerce | demo | alter | 2201058f_f266_11ea_bab4_0242c0a8b007 | online | 2020-09-09 06:32:31 | | running | -| test-0000000401 | c0- | vt_commerce | demo | alter | 2201058f_f266_11ea_bab4_0242c0a8b007 | online | 2020-09-09 06:32:31 | | running | -| test-0000000201 | 40-80 | vt_commerce | demo | alter | 2201058f_f266_11ea_bab4_0242c0a8b007 | online | 2020-09-09 06:32:31 | | running | -+-----------------+-------+--------------+-------------+------------+--------------------------------------+----------+---------------------+---------------------+------------------+ - -$ vtctlclient OnlineDDL commerce cancel 2201058f_f266_11ea_bab4_0242c0a8b007 -+-----------------+--------------+ -| Tablet | RowsAffected | -+-----------------+--------------+ -| test-0000000401 | 1 | -| test-0000000101 | 1 | -| test-0000000201 | 1 | -| test-0000000301 | 1 | -+-----------------+--------------+ - -$ vtctlclient OnlineDDL commerce show 2201058f_f266_11ea_bab4_0242c0a8b007 -+-----------------+-------+--------------+-------------+------------+--------------------------------------+----------+---------------------+---------------------+------------------+ -| Tablet | shard | mysql_schema | mysql_table | ddl_action | migration_uuid | strategy | started_timestamp | completed_timestamp | migration_status | -+-----------------+-------+--------------+-------------+------------+--------------------------------------+----------+---------------------+---------------------+------------------+ -| test-0000000401 | c0- | vt_commerce | demo | alter | 2201058f_f266_11ea_bab4_0242c0a8b007 | online | 2020-09-09 06:32:31 | | failed | -| test-0000000301 | 80-c0 | vt_commerce | demo | alter | 2201058f_f266_11ea_bab4_0242c0a8b007 | online | 2020-09-09 06:32:31 | | failed | -| test-0000000201 | 40-80 | vt_commerce | demo | alter | 2201058f_f266_11ea_bab4_0242c0a8b007 | online | 2020-09-09 06:32:31 | | failed | -| test-0000000101 | -40 | vt_commerce | demo | alter | 2201058f_f266_11ea_bab4_0242c0a8b007 | online | 2020-09-09 06:32:31 | | failed | -+-----------------+-------+--------------+-------------+------------+--------------------------------------+----------+---------------------+---------------------+------------------+ -``` - - -## Cancelling all keyspace migrations - -The user may cancel all migrations in a keyspace. A migration is cancellable if it is in `queued`, `ready` or `running` states, as described previously. It is a high impact operation and should be used with care. - -The syntax to cancelling all keyspace migrations is: - -``` -vtctlclient OnlineDDL cancel-all -``` - -Example: - -```shell -$ vtctlclient OnlineDDL commerce show all -+------------------+-------+--------------+-------------+--------------------------------------+----------+-------------------+---------------------+------------------+ -| Tablet | shard | mysql_schema | mysql_table | migration_uuid | strategy | started_timestamp | completed_timestamp | migration_status | -+------------------+-------+--------------+-------------+--------------------------------------+----------+-------------------+---------------------+------------------+ -| zone1-0000000100 | 0 | vt_commerce | corder | 2c581994_353a_11eb_8b72_f875a4d24e90 | online | | | queued | -| zone1-0000000100 | 0 | vt_commerce | corder | 2c6420c9_353a_11eb_8b72_f875a4d24e90 | online | | | queued | -| zone1-0000000100 | 0 | vt_commerce | corder | 2c7040df_353a_11eb_8b72_f875a4d24e90 | online | | | queued | -| zone1-0000000100 | 0 | vt_commerce | corder | 2c7c0572_353a_11eb_8b72_f875a4d24e90 | online | | | queued | -| zone1-0000000100 | 0 | vt_commerce | corder | 2c87f7cd_353a_11eb_8b72_f875a4d24e90 | online | | | queued | -+------------------+-------+--------------+-------------+--------------------------------------+----------+-------------------+---------------------+------------------+ - -$ vtctlclient OnlineDDL commerce cancel-all -+------------------+--------------+ -| Tablet | RowsAffected | -+------------------+--------------+ -| zone1-0000000100 | 5 | -+------------------+--------------+ - - vtctlclient OnlineDDL commerce show all -+------------------+-------+--------------+-------------+--------------------------------------+----------+-------------------+---------------------+------------------+ -| Tablet | shard | mysql_schema | mysql_table | migration_uuid | strategy | started_timestamp | completed_timestamp | migration_status | -+------------------+-------+--------------+-------------+--------------------------------------+----------+-------------------+---------------------+------------------+ -| zone1-0000000100 | 0 | vt_commerce | corder | 2c581994_353a_11eb_8b72_f875a4d24e90 | online | | | cancelled | -| zone1-0000000100 | 0 | vt_commerce | corder | 2c6420c9_353a_11eb_8b72_f875a4d24e90 | online | | | cancelled | -| zone1-0000000100 | 0 | vt_commerce | corder | 2c7040df_353a_11eb_8b72_f875a4d24e90 | online | | | cancelled | -| zone1-0000000100 | 0 | vt_commerce | corder | 2c7c0572_353a_11eb_8b72_f875a4d24e90 | online | | | cancelled | -| zone1-0000000100 | 0 | vt_commerce | corder | 2c87f7cd_353a_11eb_8b72_f875a4d24e90 | online | | | cancelled | -+------------------+-------+--------------+-------------+--------------------------------------+----------+-------------------+---------------------+------------------+ -``` - -## Retrying a migration - -The user may retry running a migration. If the migration is in `failed` or in `cancelled` state, Vitess will re-run the migration, with exact same arguments as previously intended. If the migration is in any other state, `retry` does nothing. - -It is not possible to retry a migration with different options. e.g. if the user initially runs `ALTER TABLE demo MODIFY id BIGINT` with `@@ddl_strategy='gh-ost --max-load Threads_running=200'` and the migration fails, retrying it will use exact same options. It is not possible to retry with `@@ddl_strategy='gh-ost --max-load Threads_running=500'`. - -Continuing the above example, where we cancelled a migration while running, we now retry it: - -```shell -$ vtctlclient OnlineDDL commerce show 2201058f_f266_11ea_bab4_0242c0a8b007 -+-----------------+-------+--------------+-------------+------------+--------------------------------------+----------+---------------------+---------------------+------------------+ -| Tablet | shard | mysql_schema | mysql_table | ddl_action | migration_uuid | strategy | started_timestamp | completed_timestamp | migration_status | -+-----------------+-------+--------------+-------------+------------+--------------------------------------+----------+---------------------+---------------------+------------------+ -| test-0000000401 | c0- | vt_commerce | demo | alter | 2201058f_f266_11ea_bab4_0242c0a8b007 | online | 2020-09-09 06:32:31 | | failed | -| test-0000000301 | 80-c0 | vt_commerce | demo | alter | 2201058f_f266_11ea_bab4_0242c0a8b007 | online | 2020-09-09 06:32:31 | | failed | -| test-0000000201 | 40-80 | vt_commerce | demo | alter | 2201058f_f266_11ea_bab4_0242c0a8b007 | online | 2020-09-09 06:32:31 | | failed | -| test-0000000101 | -40 | vt_commerce | demo | alter | 2201058f_f266_11ea_bab4_0242c0a8b007 | online | 2020-09-09 06:32:31 | | failed | -+-----------------+-------+--------------+-------------+------------+--------------------------------------+----------+---------------------+---------------------+------------------+ - -$ vtctlclient OnlineDDL commerce retry 2201058f_f266_11ea_bab4_0242c0a8b007 -+-----------------+--------------+ -| Tablet | RowsAffected | -+-----------------+--------------+ -| test-0000000101 | 1 | -| test-0000000201 | 1 | -| test-0000000301 | 1 | -| test-0000000401 | 1 | -+-----------------+--------------+ - -$ vtctlclient OnlineDDL commerce show 2201058f_f266_11ea_bab4_0242c0a8b007 -+-----------------+-------+--------------+-------------+------------+--------------------------------------+----------+-------------------+---------------------+------------------+ -| Tablet | shard | mysql_schema | mysql_table | ddl_action | migration_uuid | strategy | started_timestamp | completed_timestamp | migration_status | -+-----------------+-------+--------------+-------------+------------+--------------------------------------+----------+-------------------+---------------------+------------------+ -| test-0000000201 | 40-80 | vt_commerce | demo | alter | 2201058f_f266_11ea_bab4_0242c0a8b007 | online | | | queued | -| test-0000000101 | -40 | vt_commerce | demo | alter | 2201058f_f266_11ea_bab4_0242c0a8b007 | online | | | queued | -| test-0000000301 | 80-c0 | vt_commerce | demo | alter | 2201058f_f266_11ea_bab4_0242c0a8b007 | online | | | queued | -| test-0000000401 | c0- | vt_commerce | demo | alter | 2201058f_f266_11ea_bab4_0242c0a8b007 | online | | | queued | -+-----------------+-------+--------------+-------------+------------+--------------------------------------+----------+-------------------+---------------------+------------------+ - -$ vtctlclient OnlineDDL commerce show 2201058f_f266_11ea_bab4_0242c0a8b007 -+-----------------+-------+--------------+-------------+------------+--------------------------------------+----------+---------------------+---------------------+------------------+ -| Tablet | shard | mysql_schema | mysql_table | ddl_action | migration_uuid | strategy | started_timestamp | completed_timestamp | migration_status | -+-----------------+-------+--------------+-------------+------------+--------------------------------------+----------+---------------------+---------------------+------------------+ -| test-0000000101 | -40 | vt_commerce | demo | alter | 2201058f_f266_11ea_bab4_0242c0a8b007 | online | 2020-09-09 06:37:33 | | running | -| test-0000000401 | c0- | vt_commerce | demo | alter | 2201058f_f266_11ea_bab4_0242c0a8b007 | online | 2020-09-09 06:37:33 | | running | -| test-0000000201 | 40-80 | vt_commerce | demo | alter | 2201058f_f266_11ea_bab4_0242c0a8b007 | online | 2020-09-09 06:37:33 | | running | -| test-0000000301 | 80-c0 | vt_commerce | demo | alter | 2201058f_f266_11ea_bab4_0242c0a8b007 | online | 2020-09-09 06:37:33 | | running | -+-----------------+-------+--------------+-------------+------------+--------------------------------------+----------+---------------------+---------------------+------------------+ -``` - ## Auto resume after failure VReplication based migrations (`ddl_strategy="online"`) are resumable across failure and across primary failovers. @@ -465,97 +224,3 @@ The tables are kept for 24 hours after migration completion. Vitess automaticall Artifact tables are identifiable via `SELECT artifacts FROM _vt.schema_migrations` in a `VExec` command, see below. You should generally not touch these tables. It's possible to `DROP` those tables with `direct` DDL strategy. Note that dropping tables in production can be risky and lock down your database for a substantial period of time. -## VExec commands for greater control and visibility - -`vtctlclient OnlineDDL` command should provide with most needs. However, Vitess gives the user greater control through the `VExec` command and via SQL queries. - -For schema migrations, Vitess allows operations on the virtual table `_vt.schema_migrations`. Queries on this virtual table scatter to the underlying tablets and gather or manipulate data on their own, private backend tables (which incidentally are called by the same name). `VExec` only allows specific types of queries on that table. - -- `SELECT`: you may SELECT any column, or `SELECT *`. `vtctlclient OnlineDDL show` commands only present with a subset of columns, and so running ` VExec` `SELECT` provides greater visibility. Some columns that are not shown are: - - `log_path`: tablet server and path where migration logs are. - - `artifacts`: tables created by the migration. This can be used to determine which tables need cleanup. - - `alter`: the exact `alter` statement used by the migration - - `options`: any options passed by the user (e.g. `--max-load=Threads_running=200`) - - Various timestamps indicating the migration progress - Aggregate functions do not work as expected and should be avoided. `LIMIT` and `OFFSET` are not supported. -- `UPDATE`: you may directly update the status of a migration. You may only change status into `cancel` or `retry`, which Vitess interprets similarly to a `vtctlclient OnlineDDL cancel/retry` command. However, you get greater control as you may filter on a specific `shard`. -- `DELETE`: unsupported -- `INSERT`: unsupported, used internally only to advertise new migration requests to the tablets. - -The syntax to run `VExec` queries is: - -``` -vtctlclient VExec . "" -``` - -Examples: - -```shell - -$ vtctlclient VExec commerce.2201058f_f266_11ea_bab4_0242c0a8b007 "select * from _vt.schema_migrations" - -$ vtctlclient VExec commerce.91b5c953-e1e2-11ea-a097-f875a4d24e90 "update _vt.schema_migrations set migration_status='retry'" - -$ vtctlclient VExec commerce.91b5c953-e1e2-11ea-a097-f875a4d24e90 "update _vt.schema_migrations set migration_status='retry' where shard='40-80' -``` - -```shell -$ vtctlclient VExec commerce.2201058f_f266_11ea_bab4_0242c0a8b007 "select shard, mysql_table, migration_uuid, started_timestamp, completed_timestamp, migration_status from _vt.schema_migrations" -+-----------------+-------+-------------+--------------------------------------+---------------------+---------------------+------------------+ -| Tablet | shard | mysql_table | migration_uuid | started_timestamp | completed_timestamp | migration_status | -+-----------------+-------+-------------+--------------------------------------+---------------------+---------------------+------------------+ -| test-0000000301 | 80-c0 | demo | 2201058f_f266_11ea_bab4_0242c0a8b007 | 2020-09-09 06:37:33 | | failed | -| test-0000000101 | -40 | demo | 2201058f_f266_11ea_bab4_0242c0a8b007 | 2020-09-09 06:37:33 | | failed | -| test-0000000201 | 40-80 | demo | 2201058f_f266_11ea_bab4_0242c0a8b007 | 2020-09-09 08:31:47 | | failed | -| test-0000000401 | c0- | demo | 2201058f_f266_11ea_bab4_0242c0a8b007 | 2020-09-09 06:37:33 | | failed | -+-----------------+-------+-------------+--------------------------------------+---------------------+---------------------+------------------+ - -$ vtctlclient VExec commerce.2201058f_f266_11ea_bab4_0242c0a8b007 "update _vt.schema_migrations set migration_status='retry' where migration_uuid='2201058f_f266_11ea_bab4_0242c0a8b007' and shard='40-80'" -+-----------------+--------------+ -| Tablet | RowsAffected | -+-----------------+--------------+ -| test-0000000201 | 1 | -+-----------------+--------------+ - -$ vtctlclient VExec commerce.2201058f_f266_11ea_bab4_0242c0a8b007 "select shard, mysql_table, migration_uuid, started_timestamp, completed_timestamp, migration_status from _vt.schema_migrations" -+-----------------+-------+-------------+--------------------------------------+---------------------+---------------------+------------------+ -| Tablet | shard | mysql_table | migration_uuid | started_timestamp | completed_timestamp | migration_status | -+-----------------+-------+-------------+--------------------------------------+---------------------+---------------------+------------------+ -| test-0000000301 | 80-c0 | demo | 2201058f_f266_11ea_bab4_0242c0a8b007 | 2020-09-09 06:37:33 | | failed | -| test-0000000201 | 40-80 | demo | 2201058f_f266_11ea_bab4_0242c0a8b007 | 2020-09-09 08:34:59 | | running | -| test-0000000101 | -40 | demo | 2201058f_f266_11ea_bab4_0242c0a8b007 | 2020-09-09 06:37:33 | | failed | -| test-0000000401 | c0- | demo | 2201058f_f266_11ea_bab4_0242c0a8b007 | 2020-09-09 06:37:33 | | failed | -+-----------------+-------+-------------+--------------------------------------+---------------------+---------------------+------------------+ - -$ vtctlclient VExec commerce.2201058f_f266_11ea_bab4_0242c0a8b007 "update _vt.schema_migrations set migration_status='cancel' where migration_uuid='2201058f_f266_11ea_bab4_0242c0a8b007' and shard='40-80'" -+-----------------+--------------+ -| Tablet | RowsAffected | -+-----------------+--------------+ -| test-0000000201 | 1 | -+-----------------+--------------+ - -$ vtctlclient VExec commerce.2201058f_f266_11ea_bab4_0242c0a8b007 "select shard, mysql_table, migration_uuid, started_timestamp, completed_timestamp, migration_status from _vt.schema_migrations" -+-----------------+-------+-------------+--------------------------------------+---------------------+---------------------+------------------+ -| Tablet | shard | mysql_table | migration_uuid | started_timestamp | completed_timestamp | migration_status | -+-----------------+-------+-------------+--------------------------------------+---------------------+---------------------+------------------+ -| test-0000000401 | c0- | demo | 2201058f_f266_11ea_bab4_0242c0a8b007 | 2020-09-09 06:37:33 | | failed | -| test-0000000101 | -40 | demo | 2201058f_f266_11ea_bab4_0242c0a8b007 | 2020-09-09 06:37:33 | | failed | -| test-0000000201 | 40-80 | demo | 2201058f_f266_11ea_bab4_0242c0a8b007 | 2020-09-09 08:34:59 | | failed | -| test-0000000301 | 80-c0 | demo | 2201058f_f266_11ea_bab4_0242c0a8b007 | 2020-09-09 06:37:33 | | failed | -+-----------------+-------+-------------+--------------------------------------+---------------------+---------------------+------------------+ - -$ vtctlclient VExec commerce.2201058f_f266_11ea_bab4_0242c0a8b007 "update _vt.schema_migrations set migration_status='cancel' where migration_uuid='2201058f_f266_11ea_bab4_0242c0a8b007' and shard='40-80'" - - -$ vtctlclient VExec commerce.2201058f_f266_11ea_bab4_0242c0a8b007 "select shard, log_path from _vt.schema_migrations" -+-----------------+-------+-----------------------------------------------------------------------------+ -| Tablet | shard | log_path | -+-----------------+-------+-----------------------------------------------------------------------------+ -| test-0000000201 | 40-80 | 11ac2af6e63e:/tmp/online-ddl-2201058f_f266_11ea_bab4_0242c0a8b007-657478384 | -| test-0000000101 | -40 | e779a82d35d7:/tmp/online-ddl-2201058f_f266_11ea_bab4_0242c0a8b007-901629215 | -| test-0000000401 | c0- | 5aad1249ab91:/tmp/online-ddl-2201058f_f266_11ea_bab4_0242c0a8b007-039568897 | -| test-0000000301 | 80-c0 | 5e7c662679d3:/tmp/online-ddl-2201058f_f266_11ea_bab4_0242c0a8b007-532703073 | -+-----------------+-------+-----------------------------------------------------------------------------+ -``` - - diff --git a/content/en/docs/user-guides/schema-changes/revertible-migrations.md b/content/en/docs/user-guides/schema-changes/revertible-migrations.md index 757e1306b..f75b86de4 100644 --- a/content/en/docs/user-guides/schema-changes/revertible-migrations.md +++ b/content/en/docs/user-guides/schema-changes/revertible-migrations.md @@ -1,10 +1,10 @@ --- title: Revertible migrations -weight: 2 +weight: 6 aliases: ['/docs/user-guides/schema-changes/revertible-migrations/'] --- -Vitess's [managed schema changes](../managed-online-schema-changes/) offer _lossless revert_ for online schema migrations: the user may regret a table migration after completion, and roll back the tabe's schema to previous state _without loss of data_. +Vitess's [managed schema changes](../managed-online-schema-changes/) offer _lossless revert_ for online schema migrations: the user may regret a table migration after completion, and roll back the table's schema to previous state _without loss of data_. Revertible migrations supported for: diff --git a/content/en/docs/user-guides/schema-changes/unmanaged-schema-changes.md b/content/en/docs/user-guides/schema-changes/unmanaged-schema-changes.md index 81c3e6dfb..5a3338486 100644 --- a/content/en/docs/user-guides/schema-changes/unmanaged-schema-changes.md +++ b/content/en/docs/user-guides/schema-changes/unmanaged-schema-changes.md @@ -1,6 +1,6 @@ --- title: Unmanaged Schema Changes -weight: 3 +weight: 1 aliases: ['/docs/user-guides/operating-vitess/making-schema-changes', '/docs/schema-management/unmanaged-schema-changes/', '/docs/user-guides/unmanaged-schema-changes/'] --- From 6b86bd0dec79c9de83a2b8df2a5642e38580c5b6 Mon Sep 17 00:00:00 2001 From: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> Date: Thu, 1 Apr 2021 10:14:11 +0300 Subject: [PATCH 7/8] comparison table: traffic column Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> --- .../user-guides/schema-changes/ddl-strategies.md | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/content/en/docs/user-guides/schema-changes/ddl-strategies.md b/content/en/docs/user-guides/schema-changes/ddl-strategies.md index 473d9e631..d35c7983e 100644 --- a/content/en/docs/user-guides/schema-changes/ddl-strategies.md +++ b/content/en/docs/user-guides/schema-changes/ddl-strategies.md @@ -152,12 +152,12 @@ There are pros and cons to using any of the strategies. Some notable differences ## Vitess functionality comparison -| Strategy | Managed | Online | Trackable | Declarative | Revertible | -|----------|---------|--------|-----------|-------------|---------------------| -| `direct` | No | MySQL* | No | No | No | -| `pt-osc` | Yes | Yes* | Yes | Yes | `CREATE,DROP` | -| `gh-ost` | Yes | Yes* | Yes+ | Yes | `CREATE,DROP` | -| `online` | Yes | Yes* | Yes | Yes | `CREATE,DROP,ALTER` | +| Strategy | Managed | Online | Trackable | Declarative | Revertible | Traffic | +|----------|---------|--------|-----------|-------------|---------------------|---------| +| `direct` | No | MySQL* | No | No | No | Any | +| `pt-osc` | Yes | Yes* | Yes | Yes | `CREATE,DROP` | Any | +| `gh-ost` | Yes | Yes* | Yes+ | Yes | `CREATE,DROP` | Any | +| `online` | Yes | Yes* | Yes | Yes | `CREATE,DROP,ALTER` | Vitess | - **Managed**: whether Vitess schedules and operates the migration - **Online**: @@ -168,3 +168,4 @@ There are pros and cons to using any of the strategies. Some notable differences - `gh-ost` also makes available _progress %_ and _ETA seconds_ - **Declarative**: support `-declarative` flag - **Revertible**: `online` strategy supports revertible `ALTER` statements (or `ALTER`s implied by `-declarative` migrations). All managed strategies supports revertible `CREATE` and `ALTER`. +- **Traffic**: `online` migration cut-over uses Vitess specific blocking of traffic, and is therefore only safe when write traffic to the tables runs entirely through Vitess/VTGate. `gh-ost` and `pt-osc` use generic MySQL blocking/locking mechanisms, and it is safe to run some write traffic on the migrated table outside Vitess. From c01c27f0366959040256e1e3fee8496cb0f05309 Mon Sep 17 00:00:00 2001 From: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> Date: Tue, 20 Apr 2021 10:59:03 +0300 Subject: [PATCH 8/8] removed reference to Aurora Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> --- content/en/docs/user-guides/schema-changes/ddl-strategies.md | 4 ---- 1 file changed, 4 deletions(-) diff --git a/content/en/docs/user-guides/schema-changes/ddl-strategies.md b/content/en/docs/user-guides/schema-changes/ddl-strategies.md index d35c7983e..cccfac4e7 100644 --- a/content/en/docs/user-guides/schema-changes/ddl-strategies.md +++ b/content/en/docs/user-guides/schema-changes/ddl-strategies.md @@ -146,10 +146,6 @@ There are pros and cons to using any of the strategies. Some notable differences - `pt-online-schema-change` partially supports foreign keys. Neither `gh-ost` nor `VReplication` support foreign keys. -#### External MySQL compatibility - -* If you run on Aurora or RDS you will need to use the `online` strategy. This is because both `pt-online-schema-change` and `gh-ost` try to create a new user and then attempt to grant the new user SUPER privileges. - ## Vitess functionality comparison | Strategy | Managed | Online | Trackable | Declarative | Revertible | Traffic |