Skip to content

Update docs: VExec no longer a path for Online DDL operations #878

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
shlomi-noach merged 6 commits into from
Nov 22, 2021
Merged

Update docs: VExec no longer a path for Online DDL operations #878

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
cc13453
Online DDL: VExec is deprecated, removed from docs.
shlomi-noach Nov 10, 2021
dab4a55
REVERT is done via 'vtctl ApplySchema'
shlomi-noach Nov 10, 2021
19d7d49
removed reference to VExec
shlomi-noach Nov 10, 2021
7f5b093
remove reference to VExec
shlomi-noach Nov 10, 2021
5265ddc
VExec is restored in https://github.com/vitessio/vitess/pull/9225 and…
shlomi-noach Nov 14, 2021
ceaa242
Merge branch 'prod' into update-docs-deperecate-vexec
shlomi-noach Nov 14, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
99 changes: 1 addition & 98 deletions content/en/docs/user-guides/schema-changes/audit-and-control.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -574,102 +574,5 @@ Create Table: CREATE TABLE `corder` (
### Via vtctl/ApplySchema

```
$ vtctlclient OnlineDDL commerce revert 2201058f_f266_11ea_bab4_0242c0a8b007
$ vtctlclient ApplySchema -ddl_strategy "online" -sql "revert vitess_migration '1a689113_8d77_11eb_815f_f875a4d24e90'" commerce
```

## VExec commands for greater control and visibility

VExec is deprecated in version 12.0.

`vtctlclient OnlineDDL` command should provide for most needs. However, Vitess gives the user greater control through the `VExec` command and via SQL queries.

For schema migrations, Vitess allows operations on the virtual table `_vt.schema_migrations`. Queries on this virtual table scatter to the underlying tablets and gather or manipulate data on their own, private backend tables (which incidentally are called by the same name). `VExec` only allows specific types of queries on that table.

- `SELECT`: you may SELECT any column, or `SELECT *`. `vtctlclient OnlineDDL show` commands only present with a subset of columns, and so running ` VExec` `SELECT` provides greater visibility. Some columns that are not shown are:
- `log_path`: tablet server and path where migration logs are.
- `artifacts`: tables created by the migration. This can be used to determine which tables need cleanup.
- `alter`: the exact `alter` statement used by the migration
- `options`: any options passed by the user (e.g. `--max-load=Threads_running=200`)
- Various timestamps indicating the migration progress
Aggregate functions do not work as expected and should be avoided. `LIMIT` and `OFFSET` are not supported.
- `UPDATE`: you may directly update the status of a migration. You may only change status into `cancel` or `retry`, which Vitess interprets similarly to a `vtctlclient OnlineDDL cancel/retry` command. However, you get greater control as you may filter on a specific `shard`.
- `DELETE`: unsupported
- `INSERT`: unsupported, used internally only to advertise new migration requests to the tablets.

The syntax to run `VExec` queries is:

```
vtctlclient VExec <keyspace>.<migration_id> "<sql query>"
```

Examples:

```shell

$ vtctlclient VExec commerce.2201058f_f266_11ea_bab4_0242c0a8b007 "select * from _vt.schema_migrations"

$ vtctlclient VExec commerce.91b5c953-e1e2-11ea-a097-f875a4d24e90 "update _vt.schema_migrations set migration_status='retry'"

$ vtctlclient VExec commerce.91b5c953-e1e2-11ea-a097-f875a4d24e90 "update _vt.schema_migrations set migration_status='retry' where shard='40-80'
```

```shell
$ vtctlclient VExec commerce.2201058f_f266_11ea_bab4_0242c0a8b007 "select shard, mysql_table, migration_uuid, started_timestamp, completed_timestamp, migration_status from _vt.schema_migrations"
+-----------------+-------+-------------+--------------------------------------+---------------------+---------------------+------------------+
| Tablet | shard | mysql_table | migration_uuid | started_timestamp | completed_timestamp | migration_status |
+-----------------+-------+-------------+--------------------------------------+---------------------+---------------------+------------------+
| test-0000000301 | 80-c0 | demo | 2201058f_f266_11ea_bab4_0242c0a8b007 | 2020-09-09 06:37:33 | | failed |
| test-0000000101 | -40 | demo | 2201058f_f266_11ea_bab4_0242c0a8b007 | 2020-09-09 06:37:33 | | failed |
| test-0000000201 | 40-80 | demo | 2201058f_f266_11ea_bab4_0242c0a8b007 | 2020-09-09 08:31:47 | | failed |
| test-0000000401 | c0- | demo | 2201058f_f266_11ea_bab4_0242c0a8b007 | 2020-09-09 06:37:33 | | failed |
+-----------------+-------+-------------+--------------------------------------+---------------------+---------------------+------------------+

$ vtctlclient VExec commerce.2201058f_f266_11ea_bab4_0242c0a8b007 "update _vt.schema_migrations set migration_status='retry' where migration_uuid='2201058f_f266_11ea_bab4_0242c0a8b007' and shard='40-80'"
+-----------------+--------------+
| Tablet | RowsAffected |
+-----------------+--------------+
| test-0000000201 | 1 |
+-----------------+--------------+

$ vtctlclient VExec commerce.2201058f_f266_11ea_bab4_0242c0a8b007 "select shard, mysql_table, migration_uuid, started_timestamp, completed_timestamp, migration_status from _vt.schema_migrations"
+-----------------+-------+-------------+--------------------------------------+---------------------+---------------------+------------------+
| Tablet | shard | mysql_table | migration_uuid | started_timestamp | completed_timestamp | migration_status |
+-----------------+-------+-------------+--------------------------------------+---------------------+---------------------+------------------+
| test-0000000301 | 80-c0 | demo | 2201058f_f266_11ea_bab4_0242c0a8b007 | 2020-09-09 06:37:33 | | failed |
| test-0000000201 | 40-80 | demo | 2201058f_f266_11ea_bab4_0242c0a8b007 | 2020-09-09 08:34:59 | | running |
| test-0000000101 | -40 | demo | 2201058f_f266_11ea_bab4_0242c0a8b007 | 2020-09-09 06:37:33 | | failed |
| test-0000000401 | c0- | demo | 2201058f_f266_11ea_bab4_0242c0a8b007 | 2020-09-09 06:37:33 | | failed |
+-----------------+-------+-------------+--------------------------------------+---------------------+---------------------+------------------+

$ vtctlclient VExec commerce.2201058f_f266_11ea_bab4_0242c0a8b007 "update _vt.schema_migrations set migration_status='cancel' where migration_uuid='2201058f_f266_11ea_bab4_0242c0a8b007' and shard='40-80'"
+-----------------+--------------+
| Tablet | RowsAffected |
+-----------------+--------------+
| test-0000000201 | 1 |
+-----------------+--------------+

$ vtctlclient VExec commerce.2201058f_f266_11ea_bab4_0242c0a8b007 "select shard, mysql_table, migration_uuid, started_timestamp, completed_timestamp, migration_status from _vt.schema_migrations"
+-----------------+-------+-------------+--------------------------------------+---------------------+---------------------+------------------+
| Tablet | shard | mysql_table | migration_uuid | started_timestamp | completed_timestamp | migration_status |
+-----------------+-------+-------------+--------------------------------------+---------------------+---------------------+------------------+
| test-0000000401 | c0- | demo | 2201058f_f266_11ea_bab4_0242c0a8b007 | 2020-09-09 06:37:33 | | failed |
| test-0000000101 | -40 | demo | 2201058f_f266_11ea_bab4_0242c0a8b007 | 2020-09-09 06:37:33 | | failed |
| test-0000000201 | 40-80 | demo | 2201058f_f266_11ea_bab4_0242c0a8b007 | 2020-09-09 08:34:59 | | failed |
| test-0000000301 | 80-c0 | demo | 2201058f_f266_11ea_bab4_0242c0a8b007 | 2020-09-09 06:37:33 | | failed |
+-----------------+-------+-------------+--------------------------------------+---------------------+---------------------+------------------+

$ vtctlclient VExec commerce.2201058f_f266_11ea_bab4_0242c0a8b007 "update _vt.schema_migrations set migration_status='cancel' where migration_uuid='2201058f_f266_11ea_bab4_0242c0a8b007' and shard='40-80'"
<no result>

$ vtctlclient VExec commerce.2201058f_f266_11ea_bab4_0242c0a8b007 "select shard, log_path from _vt.schema_migrations"
+-----------------+-------+-----------------------------------------------------------------------------+
| Tablet | shard | log_path |
+-----------------+-------+-----------------------------------------------------------------------------+
| test-0000000201 | 40-80 | 11ac2af6e63e:/tmp/online-ddl-2201058f_f266_11ea_bab4_0242c0a8b007-657478384 |
| test-0000000101 | -40 | e779a82d35d7:/tmp/online-ddl-2201058f_f266_11ea_bab4_0242c0a8b007-901629215 |
| test-0000000401 | c0- | 5aad1249ab91:/tmp/online-ddl-2201058f_f266_11ea_bab4_0242c0a8b007-039568897 |
| test-0000000301 | 80-c0 | 5e7c662679d3:/tmp/online-ddl-2201058f_f266_11ea_bab4_0242c0a8b007-532703073 |
+-----------------+-------+-----------------------------------------------------------------------------+
```


2 changes: 1 addition & 1 deletion content/en/docs/user-guides/schema-changes/managed-online-schema-changes.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -210,5 +210,5 @@ All `ALTER` strategies leave artifacts behind. Whether successful or failed, eit

The tables are kept for 24 hours after migration completion. Vitess automatically cleans up those tables as soon as a migration completes (either successful or failed). You will normally not need to do anything.

Artifact tables are identifiable via `SELECT artifacts FROM _vt.schema_migrations` in a `VExec` command. Note that `VExec` is deprecated in version 12.0. You should generally not touch these tables. It's possible to `DROP` those tables with `direct` DDL strategy. Note that dropping tables in production can be risky and lock down your database for a substantial period of time.
Artifact tables are identifiable via `artifacts` column in a `SHOW VITESS_MIGRATION ...` command. You should generally not touch these tables. It's possible to `DROP` those tables with `direct` DDL strategy. Note that dropping tables in production can be risky and lock down your database for a substantial period of time. Dropping artifact tables also makes the migrations impossible to [revert](../revertible-migrations/).