Documenting new Online DDL via VReplication, 'online' strategy by shlomi-noach · Pull Request #718 · vitessio/website

shlomi-noach · 2021-03-03T12:50:48Z

This documents the changes in the now-merged PR vitessio/vitess#7419:

Introduction of VReplication based migrations
Introduction of online strategy
Extracted discussion about different strategies onto its own page, with usage for each, and feature comparison.

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

netlify · 2021-03-03T12:51:56Z

Deploy preview for vitess ready!

Built with commit f7ef197

https://deploy-preview-718--vitess.netlify.app

shlomi-noach · 2021-03-03T12:57:10Z

Relevant pages in netlify preview:

jmoldow · 2021-03-03T17:24:56Z

content/en/docs/user-guides/schema-changes/ddl-strategies.md

+
+#### Load
+
+- `pt-online-schema-change` uses triggers to propagate changes. This method is traditionally known to generate high load on the server. Both VReplication and `gh-ost` tail the binary logs to capture changes, and thi sapproach is known to be more lightweight.


Suggested change

- `pt-online-schema-change` uses triggers to propagate changes. This method is traditionally known to generate high load on the server. Both VReplication and `gh-ost` tail the binary logs to capture changes, and thi sapproach is known to be more lightweight.

- `pt-online-schema-change` uses triggers to propagate changes. This method is traditionally known to generate high load on the server. Both VReplication and `gh-ost` tail the binary logs to capture changes, and this approach is known to be more lightweight.

jmoldow · 2021-03-03T17:25:32Z

content/en/docs/user-guides/schema-changes/ddl-strategies.md

+
+To be able to run online schema migrations via `gh-ost`:
+
+- If you're on Linux/amd64 architecture, and on `glibc` `2.3` or similar, there are no further dependencies. Vitess comes with a built-in `gh-ost` binary, that is compatible with your system.


Should it be mentioned that this is true of the Vitess Docker images on Dockerhub?

jmoldow · 2021-03-03T17:26:14Z

content/en/docs/user-guides/schema-changes/ddl-strategies.md

+- `set @@ddl_strategy='gh-ost --max-load Threads_running=200 --critical-load Threads_running=500 --critical-load-hibernate-seconds=60 --default-retries=512';`
+- `vtctl -ddl_strategy "gh-ost --allow-nullable-unique-key --chunk-size 200" ApplySchema ...`
+
+Do not override the following flags: `alter, database, table, execute, max-lag, force-table-names, serve-socket-file, hooks-path, hooks-hint-token, panic-flag-file`.


Can you add more detail here? Will something bad happen if you do? Or will Vitess reject the request? Or will Vitess silently drop your flags?

Good point! Added.

jmoldow · 2021-03-03T17:26:56Z

content/en/docs/user-guides/schema-changes/ddl-strategies.md

+
+#### Setup
+
+- Vreplication is part of Vitess


Suggested change

- Vreplication is part of Vitess

- VReplication is part of Vitess

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

shlomi-noach · 2021-03-04T06:58:36Z

@jmoldow thank you for the review, I've addressed your comments (without technically "accepting changes" because of signoff issue).

derekperkins · 2021-03-04T14:10:24Z

This is fantastic documentation.

Neither gh-ost nor VReplication support foreign keys.

Is the schema introspected when you call alter and rejects tables with FKs? Will it silently fail during the alter? Is there a recommendation for removing them before the alter and adding them back after?

shlomi-noach · 2021-03-04T14:26:27Z

Is the schema introspected when you call alter and rejects tables with FKs?

gh-ost: yes, and migration fails (not even started).
VReplication: I need to check. Nothing I added myself, so I need to dig in.

Is there a recommendation for removing them before the alter and adding them back after?

In my understanding this leads to foreign-key broken integrity. Something will have happened during the migration that violates some foreign key constraint. "child"-side this may be worked around, "parent"-side ther is no solution that I'm aware of. I've been at this for years and years, and I can't see a way out, unfortunately.

deepthi

Beautifully written as always.
You may address the feedback in a separate PR if you prefer to do so. Mainly around incorporating changes from vitessio/vitess#7603, and a few nits/typos.

deepthi · 2021-03-05T23:29:35Z

content/en/docs/user-guides/schema-changes/ddl-strategies.md

+- `pt-online-schema-change` tool installed and is executable
+- Perl `libdbi` and `libdbd-mysql` modules installed. e.g. on Debian/Ubuntu, `sudo apt-get install libdbi-perl libdbd-mysql-perl`
+- Run `vttablet` with `-pt-osc-path=/full/path/to/pt-online-schema-change` flag.
+


This section should be updated since we merged vitessio/vitess#7603.

Clarified that pt-osc and dependencies are pre-installed in Vitess Docker images

deepthi · 2021-03-05T23:30:29Z

content/en/docs/user-guides/schema-changes/ddl-strategies.md

+
+### Comparing the options
+
+There are pros and cons to using either of the strategies. Some notable differences:


Nit: there are more than two options, so it should be "any" or "each" instead of "either".

deepthi · 2021-03-05T23:31:08Z

content/en/docs/user-guides/schema-changes/ddl-strategies.md

+
+- VReplication is part of Vitess
+- A `gh-ost` binary is embedded within the Vitess binary, compatible with `glibc 2.3` and `Linux/amd64`. The user may choose to use their own `gh-ost` binary, configured with `-gh-ost-path`.
+- `pt-online-schema-change` is not included in Vitess, and the user needs to set it up on tablet hosts.


Also needs to change.

Clarified that pt-osc and dependencies are pre-installed in Vitess Docker images

deepthi · 2021-03-05T23:32:53Z

content/en/docs/user-guides/schema-changes/managed-online-schema-changes.md

 - The value `"pt-osc"` instructs Vitess to run an `ALTER TABLE` online DDL via `pt-online-schema-change`.
 - You may specify arguments for your tool of choice, e.g. `"gh-ost --max-load Threads_running=200"`. Details follow.

+`CREATE` and `DROP` statements run in the same way for `"online"`, `"gh-ost"` and `"pt-osc"` strategies, andwe consider them all to be _online_.


typo: andwe -> and we

deepthi · 2021-03-05T23:33:27Z

content/en/docs/user-guides/schema-changes/managed-online-schema-changes.md

+- in MySQL pre `8.0.23`, a `DROP TABLE` operation can be dangerous in production as it commonly locks the buffer pool for a substantial period.

-Artifact tables are identifiable via `SELECT artifacts FROM _vt.schema_migrations` in a `VExec` command, see below.
+The tables are kept for 24 hours since migration completion. Vitess automatically cleans up those tables as soon as a migration completes (either successful or failed). You will normally not need to do anything.


Suggested change

The tables are kept for 24 hours since migration completion. Vitess automatically cleans up those tables as soon as a migration completes (either successful or failed). You will normally not need to do anything.

The tables are kept for 24 hours after migration completion. Vitess automatically cleans up those tables as soon as a migration completes (either successful or failed). You will normally not need to do anything.

deepthi · 2021-03-05T23:34:01Z

content/en/docs/user-guides/schema-changes/managed-online-schema-changes.md

 ## Table cleanup

-Both `gh-ost` and `pt-online-schema-change` leave artifacts behind. Whether successful or failed, either the original table or the _ghost_ table are left still populated at the end of the migration. Vitess explicitly configures both tools to not drop those tables. The reason is that in MySQL, a `DROP TABLE` operation can be dangerous in production as it commonly locks the buffer pool for a substantial period.
+All `ALTER` strategies leave artifacts behind. Whether successful or failed, either the original table or the _ghost_ table are left still populated at the end of the migration. Vitess explicitly makes sure the tables are not dropped at the end of the migration. This is for two reasons:


Suggested change

All `ALTER` strategies leave artifacts behind. Whether successful or failed, either the original table or the _ghost_ table are left still populated at the end of the migration. Vitess explicitly makes sure the tables are not dropped at the end of the migration. This is for two reasons:

All `ALTER` strategies leave artifacts behind. Whether successful or failed, either the original table or the _ghost_ table is left still populated at the end of the migration. Vitess explicitly makes sure the tables are not dropped at the end of the migration. This is for two reasons:

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

Documenting new Online DDL via VReplication, 'online' strategy

072ecb0

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

shlomi-noach requested review from deepthi, rohit-nayak-ps and sougou March 3, 2021 12:50

jmoldow reviewed Mar 3, 2021

View reviewed changes

changes per review

ef3e51e

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

deepthi approved these changes Mar 5, 2021

View reviewed changes

shlomi-noach added 4 commits March 7, 2021 08:50

typos per review

96385f5

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

incorporating change from a387799

8d7f3f6

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

Merge branch 'prod' into online-ddl-vrepl

254d26c

clarify pt-osc pre-installed in docker images

f7ef197

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

shlomi-noach merged commit 816539d into prod Mar 7, 2021

shlomi-noach deleted the online-ddl-vrepl branch March 7, 2021 06:58

shlomi-noach mentioned this pull request Mar 7, 2021

Fix argument ordering #717

Merged


		#### Load

		- `pt-online-schema-change` uses triggers to propagate changes. This method is traditionally known to generate high load on the server. Both VReplication and `gh-ost` tail the binary logs to capture changes, and thi sapproach is known to be more lightweight.


		To be able to run online schema migrations via `gh-ost`:

		- If you're on Linux/amd64 architecture, and on `glibc` `2.3` or similar, there are no further dependencies. Vitess comes with a built-in `gh-ost` binary, that is compatible with your system.

	- Vreplication is part of Vitess
	- VReplication is part of Vitess


		### Comparing the options

		There are pros and cons to using either of the strategies. Some notable differences:

	The tables are kept for 24 hours since migration completion. Vitess automatically cleans up those tables as soon as a migration completes (either successful or failed). You will normally not need to do anything.
	The tables are kept for 24 hours after migration completion. Vitess automatically cleans up those tables as soon as a migration completes (either successful or failed). You will normally not need to do anything.

	All `ALTER` strategies leave artifacts behind. Whether successful or failed, either the original table or the _ghost_ table are left still populated at the end of the migration. Vitess explicitly makes sure the tables are not dropped at the end of the migration. This is for two reasons:
	All `ALTER` strategies leave artifacts behind. Whether successful or failed, either the original table or the _ghost_ table is left still populated at the end of the migration. Vitess explicitly makes sure the tables are not dropped at the end of the migration. This is for two reasons:

Conversation

shlomi-noach commented Mar 3, 2021

Uh oh!

netlify bot commented Mar 3, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

shlomi-noach commented Mar 3, 2021

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

shlomi-noach commented Mar 4, 2021

Uh oh!

derekperkins commented Mar 4, 2021

Uh oh!

shlomi-noach commented Mar 4, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

deepthi left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

netlify bot commented Mar 3, 2021 •

edited

Loading

shlomi-noach commented Mar 4, 2021 •

edited

Loading

deepthi left a comment •

edited

Loading