Skip to content

[release-17.0] Incremental backup and point in time recovery for XtraBackup (#13156)#13241

Merged
GuptaManan100 merged 32 commits intorelease-17.0from
backport-13156-to-release-17.0
Jun 6, 2023
Merged

[release-17.0] Incremental backup and point in time recovery for XtraBackup (#13156)#13241
GuptaManan100 merged 32 commits intorelease-17.0from
backport-13156-to-release-17.0

Conversation

@vitess-bot
Copy link
Contributor

@vitess-bot vitess-bot bot commented Jun 6, 2023

Description

This is a backport of #13156.

shlomi-noach and others added 30 commits May 24, 2023 14:19
@shlomi-noach
incremental backup is always using 'builtin' engine
62522b4 
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
@shlomi-noach
restore: use 'builtin' for incremental restore
e7452f0 
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
@shlomi-noach
test all backup types
22f16d7 
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
@shlomi-noach
format code
b4137a8 
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
@shlomi-noach
Populate PurgedPosition
cbd1799 
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
@shlomi-noach
cleanup backups at the end of each test case
caad77f 
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
@shlomi-noach
improved cleanup
531b00c 
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
@shlomi-noach
rename variable
0aff542 
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
@shlomi-noach
record all backups
161ad59 
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
@shlomi-noach
no need to cleanup backups in between test cases, since each new case…
d11cf85 
… has a completely different UUID

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
@shlomi-noach
install xtrabackup on backup_pitr tests
eee0499 
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
@shlomi-noach
use pgzip for xtrabackup
0400f86 
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
@shlomi-noach
more debug info
6da0c1f 
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
@shlomi-noach
builtin engine: store gtid_purged in manifest
ea357bb 
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
@shlomi-noach
use backupfrom-GTID as incremental-from-GTID if first binary log has …
79c13f2 
…empty Previous-GTIDS

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
@shlomi-noach
more unit tests
ccd1499 
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
@shlomi-noach
improve error message
8a1be41 
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
@shlomi-noach
capturing MySQL's stderr and reading and logging if not empty
5edcff3 
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
@shlomi-noach
At the end of Xtrabackup restore, validate that @@gtid_purged (and th…
ff1cf2e 
…ereby @@gtid_executed) are set to the backup's @@gtid_purge. Make it so if they aren't

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
@shlomi-noach
add comperssion details into test case. Fix GTID validation of manife…
991969b 
…st file

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
@shlomi-noach
check manifest
976a15a 
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
@shlomi-noach
Refactor into function
ed2a0da 
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
@shlomi-noach
check manifest.Position.GTIDSet
1ff6e57 
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
@shlomi-noach
fix wrangler tests
92a56d5 
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
@shlomi-noach
typo
64e2881 
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
@shlomi-noach @mattlord
Update go/vt/mysqlctl/backup.go
89634a7 
Co-authored-by: Matt Lord <mattalord@gmail.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
@shlomi-noach @mattlord
Update go/vt/mysqlctl/backup.go
ff0e860 
Co-authored-by: Matt Lord <mattalord@gmail.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
@shlomi-noach @mattlord
Update go/vt/mysqlctl/backup.go
eb8f39f 
Co-authored-by: Matt Lord <mattalord@gmail.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
@shlomi-noach
typo
bece3cc 
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
@shlomi-noach @mattlord
Update go/vt/mysqlctl/mysqld.go
ede8e9d 
Co-authored-by: Matt Lord <mattalord@gmail.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
shlomi-noach and others added 2 commits June 5, 2023 17:05
@shlomi-noach @mattlord
Update go/vt/mysqlctl/mysqld.go
a57d3e2 
Co-authored-by: Matt Lord <mattalord@gmail.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
@shlomi-noach @mattlord
Update go/vt/mysqlctl/mysqld.go
b6d9c3b 
Co-authored-by: Matt Lord <mattalord@gmail.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
@vitess-bot vitess-bot bot added Backport This is a backport Component: Backup and Restore Type: Enhancement Logical improvement (somewhere between a bug and feature) NeedsDescriptionUpdate The description is not clear or comprehensive enough, and needs work NeedsIssue A linked issue is missing for this Pull Request NeedsWebsiteDocsUpdate What it says labels Jun 6, 2023
@vitess-bot
Copy link
Contributor Author

vitess-bot bot commented Jun 6, 2023

Review Checklist

Hello reviewers! 👋 Please follow this checklist when reviewing this Pull Request.

General

  • Ensure that the Pull Request has a descriptive title.
  • If this is a change that users need to know about, please apply the release notes (needs details) label so that merging is blocked unless the summary release notes document is included.
  • If a test is added or modified, there should be a documentation on top of the test to explain what the expected behavior is what the test does.

If a new flag is being introduced:

  • Is it really necessary to add this flag?
  • Flag names should be clear and intuitive (as far as possible)
  • Help text should be descriptive.
  • Flag names should use dashes (-) as word separators rather than underscores (_).

If a workflow is added or modified:

  • Each item in Jobs should be named in order to mark it as required.
  • If the workflow should be required, the maintainer team should be notified.

Bug fixes

  • There should be at least one unit or end-to-end test.
  • The Pull Request description should include a link to an issue that describes the bug.

Non-trivial changes

  • There should be some code comments as to why things are implemented the way they are.

New/Existing features

  • Should be documented, either by modifying the existing documentation or creating new documentation.
  • New features should have a link to a feature request issue or an RFC that documents the use cases, corner cases and test cases.

Backward compatibility

  • Protobuf changes should be wire-compatible.
  • Changes to _vt tables and RPCs need to be backward compatible.
  • vtctl command output order should be stable and awk-able.
  • RPC changes should be compatible with vitess-operator
  • If a flag is removed, then it should also be removed from VTop, if used there.

@github-actions github-actions bot added this to the v17.0.0 milestone Jun 6, 2023
@shlomi-noach shlomi-noach removed NeedsDescriptionUpdate The description is not clear or comprehensive enough, and needs work NeedsWebsiteDocsUpdate What it says NeedsIssue A linked issue is missing for this Pull Request labels Jun 6, 2023
@GuptaManan100
Copy link
Contributor

GuptaManan100 commented Jun 6, 2023

All tests are green. Going ahead with the merge.

@GuptaManan100 GuptaManan100 merged commit ebcaab2 into release-17.0 Jun 6, 2023
@GuptaManan100 GuptaManan100 deleted the backport-13156-to-release-17.0 branch June 6, 2023 05:40
@GuptaManan100 GuptaManan100 mentioned this pull request Jun 6, 2023
Closed
42 tasks
@jjpersch jjpersch mentioned this pull request Sep 21, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@shlomi-noach shlomi-noach shlomi-noach approved these changes

@frouioui frouioui Awaiting requested review from frouioui frouioui is a code owner

@rohit-nayak-ps rohit-nayak-ps Awaiting requested review from rohit-nayak-ps rohit-nayak-ps is a code owner

@deepthi deepthi Awaiting requested review from deepthi

@mattlord mattlord Awaiting requested review from mattlord mattlord is a code owner

@harshit-gangal harshit-gangal Awaiting requested review from harshit-gangal harshit-gangal is a code owner

+1 more reviewer

@GuptaManan100 GuptaManan100 GuptaManan100 approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

Backport This is a backport Component: Backup and Restore Type: Enhancement Logical improvement (somewhere between a bug and feature)

Projects

None yet

Milestone

v17.0.0

Development

Successfully merging this pull request may close these issues.

2 participants

@GuptaManan100 @shlomi-noach