Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Binary compatibility version for DeltaMerge's chunk; Fix disappear of PageStorage del meta #257

Merged

Conversation

JaySon-Huang
Copy link
Contributor

@JaySon-Huang JaySon-Huang commented Sep 27, 2019

DeltaMerge

  • Add binary compatibility version for chunk. We can update serialized chunk format in the future without breaking old data.

PageStorage

  • Migrate DelPage meta in doing GC.
  • Add name to PageStorage, so that we can tell which storage is happen to error when there is multiple DeltaMerge tables.
  • Remove getMaxId of PageStorage. We just need the max Page ID when we load PageStorage from disk, removing this API so that we can ignore max ID when putting new Page.

@JaySon-Huang
Copy link
Contributor Author

/run-integration-tests

@flowbehappy
Copy link
Contributor

LGTM

@JaySon-Huang
Copy link
Contributor Author

/run-integration-tests

@JaySon-Huang
Copy link
Contributor Author

/run-integration-tests

@JaySon-Huang JaySon-Huang merged commit 779acfa into pingcap:DeltaMergeEngine Sep 27, 2019
JaySon-Huang added a commit to JaySon-Huang/tiflash that referenced this pull request Oct 17, 2019
… PageStorage del meta (pingcap#257)

* Chunk binary version

* Add name to PageStorage for identify different storages

* Remove getMaxId of PageStorage

* Migrate DelPage mark in doing GC

* enable fullstack-test

* Revert "Remove getMaxId of PageStorage"

This reverts commit 34d50eb6e9fb2f229f32e2d6b219b74c340d0d0a.
JaySon-Huang added a commit to JaySon-Huang/tiflash that referenced this pull request Oct 18, 2019
… PageStorage del meta (pingcap#257)

* Chunk binary version

* Add name to PageStorage for identify different storages

* Remove getMaxId of PageStorage

* Migrate DelPage mark in doing GC

* enable fullstack-test

* Revert "Remove getMaxId of PageStorage"

This reverts commit 34d50eb6e9fb2f229f32e2d6b219b74c340d0d0a.
JaySon-Huang added a commit that referenced this pull request Oct 22, 2019
… PageStorage del meta (#257)

* Chunk binary version

* Add name to PageStorage for identify different storages

* Remove getMaxId of PageStorage

* Migrate DelPage mark in doing GC

* enable fullstack-test

* Revert "Remove getMaxId of PageStorage"

This reverts commit 34d50eb6e9fb2f229f32e2d6b219b74c340d0d0a.
JaySon-Huang added a commit to JaySon-Huang/tiflash that referenced this pull request Oct 23, 2019
… PageStorage del meta (pingcap#257)

* Chunk binary version

* Add name to PageStorage for identify different storages

* Remove getMaxId of PageStorage

* Migrate DelPage mark in doing GC

* enable fullstack-test

* Revert "Remove getMaxId of PageStorage"

This reverts commit 34d50eb6e9fb2f229f32e2d6b219b74c340d0d0a.
JaySon-Huang added a commit to JaySon-Huang/tiflash that referenced this pull request Oct 30, 2019
… PageStorage del meta (pingcap#257)

* Chunk binary version

* Add name to PageStorage for identify different storages

* Remove getMaxId of PageStorage

* Migrate DelPage mark in doing GC

* enable fullstack-test

* Revert "Remove getMaxId of PageStorage"

This reverts commit 34d50eb6e9fb2f229f32e2d6b219b74c340d0d0a.
JaySon-Huang added a commit to JaySon-Huang/tiflash that referenced this pull request Nov 1, 2019
… PageStorage del meta (pingcap#257)

* Chunk binary version

* Add name to PageStorage for identify different storages

* Remove getMaxId of PageStorage

* Migrate DelPage mark in doing GC

* enable fullstack-test

* Revert "Remove getMaxId of PageStorage"

This reverts commit 34d50eb6e9fb2f229f32e2d6b219b74c340d0d0a.
JaySon-Huang added a commit to JaySon-Huang/tiflash that referenced this pull request Nov 1, 2019
… PageStorage del meta (pingcap#257)

* Chunk binary version

* Add name to PageStorage for identify different storages

* Remove getMaxId of PageStorage

* Migrate DelPage mark in doing GC

* enable fullstack-test

* Revert "Remove getMaxId of PageStorage"

This reverts commit 34d50eb6e9fb2f229f32e2d6b219b74c340d0d0a.
JaySon-Huang added a commit to JaySon-Huang/tiflash that referenced this pull request Nov 5, 2019
… PageStorage del meta (pingcap#257)

* Chunk binary version

* Add name to PageStorage for identify different storages

* Remove getMaxId of PageStorage

* Migrate DelPage mark in doing GC

* enable fullstack-test

* Revert "Remove getMaxId of PageStorage"

This reverts commit 34d50eb6e9fb2f229f32e2d6b219b74c340d0d0a.
JaySon-Huang pushed a commit that referenced this pull request Aug 6, 2024
JaySon-Huang pushed a commit that referenced this pull request Aug 6, 2024
JaySon-Huang pushed a commit to JaySon-Huang/tiflash that referenced this pull request Aug 7, 2024
Lloyd-Pottiger pushed a commit to Lloyd-Pottiger/tiflash that referenced this pull request Aug 30, 2024
Lloyd-Pottiger pushed a commit to Lloyd-Pottiger/tiflash that referenced this pull request Aug 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants