Skip to content

Conversation

CharlieTLe
Copy link
Member

What this PR does:
In v1.15.2, ingesters configured with OOO samples ingestion enabled
could hit this bug (#5402)
where ingesters would not upload compacted blocks
(thanos-io/thanos#6462).

In v1.16.1, ingesters are configured to always upload compacted blocks
(#5625).

In v1.17, ingesters stopped uploading compacted blocks
(#5735).

This can cause problems for users upgrading from v1.15.2 with OOO
ingestion enabled to v1.17 because both versions are hard coded to
disable uploading compacted blocks from the ingesters.

The workaround was to downgrade from v1.17 to v1.16 to allow those
compacted blocks to be uploaded (and eventually deleted).

The new flag is set to true by default which reverts the behavior of the
ingester uploading compacted blocks back to v1.16.

Which issue(s) this PR fixes:
Fixes #5402

Checklist

  • Tests updated
  • Documentation added
  • CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]

… parameterize uploading compacted blocks

In v1.15.2, ingesters configured with OOO samples ingestion enabled
could hit this bug (cortexproject#5402)
where ingesters would not upload compacted blocks
(thanos-io/thanos#6462).

In v1.16.1, ingesters are configured to always upload compacted blocks
(cortexproject#5625).

In v1.17, ingesters stopped uploading compacted blocks
(cortexproject#5735).

This can cause problems for users upgrading from v1.15.2 with OOO
ingestion enabled to v1.17 because both versions are hard coded to
disable uploading compacted blocks from the ingesters.

The workaround was to downgrade from v1.17 to v1.16 to allow those
compacted blocks to be uploaded (and eventually deleted).

The new flag is set to true by default which reverts the behavior of the
ingester uploading compacted blocks back to v1.16.

Signed-off-by: Charlie Le <[email protected]>
@friedrichg friedrichg requested a review from yeya24 May 18, 2024 08:09
Copy link
Contributor

@yeya24 yeya24 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. Good catch!

@yeya24 yeya24 merged commit cb4633e into cortexproject:master May 19, 2024
yeya24 pushed a commit that referenced this pull request May 20, 2024
… parameterize uploading compacted blocks (#5959)

In v1.15.2, ingesters configured with OOO samples ingestion enabled
could hit this bug (#5402)
where ingesters would not upload compacted blocks
(thanos-io/thanos#6462).

In v1.16.1, ingesters are configured to always upload compacted blocks
(#5625).

In v1.17, ingesters stopped uploading compacted blocks
(#5735).

This can cause problems for users upgrading from v1.15.2 with OOO
ingestion enabled to v1.17 because both versions are hard coded to
disable uploading compacted blocks from the ingesters.

The workaround was to downgrade from v1.17 to v1.16 to allow those
compacted blocks to be uploaded (and eventually deleted).

The new flag is set to true by default which reverts the behavior of the
ingester uploading compacted blocks back to v1.16.

Signed-off-by: Charlie Le <[email protected]>
@yeya24 yeya24 mentioned this pull request May 20, 2024
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Unshipped blocks when out of order writes are enabled

2 participants