Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add Tombstone property #847

Merged
merged 4 commits into from
Nov 12, 2024
Merged

add Tombstone property #847

merged 4 commits into from
Nov 12, 2024

Conversation

Deflaimun
Copy link
Contributor

@Deflaimun Deflaimun commented Nov 11, 2024

Changelog

Added properties

  • tombstone_retention_ms

Description

Resolves https://github.com/redpanda-data/documentation-private/issues/
Review deadline:

Page previews

Checks

  • New feature
  • Content gap
  • Support Follow-up
  • Small fix (typos, links, copyedits, etc)

@Deflaimun Deflaimun requested a review from a team as a code owner November 11, 2024 16:03
Copy link

netlify bot commented Nov 11, 2024

Deploy Preview for redpanda-docs-preview ready!

Name Link
🔨 Latest commit 6cb3e60
🔍 Latest deploy log https://app.netlify.com/sites/redpanda-docs-preview/deploys/673373d8ce1a0d0008275b68
😎 Deploy Preview https://deploy-preview-847--redpanda-docs-preview.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

Apply suggestions from code review

remove iceberg
@Deflaimun Deflaimun changed the title Tombstone property add Tombstone property Nov 11, 2024

*Accepted values:* [`-17592186044416`, `17592186044415`]

*Default:* `null`
Copy link

@WillemKauf WillemKauf Nov 11, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if we usually add recommended values for properties that default to null in the docs, but up to your discretion as to whether we want to mention a typical default of 24 hours/86400000 ms (see kafka docs here).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we have recommended values we can add them in the description cc @kbatuigas

Copy link
Contributor Author

@Deflaimun Deflaimun Nov 11, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@WillemKauf per the code, look like this accepts negative values. Should this be a bounded_property to avoid this? I can add 0 as starting point too.

Asking because someties allowing negative numbers in time-based properties is intended

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The valid range is actually defined here: [1ms, 17592186044415] is correct.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I notice some other time-based properties that might need fixing:

https://docs.redpanda.com/current/reference/properties/cluster-properties/#raft_replica_max_flush_delay_ms is also bounded by [1ms, 17592186044415] but is negative in the docs.

https://docs.redpanda.com/current/reference/properties/cluster-properties/#raft_heartbeat_interval_ms is also bounded by [1ms, 17592186044415] but is negative in the docs.

There are a lot more properties in core that should have these bounds enforced, but don't... hm.


*Type:* integer

*Accepted values:* [`-17592186044416`, `17592186044415`]

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[1ms, 17592186044415]

@kbatuigas kbatuigas mentioned this pull request Nov 11, 2024
4 tasks
@@ -4856,6 +4856,24 @@ Transaction manager's synchronization timeout. Maximum time to wait for internal

---

=== tombstone_retention_ms

The retention time for tombstone records in a compacted topic. Cannot be enabled at the same time as any of `cloud_storage_enabled`, `cloud_storage_enable_remote_read`, or `cloud_storage_enable_remote_write`.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
The retention time for tombstone records in a compacted topic. Cannot be enabled at the same time as any of `cloud_storage_enabled`, `cloud_storage_enable_remote_read`, or `cloud_storage_enable_remote_write`.
The retention time (in ms) for tombstone records in a compacted topic. Cannot be enabled at the same time as any of `cloud_storage_enabled`, `cloud_storage_enable_remote_read`, or `cloud_storage_enable_remote_write`. A typical default setting is `86400000`, or 24 hours.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if we should add (in ms) as we already have the units field

*Accepted values:* [`-17592186044416`, `17592186044415`]

*Default:* `null`

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could also add the main doc as a related topic? The xref will be xref:manage:cluster-maintenance/compaction-settings.adoc#tombstone-record-removal[Tombstone record removal]

@kbatuigas kbatuigas merged commit 16e9cda into v-WIP/24.3 Nov 12, 2024
7 checks passed
@kbatuigas kbatuigas deleted the tombstone-property branch November 12, 2024 18:57
kbatuigas pushed a commit that referenced this pull request Nov 13, 2024
Deflaimun added a commit that referenced this pull request Nov 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants