You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Topic properties are organized into several categories based on their functionality:
182
+
The most commonly configured topic properties fall into these main categories:
183
183
184
184
=== Disk space management
185
185
186
-
Redpanda manages disk space through two main mechanisms: **compaction** (removing duplicate keys) and **retention** (removing old data). Configure the core compaction properties to control how Redpanda cleans up data:
186
+
Redpanda manages disk space through two main mechanisms: **compaction** (removing duplicate keys) and **retention** (removing old data).
187
187
188
-
- Choose cleanup strategy: deletion, compaction, or both (xref:reference:properties/topic-properties.adoc#cleanuppolicy[`cleanup.policy`]).
189
-
- Control when compaction triggers based on dirty data ratio (xref:reference:properties/topic-properties.adoc#mincleanabledirtyratio[`min.cleanable.dirty.ratio`]).
190
-
- Set maximum time before compaction is forced (xref:reference:properties/topic-properties.adoc#maxcompactionlagms[`max.compaction.lag.ms`]).
191
-
- Set minimum time before compaction can occur (xref:reference:properties/topic-properties.adoc#mincompactionlagms[`min.compaction.lag.ms`]).
188
+
Choose your cleanup strategy with xref:reference:properties/topic-properties.adoc#cleanuppolicy[`cleanup.policy`]:
189
+
- `delete` - Remove old data based on time or size limits
190
+
- `compact` - Keep only the latest value for each key
191
+
- `compact,delete` - Combine both strategies
192
192
193
-
=== Write performance and caching
193
+
=== Compaction
194
194
195
-
Control how Redpanda writes and syncs data to disk:
195
+
When using `cleanup.policy=compact` or `cleanup.policy=compact,delete`, configure:
196
196
197
-
- Cache batches until the segment appender chunk is full, instead of fsyncing for every `acks=all` write (xref:reference:properties/topic-properties.adoc#writecaching[`write.caching`]).
198
-
- Configure the maximum delay between two subsequent fsyncs (xref:reference:properties/topic-properties.adoc#flushms[`flush.ms`]).
199
-
- Set the maximum bytes not fsynced per partition (xref:reference:properties/topic-properties.adoc#flushbytes[`flush.bytes`]).
197
+
- xref:reference:properties/topic-properties.adoc#mincleanabledirtyratio[`min.cleanable.dirty.ratio`] - Control when compaction triggers based on dirty data ratio
198
+
- xref:reference:properties/topic-properties.adoc#maxcompactionlagms[`max.compaction.lag.ms`] - Set maximum time before compaction is forced
199
+
- xref:reference:properties/topic-properties.adoc#mincompactionlagms[`min.compaction.lag.ms`] - Set minimum time before compaction can occur
200
200
201
-
=== Message properties
201
+
=== Retention
202
202
203
-
Configure message-specific settings:
203
+
When using `cleanup.policy=delete` or `cleanup.policy=compact,delete`, configure:
204
204
205
-
- Set the source of a message's timestamp (xref:reference:properties/topic-properties.adoc#messagetimestamptype[`message.timestamp.type`]).
206
-
- Set the maximum size of a message (xref:reference:properties/topic-properties.adoc#maxmessagebytes[`max.message.bytes`]).
207
-
- Configure compression for stored data (xref:reference:properties/topic-properties.adoc#compressiontype[`compression.type`]).
205
+
- xref:reference:properties/topic-properties.adoc#retentionbytes[`retention.bytes`] - Maximum size before cleanup (size-based retention)
206
+
- xref:reference:properties/topic-properties.adoc#retentionms[`retention.ms`] - Maximum age before cleanup (time-based retention)
207
+
- xref:reference:properties/topic-properties.adoc#segmentbytes[`segment.bytes`] - Control how frequently cleanup can occur by setting segment size
208
208
209
-
=== Tiered Storage
209
+
=== Performance
210
210
211
-
For topics using Tiered Storage, configure:
211
+
Essential performance tuning properties:
212
212
213
-
- Upload and fetch data to and from object storage (xref:reference:properties/topic-properties.adoc#redpandaremotewrite[`redpanda.remote.write`] and xref:reference:properties/topic-properties.adoc#redpandaremoteread[`redpanda.remote.read`]).
214
-
- Recover data from object storage (xref:reference:properties/topic-properties.adoc#redpandaremoterecovery[`redpanda.remote.recovery`]).
215
-
- Configure local storage retention limits (xref:reference:properties/topic-properties.adoc#retentionlocaltargetbytes[`retention.local.target.bytes`] and xref:reference:properties/topic-properties.adoc#retentionlocaltargetms[`retention.local.target.ms`]).
216
-
217
-
=== Replication and leadership
218
-
219
-
Configure replication and leadership settings:
220
-
221
-
- Number of replicas for the topic (xref:reference:properties/topic-properties.adoc#replicationfactor[`replication.factor`]).
222
-
- Preferred location for partition leaders (xref:reference:properties/topic-properties.adoc#redpandaleaderspreference[`redpanda.leaders.preference`]).
213
+
- xref:reference:properties/topic-properties.adoc#writecaching[`write.caching`] - Cache writes for lower latency with `acks=all`
214
+
- xref:reference:properties/topic-properties.adoc#maxmessagebytes[`max.message.bytes`] - Set maximum message size
215
+
- xref:reference:properties/topic-properties.adoc#replicationfactor[`replication.factor`] - Number of replicas for durability vs. performance
223
216
224
217
For complete details about all available topic properties, see xref:reference:properties/topic-properties.adoc[Topic Configuration Properties].
0 commit comments