Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions modules/manage/pages/iceberg/specify-iceberg-schema.adoc
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting, does Antora/AsciiDoc not automatically generate the anchor IDs for these?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought it did but the anchor was not resolving. Not sure why.

Original file line number Diff line number Diff line change
Expand Up @@ -25,16 +25,19 @@ ifdef::env-cloud[]
Redpanda supports the following modes for Iceberg topics:
endif::[]

[[key_value]]
=== key_value

Creates an Iceberg table using a simple schema, consisting of two columns, one for the record metadata including the key, and another binary column for the record's value.

[[value_schema_id_prefix]]
=== value_schema_id_prefix

Creates an Iceberg table whose structure matches the Redpanda schema for the topic, with columns corresponding to each field. You must register a schema in the xref:manage:schema-reg/schema-reg-overview.adoc[Schema Registry] and producers must write to the topic using the Schema Registry wire format.

In the xref:manage:schema-reg/schema-reg-overview.adoc#wire-format[Schema Registry wire format], a "magic byte" and schema ID are embedded in the message payload header. Producers to the topic must use the wire format in the serialization process so Redpanda can determine the schema used for each record, use the schema to define the Iceberg table, and store the topic values in the corresponding table columns.

[[value_schema_latest]]
=== value_schema_latest

Creates an Iceberg table whose structure matches the latest schema registered for the subject in the Schema Registry. You must register a schema in the Schema Registry.
Expand All @@ -45,6 +48,7 @@ NOTE: The `value_schema_latest` mode is not compatible with the xref:reference:r

The latest schema is cached periodically. The cache period is defined by the cluster property `iceberg_latest_schema_cache_ttl_ms` (default: 5 minutes).

[[disabled]]
=== disabled

Default for `redpanda.iceberg.mode`. Disables writing to an Iceberg table for the topic.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2201,7 +2201,7 @@ endif::[]

=== iceberg_latest_schema_cache_ttl_ms

The TTL for caching the latest schema during translation when using the `latest_protobuf_value` iceberg mode. This setting controls how long the latest schema remains cached during translation, which affects schema refresh behavior and performance.
The TTL for caching the latest schema during translation when using the xref:manage:iceberg/specify-iceberg-schema.adoc#value_schema_latest[`value_schema_latest`] iceberg mode. This setting controls how long the latest schema remains cached during translation, which affects schema refresh behavior and performance.

*Unit:* milliseconds

Expand Down