Update Glue doc with Cloud-specific guidance #1275
Conversation
|
Important Review skippedAuto incremental reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the You can disable this status message by setting the 📝 WalkthroughWalkthroughThis pull request updates the Antora playbook configuration to reference the Sequence Diagram(s)sequenceDiagram
participant User
participant Docs
participant Redpanda Cluster
User->>Docs: Access Iceberg AWS Glue documentation
Docs-->>User: Show prerequisites (version, install doc link)
alt Non-cloud environment
User->>Docs: View non-cloud configuration
Docs-->>User: Show S3/shared credentials or override options
User->>Redpanda Cluster: Configure cluster with S3 or Glue credentials
else Cloud environment
User->>Docs: View cloud configuration
Docs-->>User: Show AWS-specific cluster property requirements
User->>Redpanda Cluster: Configure cluster with required AWS Glue properties (access_key, secret_key, region)
end
Estimated code review effort🎯 2 (Simple) | ⏱️ ~7 minutes Assessment against linked issues
Assessment against linked issues: Out-of-scope changesNo out-of-scope changes found. Possibly related PRs
Suggested reviewers
✨ Finishing Touches🧪 Generate unit tests
🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
✅ Deploy Preview for redpanda-docs-preview ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (4)
modules/manage/pages/iceberg/iceberg-topics-aws-glue.adoc (4)
118-124: Commented credentials block – consider showing both credential flavoursRight now the example shows the override properties only as commented lines. Users may miss that they still need
cloud_storage_*if they stick with shared credentials. A brief comment above the snippet such as “Either keep these commented lines or delete them to reuse S3 credentials” would reduce ambiguity.
128-132: Placeholder list: missing full stop causes Antora warningAsciiDoc treats the
--delimited block as a dlist description. Term lines must end with::or a period. Add a trailing period or convert to an unordered list to avoid “dangling description” warnings.- - `<bucket-name>` and `<warehouse-path>`: AWS Glue requires ... + - `<bucket-name>` and `<warehouse-path>`:: AWS Glue requires ...
149-151: Secret macro render-ability
${secrets.<glue-secret-key-name>}renders literally in the docs site because Antora does not evaluate shell-style expansions.
Wrap it in backticks or change tosecrets.<glue-secret-key-name>so the text is shown as-is.- iceberg_rest_catalog_aws_secret_key=${secrets.<glue-secret-key-name>} + iceberg_rest_catalog_aws_secret_key=`secrets.<glue-secret-key-name>`
157-162: Consistent placeholder orderingSwap
<glue-region>and<glue-access-key>to keep region-specific placeholders grouped together (<aws-region>,<glue-region>).
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
local-antora-playbook.yml(1 hunks)modules/manage/pages/iceberg/iceberg-topics-aws-glue.adoc(4 hunks)
🧰 Additional context used
🧠 Learnings (2)
📚 Learning: in the redpanda documentation, topic property cross-references like <> and <<...
Learnt from: Feediver1
PR: redpanda-data/docs#1153
File: modules/reference/pages/properties/topic-properties.adoc:45-50
Timestamp: 2025-07-16T19:33:20.420Z
Learning: In the Redpanda documentation, topic property cross-references like <<max.compaction.lag.ms>> and <<min.compaction.lag.ms>> require corresponding property definition sections with anchors like [[maxcompactionlagms]] and [[mincompactionlagms]] to prevent broken links.
Applied to files:
modules/manage/pages/iceberg/iceberg-topics-aws-glue.adoc
📚 Learning: in asciidoc documentation for redpanda, the syntax `+` and `--` around content blocks within a `[tab...
Learnt from: kbatuigas
PR: redpanda-data/docs#1113
File: modules/manage/partials/iceberg/use-iceberg-catalogs.adoc:100-107
Timestamp: 2025-05-07T01:06:00.937Z
Learning: In AsciiDoc documentation for Redpanda, the syntax `+` and `--` around content blocks within a `[tabs]` section are valid AsciiDoc formatting elements for tabbed content. The `+` after a tab name (like `rpk::`) indicates that the following block belongs to that tab, and the `--` markers enclose the content for that tab. These are not diff artifacts and should not be removed.
Applied to files:
modules/manage/pages/iceberg/iceberg-topics-aws-glue.adoc
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
- GitHub Check: Redirect rules - redpanda-docs-preview
- GitHub Check: Header rules - redpanda-docs-preview
- GitHub Check: Pages changed - redpanda-docs-preview
🔇 Additional comments (5)
local-antora-playbook.yml (1)
20-20: Branch pin may break once the feature branch is deletedPointing the content source to a short-lived feature branch (
DOC-1377-cloud-aws-glue) means the docs build will 404 once that branch is deleted after merge.
Consider either:- - url: https://github.com/redpanda-data/cloud-docs - branches: 'DOC-1377-cloud-aws-glue' + - url: https://github.com/redpanda-data/cloud-docs + branches: ['main', 'DOC-1377-cloud-aws-glue'] # fall back to mainor reverting this change as soon as the feature branch is merged into
main.modules/manage/pages/iceberg/iceberg-topics-aws-glue.adoc (4)
26-27: Attribute reuse looks goodUsing
{rp_version}and{rpk_install_doc}removes hard-coding and will auto-update when the attributes are bumped.
83-92: Cross-reference anchors could be missingThe four new
iceberg_rest_catalog_aws_*properties are referenced viaconfig_ref. Ensure they are defined in
modules/reference/pages/properties/cluster-properties.adoc(each needs[[iceberg_rest_catalog_aws_access_key]]etc.).
A missing anchor silently renders as plain text and breaks the link checker.
94-100: Secret reference wording is clearCalling out that
iceberg_rest_catalog_aws_secret_keymust be a secret and pointing to the next section is helpful.
12-20: Version split is intentional for Cloud (beta) vs self-hosted (GA)
The AWS Glue integration is marked Beta on Redpanda Cloud—hence the:rp_version: 25.2in the Cloud block—while self-hosted users need only the minimum GA release (25.1.7) as noted in the release notes (“The AWS Glue catalog integration is available in Redpanda version 25.1.7 and later”). No change needed.Likely an incorrect or invalid review comment.
|
@simon0191 @mattschumpert not currently part of the diff - How should the guidance in the Authorize access to AWS Glue section change for Cloud? For example, I don't know if this is relevant to Cloud:
Do customers create a new IAM user in AWS and attach a policy with Glue permissions like in the example provided? The instructions for using |
|
@kbatuigas For self-hosted its different. They must be creating the IAM role/policy or using access keys for Tiered Storage on their own. |
| endif::[] | ||
|
|
||
| ifdef::env-cloud[] | ||
| You must configure credentials for the AWS Glue Data Catalog integration using the following properties: |
There was a problem hiding this comment.
Just double checking, are these always needed? Do the cloud_storage_* credentials not work in cloud?
There was a problem hiding this comment.
That's right -- cloud_storage_* properties are not exposed in Cloud because we manage the cloud storage bucket.
kbatuigas
deleted the
DOC-1548-aws-glue-in-cloud-must-use-aws-specific-cluster-pr
branch
Description
Related PR to add Glue doc to Cloud: redpanda-data/cloud-docs#363
This pull request introduces updates to the Antora playbook and documentation for integrating Redpanda topics with AWS Glue Data Catalog. The changes include branch updates in the playbook, dynamic versioning for prerequisites, and detailed instructions for configuring credentials and cluster properties for AWS Glue integration. Below is a summary of the most important changes:
Playbook Updates
cloud-docsrepository in thelocal-antora-playbook.ymlfile toDOC-1377-cloud-aws-gluefor more targeted documentation updates.Documentation Enhancements for AWS Glue Integration
Dynamic Versioning and Prerequisite Updates:
rp_versionandrpk_install_doc) to specify Redpanda version andrpkinstallation instructions based on the environment (env-cloudor non-cloud). This ensures accurate and up-to-date prerequisites for users.Credential Configuration:
Cluster Property Placeholders:
<aws-region>,<bucket-name>,<warehouse-path>,<glue-access-key>, and<glue-secret-key-name>to guide users in setting up their configurations securely and correctly. [1] [2]These changes aim to improve clarity, flexibility, and precision in the documentation, making it easier for users to integrate Redpanda with AWS Glue.
Resolves https://redpandadata.atlassian.net/browse/
Review deadline: 4 Aug
Page previews
Cloud: Query Iceberg Topics Using AWS Glue
Self-managed version: Query Iceberg Topics Using AWS Glue
Checks