feat: add object storage archival docs for audit logs - #4992
Conversation
|
Warning Review limit reachedYou’ve reached a temporary PR review limit under our Fair Usage Limits Policy. Next review available in: 31 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughThis PR updates the audit logs documentation to describe object storage archival for audit events. It adds feature and configuration references, explains dual-write archival behavior, and documents backend examples plus supported object_storage fields. ChangesObject Storage Archival Documentation
Estimated code review effort: 2 (Simple) | ~10 minutes 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
084ae00 to
f8b9147
Compare
b7ffb52 to
32efb42
Compare
f8b9147 to
d570383
Compare
32efb42 to
f7cfc27
Compare
d570383 to
adda25b
Compare
f7cfc27 to
ab24984
Compare
ab24984 to
7df7227
Compare
98750cd to
ef9f232
Compare
7df7227 to
8c146d6
Compare
8c146d6 to
2165d26
Compare
ef9f232 to
c8214e9
Compare
Confidence Score: 4/5This is close, but I would fix the docs before merging.
docs/enterprise/audit-logs.mdx Important Files Changed
Reviews (5): Last reviewed commit: "docs: object storage for audit logs docs" | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
354a812 to
40b27ee
Compare
2165d26 to
4e49611
Compare
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
4e49611 to
040fedc
Compare
40b27ee to
aaa2088
Compare
040fedc to
8c11aeb
Compare
| | `secret_access_key` | string | No | AWS secret access key. Supports `env.` references. | | ||
| | `session_token` | string | No | STS temporary session token. | |
There was a problem hiding this comment.
The table still makes secret_access_key and session_token look independently optional. The config schema requires secret_access_key to be paired with access_key_id, and requires session_token to be paired with both static credential fields. With the current wording, an operator can set only secret_access_key, or only session_token, and get a config validation failure at startup even though the table does not warn about that dependency.
| | `secret_access_key` | string | No | AWS secret access key. Supports `env.` references. | | |
| | `session_token` | string | No | STS temporary session token. | | |
| | `secret_access_key` | string | No | AWS secret access key. Supports `env.` references. Requires `access_key_id`. | | |
| | `session_token` | string | No | STS temporary session token. Requires both `access_key_id` and `secret_access_key`. | |
Rule Used: transports/config.schema.json is the source of tru... (source)
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
Merge activity
|
The base branch was changed.
8c11aeb to
5522720
Compare
|
|
||
| By default, audit events live only in the database, which is the source of truth for everything you see in the dashboard (viewing, filtering, HMAC verification, and export). Databases, however, are not ideal for multi-year compliance retention or off-box durability. | ||
|
|
||
| When you configure `object_storage`, Bifrost **additionally** writes every audit event to an S3-compatible bucket (S3, GCS, MinIO, R2) as it is recorded. This is a dual-write, not an offload: the full event goes to **both** the database and object storage, so each store holds a complete, independent copy. |
There was a problem hiding this comment.
This still describes object storage as a complete independent copy, but the callout below says failed uploads are only logged and the batch is missing from the bucket. An operator can rely on this paragraph for complete long-term retention, then lose a bucket batch during a transient S3/GCS failure while the database write still succeeds. Please qualify this sentence with the best-effort behavior so the completeness guarantee matches the documented failure path.
| | `secret_access_key` | string | No | AWS secret access key. Supports `env.` references. | | ||
| | `session_token` | string | No | STS temporary session token. | |
There was a problem hiding this comment.
These rows still make secret_access_key and session_token look independently optional. The config schema requires secret_access_key to be paired with access_key_id, and requires session_token to be paired with both static credential fields. A user can follow this table, set only secret_access_key or only session_token, and hit config validation failure at startup even though the dependency was not documented on the row they used.
Rule Used: transports/config.schema.json is the source of tru... (source)
## Summary
Documents the new object storage archival feature for audit logs, which enables continuous mirroring of audit events to S3-compatible buckets (S3, GCS, MinIO, R2) for long-term, off-box, compliance-grade retention alongside the existing database store.
## Changes
- Added `object_storage` to the audit log configuration table, with a clarifying note that `retention_days` only governs database retention and does not affect archived objects.
- Added a new "Object storage archival" capability row to the feature overview table.
- Added a full "Archiving to Object Storage" section covering:
- The dual-write model (database remains source of truth; object storage is a complete mirror, not an offload).
- A comparison table distinguishing database vs. object storage roles, retention, and population behavior.
- Object key structure (`{prefix}/audit-logs/{YYYY}/{MM}/{DD}/{HH}/{batchID}.jsonl.gz`) with per-segment explanations.
- Durability guarantees: uploads are synchronous within the flush, failures are logged but never block request handling or database writes.
- Tabbed configuration examples for S3/MinIO/R2 and GCS.
- A full field reference table for all `object_storage` sub-fields, including credential chain behavior and MinIO path-style requirements.
- A note distinguishing this from Log Exports, where object storage offloads payload data rather than mirroring it.
## Type of change
- [ ] Bug fix
- [ ] Feature
- [ ] Refactor
- [x] Documentation
- [ ] Chore/CI
## Affected areas
- [ ] Core (Go)
- [ ] Transports (HTTP)
- [ ] Providers/Integrations
- [ ] Plugins
- [ ] UI (React)
- [x] Docs
## How to test
Review the rendered documentation to confirm:
- The new "Archiving to Object Storage" section renders correctly with tabs, tables, notes, and info callouts.
- The `object_storage` field appears in the configuration table with a working anchor link to the new section.
- The `retention_days` description correctly references the archiving section.
- Tab groups for S3 and GCS configuration examples display and switch correctly.
## Breaking changes
- [ ] Yes
- [x] No
## Related issues
## Security considerations
Configuration examples demonstrate use of `env.` prefixes for secrets (`AUDIT_HMAC_KEY`, `AUDIT_S3_SECRET`, `AUDIT_GCS_CREDENTIALS`), reinforcing that credentials should never be hardcoded in config files. The archival bucket can be configured as write-once/WORM via S3 Object Lock for tamper-evident compliance retention.
## Checklist
- [ ] I read `docs/contributing/README.md` and followed the guidelines
- [ ] I added/updated tests where appropriate
- [x] I updated documentation where needed
- [ ] I verified builds succeed (Go and UI)
- [ ] I verified the CI pipeline passes locally if applicable
## Summary
Documents the new object storage archival feature for audit logs, which enables continuous mirroring of audit events to S3-compatible buckets (S3, GCS, MinIO, R2) for long-term, off-box, compliance-grade retention alongside the existing database store.
## Changes
- Added `object_storage` to the audit log configuration table, with a clarifying note that `retention_days` only governs database retention and does not affect archived objects.
- Added a new "Object storage archival" capability row to the feature overview table.
- Added a full "Archiving to Object Storage" section covering:
- The dual-write model (database remains source of truth; object storage is a complete mirror, not an offload).
- A comparison table distinguishing database vs. object storage roles, retention, and population behavior.
- Object key structure (`{prefix}/audit-logs/{YYYY}/{MM}/{DD}/{HH}/{batchID}.jsonl.gz`) with per-segment explanations.
- Durability guarantees: uploads are synchronous within the flush, failures are logged but never block request handling or database writes.
- Tabbed configuration examples for S3/MinIO/R2 and GCS.
- A full field reference table for all `object_storage` sub-fields, including credential chain behavior and MinIO path-style requirements.
- A note distinguishing this from Log Exports, where object storage offloads payload data rather than mirroring it.
## Type of change
- [ ] Bug fix
- [ ] Feature
- [ ] Refactor
- [x] Documentation
- [ ] Chore/CI
## Affected areas
- [ ] Core (Go)
- [ ] Transports (HTTP)
- [ ] Providers/Integrations
- [ ] Plugins
- [ ] UI (React)
- [x] Docs
## How to test
Review the rendered documentation to confirm:
- The new "Archiving to Object Storage" section renders correctly with tabs, tables, notes, and info callouts.
- The `object_storage` field appears in the configuration table with a working anchor link to the new section.
- The `retention_days` description correctly references the archiving section.
- Tab groups for S3 and GCS configuration examples display and switch correctly.
## Breaking changes
- [ ] Yes
- [x] No
## Related issues
## Security considerations
Configuration examples demonstrate use of `env.` prefixes for secrets (`AUDIT_HMAC_KEY`, `AUDIT_S3_SECRET`, `AUDIT_GCS_CREDENTIALS`), reinforcing that credentials should never be hardcoded in config files. The archival bucket can be configured as write-once/WORM via S3 Object Lock for tamper-evident compliance retention.
## Checklist
- [ ] I read `docs/contributing/README.md` and followed the guidelines
- [ ] I added/updated tests where appropriate
- [x] I updated documentation where needed
- [ ] I verified builds succeed (Go and UI)
- [ ] I verified the CI pipeline passes locally if applicable
2 participants
Summary
Documents the new object storage archival feature for audit logs, which enables continuous mirroring of audit events to S3-compatible buckets (S3, GCS, MinIO, R2) for long-term, off-box, compliance-grade retention alongside the existing database store.
Changes
object_storageto the audit log configuration table, with a clarifying note thatretention_daysonly governs database retention and does not affect archived objects.{prefix}/audit-logs/{YYYY}/{MM}/{DD}/{HH}/{batchID}.jsonl.gz) with per-segment explanations.object_storagesub-fields, including credential chain behavior and MinIO path-style requirements.Type of change
Affected areas
How to test
Review the rendered documentation to confirm:
object_storagefield appears in the configuration table with a working anchor link to the new section.retention_daysdescription correctly references the archiving section.Breaking changes
Related issues
Security considerations
Configuration examples demonstrate use of
env.prefixes for secrets (AUDIT_HMAC_KEY,AUDIT_S3_SECRET,AUDIT_GCS_CREDENTIALS), reinforcing that credentials should never be hardcoded in config files. The archival bucket can be configured as write-once/WORM via S3 Object Lock for tamper-evident compliance retention.Checklist
docs/contributing/README.mdand followed the guidelines