docs: update audit log archival to document windowed, manifest-based archival with tuning fields - #5234
Merged
Pratham-Mishra04 merged 1 commit intoJul 17, 2026
Conversation
Contributor
|
Warning Review limit reachedYou’ve reached a temporary PR review limit under our Fair Usage Limits Policy. Next review available in: 21 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughUpdates Audit Logs documentation for periodic, windowed object-storage mirroring with new tuning fields, manifest-based progress, retry behavior, and restart or shutdown handling. ChangesAudit log object-storage archival
Estimated code review effort: 3 (Moderate) | ~20 minutes Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
impoiler
marked this pull request as ready for review
July 15, 2026 09:14
18 tasks
Contributor
Author
impoiler
changed the title
Contributor
Confidence Score: 4/5The configuration guidance should be corrected before merging.
docs/enterprise/audit-logs.mdx Important Files Changed
Reviews (4): Last reviewed commit: "feat: allow time interval and size flexi..." | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
impoiler
force-pushed
the
07-15-feat_allow_time_interval_and_size_flexibility_in_audit_logs_in_object_storage_docs
branch
from
July 15, 2026 09:37
7bb37d8 to
0b79936
Compare
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/enterprise/audit-logs.mdx`:
- Line 115: Update the archive lag wording in the “When populated” table entry
from “at least archive_interval + archive_grace_period” to “up to
archive_interval + archive_grace_period,” preserving the existing conditions and
explanation.
- Around line 163-176: Correct the manifest example around the parts array so
the reported bytes for part-00000 align with the documented
archive_max_object_bytes rolling rule. Adjust the example’s byte value or the
stated rolling explanation consistently, while preserving the example’s event
counts and overall manifest structure.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: 2026354c-1aae-40d2-bf33-d467b220a4d4
📒 Files selected for processing (1)
docs/enterprise/audit-logs.mdx
9 tasks
impoiler
force-pushed
the
07-15-feat_allow_time_interval_and_size_flexibility_in_audit_logs_in_object_storage
branch
from
July 16, 2026 06:08
78dadcf to
c0ea503
Compare
impoiler
force-pushed
the
07-15-feat_allow_time_interval_and_size_flexibility_in_audit_logs_in_object_storage_docs
branch
from
July 16, 2026 06:08
0b79936 to
b7f51ef
Compare
This was referenced Jul 16, 2026
impoiler
force-pushed
the
07-15-feat_allow_time_interval_and_size_flexibility_in_audit_logs_in_object_storage_docs
branch
from
July 16, 2026 06:09
b7f51ef to
e11b090
Compare
impoiler
force-pushed
the
07-15-feat_allow_time_interval_and_size_flexibility_in_audit_logs_in_object_storage
branch
from
July 16, 2026 06:09
c0ea503 to
a6e8d94
Compare
This was referenced Jul 16, 2026
Collaborator
Merge activity
|
Pratham-Mishra04
changed the base branch from
07-15-feat_allow_time_interval_and_size_flexibility_in_audit_logs_in_object_storage
to
graphite-base/5234
July 17, 2026 11:40
Pratham-Mishra04
dismissed
coderabbitai[bot]’s stale review
July 17, 2026 11:43
The base branch was changed.
Pratham-Mishra04
force-pushed
the
07-15-feat_allow_time_interval_and_size_flexibility_in_audit_logs_in_object_storage_docs
branch
from
July 17, 2026 11:43
e11b090 to
5895115
Compare
| <Note> | ||
| The three `archive_*` fields only take effect when `object_storage` is configured. | ||
|
|
||
| `config.schema.json` constrains these fields, so an editor wired to it (via `$schema`) will flag a malformed duration like `"6 hours"` or an out-of-range `archive_max_object_bytes` as you type. The gateway itself does **not** reject such values at startup: each field is normalized independently at runtime, so a value that gets past the schema **falls back to its default or is clamped into range rather than failing startup**. A typo like `"6 hours"` silently yields the 24h default, and `archive_interval: "10s"` is quietly raised to the 5m floor. |
Contributor
There was a problem hiding this comment.
transports/config.schema.json while loading it. A malformed duration such as "6 hours" or an object size outside 1 MiB–4 GiB therefore fails validation before runtime normalization runs. Only schema-valid values, such as archive_interval: "10s", can reach the clamping behavior described here. As written, operators can expect a silent fallback but instead see the gateway fail to start.
Suggested change
| `config.schema.json` constrains these fields, so an editor wired to it (via `$schema`) will flag a malformed duration like `"6 hours"` or an out-of-range `archive_max_object_bytes` as you type. The gateway itself does **not** reject such values at startup: each field is normalized independently at runtime, so a value that gets past the schema **falls back to its default or is clamped into range rather than failing startup**. A typo like `"6 hours"` silently yields the 24h default, and `archive_interval: "10s"` is quietly raised to the 5m floor. | |
| `config.schema.json` constrains these fields, and the gateway validates configuration against it at startup. A malformed duration like `"6 hours"` or an out-of-range `archive_max_object_bytes` therefore fails startup validation. Values that pass schema validation are then normalized at runtime; for example, `archive_interval: "10s"` is quietly raised to the 5m floor. |
Pratham-Mishra04
deleted the
07-15-feat_allow_time_interval_and_size_flexibility_in_audit_logs_in_object_storage_docs
branch
July 17, 2026 11:45
akshaydeo
pushed a commit
that referenced
this pull request
Jul 17, 2026
…archival with tuning fields (#5234) ## Summary Updates the audit log archival documentation to reflect a redesigned background-job-based archival system, replacing the previous per-flush, best-effort write model with a windowed, manifest-committed approach that provides stronger delivery guarantees. ## Changes - Clarified that archival is now a **periodic background job** operating on fixed time windows rather than a synchronous per-flush upload, and updated all descriptions accordingly. - Documented three new configuration fields: `archive_interval`, `archive_grace_period`, and `archive_max_object_bytes`, including their defaults, validation behavior, and silent fallback-to-default on invalid values. - Updated the object key schema from per-batch UUIDs under an hourly prefix to a windowed `{start}-{end}/part-NNNNN.jsonl[.gz]` layout with a `manifest.json` commit record. - Added a full explanation of the archival lifecycle: window eligibility via grace period, job deduplication across multi-node clusters, part rolling by size, and manifest-as-commit-point semantics. - Added a manifest JSON example and guidance that consumers must read windows through their manifests to avoid orphaned parts from retried runs. - Replaced the "best-effort" warning with a delivery guarantees table covering at-least-once, contiguous, and first-complete-write-wins semantics. - Added a warning that `retention_days` and archival are independent loops, with explicit guidance that `retention_days: 1` races the archiver under default settings. - Documented catch-up behavior after outages, watermark persistence across restarts, and graceful shutdown behavior. - Added a tuning section covering the memory implications of `archive_max_object_bytes` and when to adjust `archive_grace_period`. - Added `archive_interval`, `archive_grace_period`, and `archive_max_object_bytes` to both the S3 and GCS configuration examples. ## Type of change - [ ] Bug fix - [ ] Feature - [ ] Refactor - [x] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [x] Docs ## How to test Review the rendered documentation for accuracy against the archival implementation. Confirm: - The object key examples match what the archiver actually writes. - The manifest JSON schema matches the struct the archiver serializes. - Default values for `archive_interval` (`24h`), `archive_grace_period` (`15m`), and `archive_max_object_bytes` (`134217728`) match the implementation. - The `retention_days: 2` minimum safe value holds given the default window and grace period. ## Breaking changes - [x] Yes - [ ] No The object key layout has changed from `{prefix}/audit-logs/{YYYY}/{MM}/{DD}/{HH}/{batchID}.jsonl.gz` to `{prefix}/audit-logs/{YYYY}/{MM}/{DD}/{start}-{end}/part-NNNNN.jsonl[.gz]`. Any existing tooling, lifecycle rules, or SIEM ingestion pipelines keyed on the old path structure will need to be updated. Objects written under the old scheme are not affected retroactively, but new archives will not appear at the old paths. ## Related issues N/A ## Security considerations No new secrets or PII handling introduced. The `archive_grace_period` and watermark persistence are relevant to compliance posture — operators should confirm `retention_days` is set high enough to avoid rows being deleted before they are archived. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [x] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
akshaydeo
pushed a commit
that referenced
this pull request
Jul 18, 2026
…archival with tuning fields (#5234) ## Summary Updates the audit log archival documentation to reflect a redesigned background-job-based archival system, replacing the previous per-flush, best-effort write model with a windowed, manifest-committed approach that provides stronger delivery guarantees. ## Changes - Clarified that archival is now a **periodic background job** operating on fixed time windows rather than a synchronous per-flush upload, and updated all descriptions accordingly. - Documented three new configuration fields: `archive_interval`, `archive_grace_period`, and `archive_max_object_bytes`, including their defaults, validation behavior, and silent fallback-to-default on invalid values. - Updated the object key schema from per-batch UUIDs under an hourly prefix to a windowed `{start}-{end}/part-NNNNN.jsonl[.gz]` layout with a `manifest.json` commit record. - Added a full explanation of the archival lifecycle: window eligibility via grace period, job deduplication across multi-node clusters, part rolling by size, and manifest-as-commit-point semantics. - Added a manifest JSON example and guidance that consumers must read windows through their manifests to avoid orphaned parts from retried runs. - Replaced the "best-effort" warning with a delivery guarantees table covering at-least-once, contiguous, and first-complete-write-wins semantics. - Added a warning that `retention_days` and archival are independent loops, with explicit guidance that `retention_days: 1` races the archiver under default settings. - Documented catch-up behavior after outages, watermark persistence across restarts, and graceful shutdown behavior. - Added a tuning section covering the memory implications of `archive_max_object_bytes` and when to adjust `archive_grace_period`. - Added `archive_interval`, `archive_grace_period`, and `archive_max_object_bytes` to both the S3 and GCS configuration examples. ## Type of change - [ ] Bug fix - [ ] Feature - [ ] Refactor - [x] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [x] Docs ## How to test Review the rendered documentation for accuracy against the archival implementation. Confirm: - The object key examples match what the archiver actually writes. - The manifest JSON schema matches the struct the archiver serializes. - Default values for `archive_interval` (`24h`), `archive_grace_period` (`15m`), and `archive_max_object_bytes` (`134217728`) match the implementation. - The `retention_days: 2` minimum safe value holds given the default window and grace period. ## Breaking changes - [x] Yes - [ ] No The object key layout has changed from `{prefix}/audit-logs/{YYYY}/{MM}/{DD}/{HH}/{batchID}.jsonl.gz` to `{prefix}/audit-logs/{YYYY}/{MM}/{DD}/{start}-{end}/part-NNNNN.jsonl[.gz]`. Any existing tooling, lifecycle rules, or SIEM ingestion pipelines keyed on the old path structure will need to be updated. Objects written under the old scheme are not affected retroactively, but new archives will not appear at the old paths. ## Related issues N/A ## Security considerations No new secrets or PII handling introduced. The `archive_grace_period` and watermark persistence are relevant to compliance posture — operators should confirm `retention_days` is set high enough to avoid rows being deleted before they are archived. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [x] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
1 task
akhsaul
pushed a commit
to akhsaul/bifrost
that referenced
this pull request
Aug 27, 2026
…archival with tuning fields (maximhq#5234) ## Summary Updates the audit log archival documentation to reflect a redesigned background-job-based archival system, replacing the previous per-flush, best-effort write model with a windowed, manifest-committed approach that provides stronger delivery guarantees. ## Changes - Clarified that archival is now a **periodic background job** operating on fixed time windows rather than a synchronous per-flush upload, and updated all descriptions accordingly. - Documented three new configuration fields: `archive_interval`, `archive_grace_period`, and `archive_max_object_bytes`, including their defaults, validation behavior, and silent fallback-to-default on invalid values. - Updated the object key schema from per-batch UUIDs under an hourly prefix to a windowed `{start}-{end}/part-NNNNN.jsonl[.gz]` layout with a `manifest.json` commit record. - Added a full explanation of the archival lifecycle: window eligibility via grace period, job deduplication across multi-node clusters, part rolling by size, and manifest-as-commit-point semantics. - Added a manifest JSON example and guidance that consumers must read windows through their manifests to avoid orphaned parts from retried runs. - Replaced the "best-effort" warning with a delivery guarantees table covering at-least-once, contiguous, and first-complete-write-wins semantics. - Added a warning that `retention_days` and archival are independent loops, with explicit guidance that `retention_days: 1` races the archiver under default settings. - Documented catch-up behavior after outages, watermark persistence across restarts, and graceful shutdown behavior. - Added a tuning section covering the memory implications of `archive_max_object_bytes` and when to adjust `archive_grace_period`. - Added `archive_interval`, `archive_grace_period`, and `archive_max_object_bytes` to both the S3 and GCS configuration examples. ## Type of change - [ ] Bug fix - [ ] Feature - [ ] Refactor - [x] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [x] Docs ## How to test Review the rendered documentation for accuracy against the archival implementation. Confirm: - The object key examples match what the archiver actually writes. - The manifest JSON schema matches the struct the archiver serializes. - Default values for `archive_interval` (`24h`), `archive_grace_period` (`15m`), and `archive_max_object_bytes` (`134217728`) match the implementation. - The `retention_days: 2` minimum safe value holds given the default window and grace period. ## Breaking changes - [x] Yes - [ ] No The object key layout has changed from `{prefix}/audit-logs/{YYYY}/{MM}/{DD}/{HH}/{batchID}.jsonl.gz` to `{prefix}/audit-logs/{YYYY}/{MM}/{DD}/{start}-{end}/part-NNNNN.jsonl[.gz]`. Any existing tooling, lifecycle rules, or SIEM ingestion pipelines keyed on the old path structure will need to be updated. Objects written under the old scheme are not affected retroactively, but new archives will not appear at the old paths. ## Related issues N/A ## Security considerations No new secrets or PII handling introduced. The `archive_grace_period` and watermark persistence are relevant to compliance posture — operators should confirm `retention_days` is set high enough to avoid rows being deleted before they are archived. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [x] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Updates the audit log archival documentation to reflect a redesigned background-job-based archival system, replacing the previous per-flush, best-effort write model with a windowed, manifest-committed approach that provides stronger delivery guarantees.
Changes
archive_interval,archive_grace_period, andarchive_max_object_bytes, including their defaults, validation behavior, and silent fallback-to-default on invalid values.{start}-{end}/part-NNNNN.jsonl[.gz]layout with amanifest.jsoncommit record.retention_daysand archival are independent loops, with explicit guidance thatretention_days: 1races the archiver under default settings.archive_max_object_bytesand when to adjustarchive_grace_period.archive_interval,archive_grace_period, andarchive_max_object_bytesto both the S3 and GCS configuration examples.Type of change
Affected areas
How to test
Review the rendered documentation for accuracy against the archival implementation. Confirm:
archive_interval(24h),archive_grace_period(15m), andarchive_max_object_bytes(134217728) match the implementation.retention_days: 2minimum safe value holds given the default window and grace period.Breaking changes
The object key layout has changed from
{prefix}/audit-logs/{YYYY}/{MM}/{DD}/{HH}/{batchID}.jsonl.gzto{prefix}/audit-logs/{YYYY}/{MM}/{DD}/{start}-{end}/part-NNNNN.jsonl[.gz]. Any existing tooling, lifecycle rules, or SIEM ingestion pipelines keyed on the old path structure will need to be updated. Objects written under the old scheme are not affected retroactively, but new archives will not appear at the old paths.Related issues
N/A
Security considerations
No new secrets or PII handling introduced. The
archive_grace_periodand watermark persistence are relevant to compliance posture — operators should confirmretention_daysis set high enough to avoid rows being deleted before they are archived.Checklist
docs/contributing/README.mdand followed the guidelines