Skip to content

docs(rfc): explicit OURIOS_S3_* credential env keys (RFC 0019 §3.4 + RFC0019.8) - #306

Merged
jensholdgaard merged 5 commits into
mainfrom
rfc0019-credential-config-spec
Jun 28, 2026
Merged

docs(rfc): explicit OURIOS_S3_* credential env keys (RFC 0019 §3.4 + RFC0019.8)#306
jensholdgaard merged 5 commits into
mainfrom
rfc0019-credential-config-spec

Conversation

@jensholdgaard

@jensholdgaard jensholdgaard commented Jun 28, 2026

Copy link
Copy Markdown
Owner

What

Amends RFC 0019 §3.4 so S3 credentials can be supplied via S3-named env
vars (OURIOS_S3_ACCESS_KEY_ID / OURIOS_S3_SECRET_ACCESS_KEY /
OURIOS_S3_SESSION_TOKEN) instead of only the AWS-SDK-named credential chain.
Ourios is S3-compatible (AWS S3 and MinIO/R2/Hetzner/Ceph/…), so its own
credential surface should not read as AWS-specific.

Spec-only — no code. The implementation lands in a follow-up PR; the
S3-native Helm chart (#304) then adopts the S3-named Secret keys and merges.

Design (two-layer, explicit-over-chain)

  1. Explicit Ourios configOURIOS_S3_* keys are read and applied to the
    AmazonS3Builder (with_access_key_id / with_secret_access_key /
    with_token). These authenticate AWS S3 and every S3-compatible store
    identically.
  2. Credential chain (fallback) — when the explicit keys are unset,
    AmazonS3Builder::from_env() resolves the usual way (static AWS_*, shared
    profile, IRSA, instance metadata). Retained deliberately: AWS IRSA
    injects its own AWS_ROLE_ARN / AWS_WEB_IDENTITY_TOKEN_FILE (EKS webhook),
    which have no Ourios-named equivalent — on EKS the chain is the source.

Invariant touched — §3.4 secret hygiene (RFC0019.6)

Widened, not relaxed. Ourios now reads credential material (it read none
before), so it owns redaction: an OURIOS_S3_* secret value is never echoed in
a config error, StoreError, log line, or metric attribute. The
missing-required error still names only the key (OURIOS_S3_BUCKET). New
scenario RFC0019.8 gates both the explicit-credential authentication path
(localstack, OURIOS_S3_* only, no AWS_*) and the widened redaction
(extends the rfc0019_6_* unit test).

Verification

  • mdbook build — clean.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Documentation
    • Updated the storage backend selection RFC with a new amendment covering optional S3 credential environment variables.
    • Clarified how explicit S3 credentials interact with existing AWS credential handling.
    • Added guidance on validation rules, secret redaction, acceptance criteria, and test coverage expectations.

…RFC0019.8)

RFC 0019 §3.4 currently states "No Ourios-specific credential config is
introduced" and relies entirely on the AWS credential chain (the AWS_*-named
env vars object_store's AmazonS3Builder::from_env() reads). Ourios is
S3-compatible, not AWS-specific, so the credential env surface should be
S3-named too.

Amend §3.4 to a two-layer, explicit-over-chain model: Ourios reads
OURIOS_S3_ACCESS_KEY_ID / OURIOS_S3_SECRET_ACCESS_KEY / OURIOS_S3_SESSION_TOKEN
and applies them to the builder; the from_env() chain is retained as the
fallback (it is the only path for AWS IRSA, whose AWS_ROLE_ARN /
AWS_WEB_IDENTITY_TOKEN_FILE the EKS webhook injects and which have no
Ourios-named equivalent).

Secret hygiene (§3.4 / RFC0019.6) is widened, not relaxed: Ourios now *reads*
credential material (it read none before), so it owns redaction of the
OURIOS_S3_* secret keys across config errors, StoreError, logs, and metric
attributes. New scenario RFC0019.8 gates both the explicit-credential
authentication path and the widened redaction; it is greened by the
implementation PR.

Spec-only (no code). mdbook build verified.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@jensholdgaard
jensholdgaard requested a review from Copilot June 28, 2026 19:07
@coderabbitai

coderabbitai Bot commented Jun 28, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@jensholdgaard, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 28 minutes and 42 seconds. Learn how PR review limits work.

Your organization has used up its prepaid credits, and credit purchases are no longer available. Enable the review add-on in the billing tab to keep reviews running — you're only billed for reviews past your plan's rate limits ($0.25/file).

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based credits.

🚦 How do rate limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: ee33c7ed-0065-41d9-aaff-e0e36f2507d4

📥 Commits

Reviewing files that changed from the base of the PR and between 2bedf75 and e1af838.

📒 Files selected for processing (1)
  • docs/rfcs/0019-storage-backend-selection.md
📝 Walkthrough

Walkthrough

Adds a "§9 Amendment (2026-06-28)" to RFC0019, defining optional OURIOS_S3_* credential environment variables, a two-layer resolution model over the standard AWS chain, expanded secret-redaction rules, a new acceptance criterion (RFC0019.8), and a testing plan. Forward-reference notes are inserted in §3.1 and §3.4.

RFC0019 §9 Amendment

Layer / File(s) Summary
Forward references in §3.1 and §3.4
docs/rfcs/0019-storage-backend-selection.md
§3.1 gains a note pointing to the §9 amendment for OURIOS_S3_* config; §3.4 gains a dated note that the amendment is pending and the existing green behavior remains valid.
§9 Amendment body: vars, resolution, hygiene, criteria, tests
docs/rfcs/0019-storage-backend-selection.md
Defines OURIOS_S3_ACCESS_KEY_ID, OURIOS_S3_SECRET_ACCESS_KEY, OURIOS_S3_SESSION_TOKEN; specifies two-layer resolution (explicit via AmazonS3Builder, fallback to AmazonS3Builder::from_env()); fail-fast validation for partial key sets with value redaction; expanded OURIOS_S3_* redaction across StoreError, logs, metrics, and Debug; acceptance criterion RFC0019.8; and a localstack integration test gated to the S3 CI job.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Possibly related PRs

  • jensholdgaard/ourios#287: Directly precedes this amendment on the same RFC0019 storage-backend and S3 credential-chain specification.

Poem

🐇 Hoppity hop through the credential chain,
OURIOS_S3_* vars join the domain!
Two layers resolve, fail-fast on a pair,
Secrets redacted with utmost care.
The RFC grows, the rabbit's pleased—
No keys shall leak, the logs are clean! 🔐

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title is concise and accurately reflects the RFC change to explicit OURIOS_S3_* credential env keys.
Description check ✅ Passed The description covers the summary, related context, design, and verification, but it uses custom headings instead of the template's exact sections.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch rfc0019-credential-config-spec

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR amends RFC 0019’s storage-backend selection spec to allow S3 credentials to be provided via Ourios-named OURIOS_S3_* environment variables (while retaining the standard AWS credential chain as a fallback), and updates the RFC’s acceptance criteria/testing strategy accordingly.

Changes:

  • Specify a two-layer S3 credential resolution model: explicit OURIOS_S3_* credentials preferred, credential-chain fallback retained (e.g., for IRSA).
  • Extend the RFC’s config surface documentation to include OURIOS_S3_ACCESS_KEY_ID, OURIOS_S3_SECRET_ACCESS_KEY, and OURIOS_S3_SESSION_TOKEN.
  • Add a new scenario (RFC0019.8) covering explicit credentials and widened secret-redaction expectations.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread docs/rfcs/0019-storage-backend-selection.md Outdated
Comment thread docs/rfcs/0019-storage-backend-selection.md Outdated
…-cred fail-fast

Two review points on the spec:

1. status consistency (docs/rfcs/README.md: green = all §5 criteria pass) — the
   amendment added RFC0019.8 to §5 while the RFC stays green. Resolved by keeping
   the green body intact and moving the entire change into a dated §9 amendment
   (specified, pending impl); the implementation PR promotes RFC0019.8 into §5
   and folds the env vars/design into §3.1/§3.4, keeping green honest.

2. partial/invalid credential combinations were unspecified. §9.3 now states a
   fail-fast rule: access key id and secret are a pair (set one → both required);
   a session token without that pair fails too; the error names only the key,
   never a value (§3.4/RFC0019.6). Otherwise fall back to the credential chain.

mdbook build verified.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
docs/rfcs/0019-storage-backend-selection.md (1)

326-332: 🧹 Nitpick | 🔵 Trivial

Clarify "all unset" to avoid ambiguity with the optional session token.

The phrase "When the explicit keys are all unset" could be read as requiring all three OURIOS_S3_* variables to be unset before falling back to the credential chain. Since OURIOS_S3_SESSION_TOKEN is optional and meaningless without the static pair, consider rephrasing to "When the explicit static credential pair is unset" or similar to make clear only the access key / secret key pair gates the fallback.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/rfcs/0019-storage-backend-selection.md` around lines 326 - 332, The
fallback wording is ambiguous because “all unset” could imply the optional
session token must also be absent before using the standard credential chain.
Update the RFC text in the storage backend selection section to say that the
fallback applies when the explicit static credential pair is unset, and make
clear that OURIOS_S3_SESSION_TOKEN is optional and does not by itself gate
AmazonS3Builder::from_env() fallback behavior.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@docs/rfcs/0019-storage-backend-selection.md`:
- Around line 326-332: The fallback wording is ambiguous because “all unset”
could imply the optional session token must also be absent before using the
standard credential chain. Update the RFC text in the storage backend selection
section to say that the fallback applies when the explicit static credential
pair is unset, and make clear that OURIOS_S3_SESSION_TOKEN is optional and does
not by itself gate AmazonS3Builder::from_env() fallback behavior.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 63508733-3a09-45ee-a6a1-d99d9f576b60

📥 Commits

Reviewing files that changed from the base of the PR and between e15d264 and 2bedf75.

📒 Files selected for processing (1)
  • docs/rfcs/0019-storage-backend-selection.md

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 3 comments.

Comment thread docs/rfcs/0019-storage-backend-selection.md Outdated
Comment thread docs/rfcs/0019-storage-backend-selection.md Outdated
Comment thread docs/rfcs/0019-storage-backend-selection.md Outdated
…qualify from_env

- §3.4 note: name the OURIOS_S3_ACCESS_KEY_ID/SECRET_ACCESS_KEY/SESSION_TOKEN
  credential keys explicitly, distinct from the non-secret OURIOS_S3_* addressing
  keys (the bare prefix was ambiguous).
- §9.2 table: OURIOS_S3_SESSION_TOKEN default now states it is valid only with
  the static key pair (§9.3), not independently supplied.
- §9.5 RFC0019.8: fully-qualify AmazonS3Builder::from_env().

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 2 comments.

Comment thread docs/rfcs/0019-storage-backend-selection.md Outdated
Comment thread docs/rfcs/0019-storage-backend-selection.md Outdated
…een status

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 3 comments.

Comment thread docs/rfcs/0019-storage-backend-selection.md Outdated
Comment thread docs/rfcs/0019-storage-backend-selection.md Outdated
Comment thread docs/rfcs/0019-storage-backend-selection.md
…_* shorthand

Copilot flagged every OURIOS_S3_* in §9 as ambiguous with the non-secret
addressing keys. Define the term "S3 credential keys" once in §9.2 and use it in
§9.4/§9.5/§9.6 instead of the prefix shorthand.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.

@jensholdgaard
jensholdgaard merged commit 113cbb1 into main Jun 28, 2026
21 checks passed
@jensholdgaard
jensholdgaard deleted the rfc0019-credential-config-spec branch June 28, 2026 20:16
jensholdgaard added a commit that referenced this pull request Jul 10, 2026
* docs(rfc): accept RFC 0019 — storage-backend selection (maintainer sign-off)

Promote RFC 0019 green -> accepted (terminal). All eight §5 criteria
pass (green since #301, amended #306/#307): backend selection +
credential scrub unit tests, and the localstack S3 integration covering
WAL-stays-local, ingest→query on S3, the compactor manifest swap, and
cross-tenant isolation. No validated stage applies — it's server wiring,
not a thesis gate — so acceptance follows green directly (RFC 0001/0008
precedent). §9 records the sign-off.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X467Druw4cQeEPVp52DTG3

* docs(rfc): sync §6 stage line with accepted status (RFC 0019)

Copilot review on #455: §6 still said "the RFC is `green`" after the
status flipped to `accepted` — internal inconsistency. Now "accepted (§9)".

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X467Druw4cQeEPVp52DTG3

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants