Skip to content

feat(providers): AWS STS AssumeRole refresh strategy and aws-s3 profile - #1782

Merged
pimlock merged 1 commit into
NVIDIA:mainfrom
russellb:feat/1576-aws-sts-with-sigv4
Jul 16, 2026
Merged

feat(providers): AWS STS AssumeRole refresh strategy and aws-s3 profile#1782
pimlock merged 1 commit into
NVIDIA:mainfrom
russellb:feat/1576-aws-sts-with-sigv4

Conversation

@russellb

@russellb russellb commented Jun 5, 2026

Copy link
Copy Markdown
Contributor

Summary

Add gateway-owned AWS STS AssumeRole as a credential refresh strategy, ship aws and aws-s3 provider profiles, and extend policy validation to support single-label * wildcards in middle DNS labels.

Combined with the SigV4 proxy re-signing from #1638 (now merged), sandboxes can access S3 and other AWS services without ever seeing real credentials — the gateway mints short-lived STS credentials and the proxy re-signs requests on the fly.

Related Issue

Refs #1576

Changes

STS credential refresh (openshell-server)

  • New AwsStsAssumeRole variant in ProviderCredentialRefreshStrategy (proto field AWS_STS_ASSUME_ROLE = 6).
  • provider_refresh.rs implements the refresh loop: calls sts:AssumeRole using the gateway's ambient AWS credentials (or explicit long-lived keys from refresh material), stores the resulting AccessKeyId, SecretAccessKey, and SessionToken on the provider, and schedules re-rotation before expiry.
  • ConfigureProviderRefresh and RotateProviderCredential RPCs gated behind the providers_v2_enabled setting.
  • Validates that configuring STS refresh for one provider won't collide with credential keys already held by another provider attached to the same sandbox.

Provider profiles (openshell-providers)

AWS provider profiles follow the same pattern as the existing aws-bedrock profile: a generic base plus service-specific variants.

  • providers/aws.yaml — base AWS profile with STS refresh material (role_arn, session_name, external_id, aws_region, optional long-lived keys) but no endpoints or binaries allowlist. Intended for AWS services that don't yet have a dedicated profile — the user attaches their own policy to supply endpoints. Same role google-cloud plays relative to google-vertex-ai.
  • providers/aws-s3.yaml — S3-specific profile that adds pre-configured endpoints covering regional (*.s3.*.amazonaws.com), global (*.s3.amazonaws.com with signing_region: us-east-1), and dualstack variants. All endpoints use credential_signing: sigv4 and signing_service: s3. Includes a binaries allowlist for Python, curl, and the AWS CLI.

Policy: single-label * wildcard in middle DNS labels (openshell-policy)

  • validate_host_wildcard now permits * as a complete middle label (e.g. *.s3.*.amazonaws.com) in addition to leading-label and intra-label positions. Matches exactly one DNS label via Rego's glob.match with . separator, consistent with TLS wildcard semantics.
  • OPA endpoint matching and L7 config lookup updated: host_matches_wildcard_middle rule and endpoint_config_for_middle_wildcard in Rego.
  • Validation rejects ** in any position and partial wildcards in middle labels.

TUI (openshell-tui)

  • Added AwsStsAssumeRole match arm to the refresh strategy label in the provider settings view.

Documentation

  • docs/sandboxes/manage-providers.mdx: STS refresh setup instructions
  • examples/aws-s3-sts.md: end-to-end manual test guide (create IAM role, configure provider, test S3 PUT/GET/LIST from sandbox)

Testing

  • cargo test -p openshell-providers — 65 passed (STS serde roundtrip, aws/aws-s3 profile parsing, endpoint validation)
  • cargo test -p openshell-policy — 91 passed (wildcard DNS validation, middle-label wildcards, credential_signing validation)
  • cargo test -p openshell-server — all passed (STS configure v2 gate, success path, credential key collision)
  • cargo test -p openshell-supervisor-network — all passed (wildcard host matching, OPA endpoint config)
  • cargo test -p openshell-tui — 24 passed
  • End-to-end manual test: Podman sandbox → boto3 → S3 PUT/GET/LIST with STS-minted credentials and proxy-side SigV4 re-signing against real AWS (all passed)

Checklist

  • Follows Conventional Commits
  • Commits are signed off (DCO)

@copy-pr-bot

copy-pr-bot Bot commented Jun 5, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

Comment thread providers/aws-s3.yaml Outdated
Comment thread providers/aws-s3.yaml
@github-actions

Copy link
Copy Markdown

This pull request has had no activity for 14 days and is now marked stale. It may be closed in 7 days if there is no further activity.

@github-actions github-actions Bot added the state:stale Inactive item at risk of automatic closure. label Jun 23, 2026
@russellb

Copy link
Copy Markdown
Contributor Author

This is still waiting on #1638 to go in first.

@github-actions github-actions Bot removed the state:stale Inactive item at risk of automatic closure. label Jun 24, 2026
@russellb
russellb force-pushed the feat/1576-aws-sts-with-sigv4 branch 2 times, most recently from 1b1cbd3 to cb10f83 Compare June 30, 2026 18:22
@russellb
russellb marked this pull request as ready for review June 30, 2026 18:22
@russellb

russellb commented Jul 6, 2026

Copy link
Copy Markdown
Contributor Author

@pimlock willing to take a look at this one since you reviewed #1638 ?

@pimlock

pimlock commented Jul 7, 2026

Copy link
Copy Markdown
Collaborator

/ok to test cb10f83

@pimlock pimlock left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @russellb, I did a first pass, but I run out of time to get through this fully. I needed to check this out locally and run this to understand how it worked (wasn't as familiar with the cred refresh).

I added a few comments.
The biggest item I had was around how the implicit extra env vars that are managed by the refresh, but not defined as such in the provider profile.
I did investigation around this and options. I would need to spend more time figuring out what makes sense, but I will need to come back to it on Thursday. In the meantime, I can share raw notes from agent, but I haven't checked these very deeply yet: https://gist.github.com/pimlock/6a99b3d9c4e19c6efc3b0d2da59b6b63

Comment thread docs/sandboxes/manage-providers.mdx Outdated
Comment thread crates/openshell-server/src/provider_refresh.rs Outdated
Comment thread providers/aws-s3.yaml
@pimlock pimlock self-assigned this Jul 7, 2026
@pimlock pimlock added area:gateway Gateway server and control-plane work area:providers labels Jul 7, 2026
@russellb

russellb commented Jul 8, 2026

Copy link
Copy Markdown
Contributor Author

@pimlock thank you very much for the review! I know it can be very time-consuming. I will dig into the comments tomorrow.

@pimlock

pimlock commented Jul 8, 2026

Copy link
Copy Markdown
Collaborator

/ok to test cbe9b43

@pimlock

pimlock commented Jul 14, 2026

Copy link
Copy Markdown
Collaborator

@russellb Thanks for addressing the feedback! I'm running a bit behind, so I'll do a final review tomorrow and follow up.

@russellb

Copy link
Copy Markdown
Contributor Author

@russellb Thanks for addressing the feedback! I'm running a bit behind, so I'll do a final review tomorrow and follow up.

No problem!

@russellb
russellb force-pushed the feat/1576-aws-sts-with-sigv4 branch from cbe9b43 to 2c6dfb8 Compare July 14, 2026 15:22
@pimlock

pimlock commented Jul 14, 2026

Copy link
Copy Markdown
Collaborator

/ok to test 2c6dfb8

@pimlock pimlock added the test:e2e Requires end-to-end coverage label Jul 14, 2026
@github-actions

Copy link
Copy Markdown

Label test:e2e applied for 2c6dfb8. Open the existing run and click Re-run all jobs to execute with the label set. The run will execute the standard E2E suite after building the required gateway and supervisor images once. The matching required CI gate status on this PR will flip green automatically once the run finishes.

@pimlock

pimlock commented Jul 14, 2026

Copy link
Copy Markdown
Collaborator

gator-agent

PR Review Status

Validation: This PR is project-valid because it implements the linked provider-v2 feature request #1576 and completes the AWS credential path enabled by merged PR #1638.

Head SHA: 2c6dfb8022272983f8ff1223bcd94cd6f1b94f8e

Thanks @pimlock — I checked the runtime-credential resolution, duplicated mintability logic, declarative multi-output mapping, and collision-reservation concerns from your July 7 review. Thanks @russellb — the July 8 update does address those original concerns with the shared mintability check and pinned additional_outputs mapping. The independent code review found additional blockers:

  • Critical — CWE-918 (SSRF): crates/openshell-server/src/provider_refresh.rs:603-608 accepts undocumented sts_endpoint_url material and lets a caller direct an AWS-signed STS request to an arbitrary loopback HTTP service. Remove the production material override; inject test endpoints only through test-only code, and reject this material key at the configure boundary.
  • Blocking — CWE-20: crates/openshell-server/src/grpc/provider.rs:1500-1532,2103-2115 permits STS configuration without requiring a matching profile refresh declaration or canonical primary env key, while provider_refresh.rs:651-670 falls back to hardcoded sibling keys. Resolve and validate one canonical profile binding (strategy, AWS_ACCESS_KEY_ID, and both declared outputs), store the canonical primary key, and remove the fallback.
  • Blocking — CWE-20: provider_refresh.rs:586-598 silently ignores a partial explicit AWS source-credential pair and falls back to the gateway's ambient identity. Require aws_access_key_id and aws_secret_access_key to be both present or both absent.
  • Blocking: The v2 gate is enforced only during configuration (grpc/provider.rs:2019-2030); manual and worker rotations can keep minting STS credentials after providers_v2_enabled is disabled. Enforce the gate in the shared refresh execution path.
  • Blocking: Deleting an STS refresh clears only the primary expiry (grpc/provider.rs:2274-2300). Clear refresh-owned expiry metadata for the primary and every pinned additional output in one provider update, while preserving independently changed expiries.
  • Blocking docs/security — CWE-200: docs/sandboxes/manage-providers.mdx:209-215 expands the long-lived AWS secret into process arguments. Use --secret-material-env aws_secret_access_key=AWS_SECRET_ACCESS_KEY instead.

Please add regression coverage for the endpoint override rejection, partial source credentials, profile/strategy/canonical-key validation, v2 disable-after-config rotation, and additional-output expiry cleanup. No local tests or builds were run as part of this code-only review.

Docs: The feature is otherwise covered in existing Fern-managed pages and needs no navigation change, but the docs gate fails until the secret-handling example is corrected.

CI: test:e2e is applied and the required E2E gate is currently pending; author changes are required before pipeline status can advance the PR.

Next state: gator:in-review

@pimlock pimlock added the gator:in-review Gator is reviewing or awaiting PR review feedback label Jul 14, 2026
@russellb

russellb commented Jul 14, 2026

Copy link
Copy Markdown
Contributor Author

Addressed each finding in its own commit on top of 2c6dfb8. Summary below; commit SHAs referenced per item.

Critical — CWE-918 (SSRF): provider_refresh.rs:603-608 accepts undocumented sts_endpoint_url material and lets a caller direct an AWS-signed STS request to an arbitrary loopback HTTP service. Remove the production material override; inject test endpoints only through test-only code, and reject this material key at the configure boundary.

Fixed in e7a978c. The endpoint override is now a test_sts_endpoint_override helper that is #[cfg(test)]-only — the production build returns None and never reads material, so the STS endpoint is always region-resolved. handle_configure_provider_refresh rejects the sts_endpoint_url material key outright, so it can never reach a stored refresh state outside tests.

Blocking — CWE-20: grpc/provider.rs:1500-1532,2103-2115 permits STS configuration without requiring a matching profile refresh declaration or canonical primary env key, while provider_refresh.rs:651-670 falls back to hardcoded sibling keys. Resolve and validate one canonical profile binding (strategy, AWS_ACCESS_KEY_ID, and both declared outputs), store the canonical primary key, and remove the fallback.

Fixed in 916e9d2. Added validate_strategy_profile_binding, called at configure: for strategies that pin canonical env keys, it requires (1) a profile refresh declaration whose strategy matches, (2) the canonical primary env key (AWS_ACCESS_KEY_ID), and (3) every required strategy output resolved to its canonical env key from the profile's additional_outputs. The canonical primary key is what's stored as the refresh credential key. The mint fallback to hardcoded AWS_SECRET_ACCESS_KEY/AWS_SESSION_TOKEN is removed — a missing resolved mapping now fails closed instead of guessing.

Blocking — CWE-20: provider_refresh.rs:586-598 silently ignores a partial explicit AWS source-credential pair and falls back to the gateway's ambient identity. Require aws_access_key_id and aws_secret_access_key to be both present or both absent.

Fixed in 29970bb. The mint path now treats the pair as all-or-nothing and errors on a partial pair instead of falling through to the ambient identity; configure rejects the partial pair early with the same message.

Blocking: The v2 gate is enforced only during configuration (grpc/provider.rs:2019-2030); manual and worker rotations can keep minting STS credentials after providers_v2_enabled is disabled. Enforce the gate in the shared refresh execution path.

Fixed in a726dfc. The gate now runs in refresh_provider_credential — the shared path for both the worker sweep and manual rotation — so disabling providers_v2_enabled halts further mints. The rejection is recorded on the refresh state for observability.

Blocking: Deleting an STS refresh clears only the primary expiry (grpc/provider.rs:2274-2300). Clear refresh-owned expiry metadata for the primary and every pinned additional output in one provider update, while preserving independently changed expiries.

Fixed in 6799dcc. handle_delete_provider_refresh now clears the expiry for the primary and every pinned additional output whose stored expiry still matches the refresh's, in a single provider update. Expiries that were changed independently (no longer matching the refresh) are left untouched.

Blocking docs/security — CWE-200: docs/sandboxes/manage-providers.mdx:209-215 expands the long-lived AWS secret into process arguments. Use --secret-material-env aws_secret_access_key=AWS_SECRET_ACCESS_KEY instead.

Fixed in e8495fd. The off-AWS example now uses --secret-material-env aws_secret_access_key=AWS_SECRET_ACCESS_KEY so the CLI reads the secret from its environment instead of the process table.

Please add regression coverage for the endpoint override rejection, partial source credentials, profile/strategy/canonical-key validation, v2 disable-after-config rotation, and additional-output expiry cleanup.

Regression tests land with each corresponding fix rather than as a separate commit: endpoint-override rejection and partial-source-credential rejection (configure + mint), profile/strategy/canonical-key validation (no-profile and non-canonical-key cases), v2 disable-after-config rotation, and additional-output expiry cleanup (with an independently-changed expiry preserved).

Verification: openshell-server (912) and openshell-providers (79) unit tests pass; clippy clean with -D warnings. Also ran the documented AWS STS scenario manually against a live Podman-backed gateway (real CLI → real gateway) — 12/12 checks: the documented happy path configures and reaches the real STS call, and all five negative/gate cases behave as intended.

One note for the CI gate: the test:e2e lanes are Docker/Podman-backed and I couldn't run the automated suite in this environment (the Docker CLI here is a Podman compat shim whose docker info health check fails), so the required E2E gate still needs a CI run.

@pimlock

pimlock commented Jul 14, 2026

Copy link
Copy Markdown
Collaborator

gator-agent

Re-check After Author Update

Thanks @russellb. I reviewed current head e8495fdf0b81e49853fa6a4533095bacdec5f298 after your July 14 response explaining how each finding was addressed.

What I checked: the full current diff, the six fix commits, the STS configure/mint/delete paths, regression coverage, middle-label wildcard consumers, Fern docs, and the current CI state. The endpoint-override, canonical profile binding, partial source-credential, providers-v2 gate, and process-argument secret findings are resolved. Multi-key expiry cleanup is present, but its preservation guarantee is not concurrency-safe.

Disposition: partially resolved; author changes are still required.

Remaining items:

  • Blocking — CWE-362: crates/openshell-server/src/grpc/provider.rs:2358-2392 decides which expiries to clear from the provider snapshot fetched at line 2335, then update_provider_record fetches and merges into a potentially newer provider. A concurrent rotation or provider update can change an expiry between those operations, after which the unconditional zero removes the new value. Move the equality check and removal into the provider CAS closure, and add a regression test that changes an output expiry after the handler's initial read but before its write.
  • Warning — CWE-693: crates/openshell-prover/src/credentials.rs:182-208 still supports only first-label wildcard matching, while this PR adds runtime support for whole middle-label wildcards such as *.s3.*.amazonaws.com. Update the prover to mirror the runtime's label-aware semantics and cover matching plus non-matching label counts so it does not under-report credential-associated policy risk.
  • Docs/example: examples/aws-s3-sts.md:166 installs boto3 inside a sandbox whose aws-s3 profile does not allow PyPI; preinstall boto3 or attach the pypi profile. The cleanup should also delete s3-curl.

No local tests or builds were run as part of this code-only review.

Docs: the Fern-managed pages are updated and need no navigation change; the remaining example corrections are listed above.

CI: test:e2e remains appropriate. Required Branch Checks, Helm Lint, and E2E are waiting for the copy-PR mirror on this head, but author changes are required before pipeline monitoring should advance.

Next state: gator:in-review

@russellb

Copy link
Copy Markdown
Contributor Author

Addressed the three remaining items from the re-check, each in its own commit on top of e8495fd.

Blocking — CWE-362: crates/openshell-server/src/grpc/provider.rs:2358-2392 decides which expiries to clear from the provider snapshot fetched at line 2335, then update_provider_record fetches and merges into a potentially newer provider. A concurrent rotation or provider update can change an expiry between those operations, after which the unconditional zero removes the new value. Move the equality check and removal into the provider CAS closure, and add a regression test that changes an output expiry after the handler's initial read but before its write.

Fixed in 557221f. The equality check and removal now live in a pure helper (clear_refresh_owned_expiries) invoked inside the update_message_cas closure, so the decision runs against the CAS-locked current provider rather than the snapshot read earlier in the handler — a concurrently changed expiry is preserved instead of clobbered. Regression test clear_refresh_owned_expiries_preserves_concurrently_changed_expiry exercises the closure logic against a provider whose output expiry was changed after the refresh's value (as a concurrent rotation would leave it): the still-owned keys are cleared and the changed one is preserved. (A fully interleaved mid-update_message_cas write isn't reproducible without a test seam, so the regression targets the closure's decision function directly — that decision is now the only thing that clears anything.)

Warning — CWE-693: crates/openshell-prover/src/credentials.rs:182-208 still supports only first-label wildcard matching, while this PR adds runtime support for whole middle-label wildcards such as *.s3.*.amazonaws.com. Update the prover to mirror the runtime's label-aware semantics and cover matching plus non-matching label counts so it does not under-report credential-associated policy risk.

Fixed in 623512e. host_pattern_covers now matches label-for-label via a labels_match helper: a * (or a partial * glob) matches exactly one label in any position, literal labels compare literally, and a leading ** still matches one or more labels. Added host_patterns_overlap_matches_whole_middle_label_wildcard covering middle-label matches (*.s3.*.amazonaws.com, s3.*.amazonaws.com) and non-matching label counts (too few / too many labels, literal mismatch). Updated the prior test that asserted middle-label wildcards did not overlap, since that encoded the under-reporting behavior.

Docs/example: examples/aws-s3-sts.md:166 installs boto3 inside a sandbox whose aws-s3 profile does not allow PyPI; preinstall boto3 or attach the pypi profile. The cleanup should also delete s3-curl.

Fixed in 6c58d2a. The boto3 walkthrough now creates a pypi provider and attaches it (--provider s3-test --provider pypi) so pip can reach PyPI, with a note about the custom-image alternative. Cleanup now deletes the s3-curl sandbox and the pypi provider.

Verification: openshell-server (913) and openshell-prover (26) unit tests pass; clippy clean with -D warnings on both. The test:e2e gate still needs a CI run against this head.

@pimlock

pimlock commented Jul 14, 2026

Copy link
Copy Markdown
Collaborator

gator-agent

Re-check After Author Update

Thanks @russellb. I reviewed current head 623512efc2e50e133e63b9b96e4c4f28d8d97dd9 after your July 14 response explaining the CAS cleanup, prover wildcard, and AWS S3 example updates.

What I checked: the full current diff, the focused commits since e8495fdf, refresh configure/mint/delete concurrency, the middle-label wildcard implementation and tests, the Fern docs, the example cleanup, and the current CI state. The three items from the prior re-check are resolved: expiry ownership is now decided inside the provider CAS closure, prover matching covers exact-one-label middle wildcards, and the boto3 example now has PyPI egress with complete cleanup.

Disposition: partially resolved; one blocking correctness/security issue remains.

Remaining items:

  • Blocking — CWE-362: crates/openshell-server/src/provider_refresh.rs:300-308,341-386 reads refresh state, awaits the STS request, writes credentials, and then persists that stale state. Meanwhile, crates/openshell-server/src/grpc/provider.rs:2369-2405 can delete the refresh and return success. If deletion completes while STS is in flight, the rotation can subsequently mint credentials and recreate the deleted refresh, including stored source-credential material. Add a store-backed generation/lease or tombstone protocol shared by rotate/configure/delete so the minted result atomically verifies that its generation still exists before modifying the provider and cannot recreate deleted state. Please add a deterministic regression test that pauses mock STS, deletes the refresh, releases STS, and verifies that both refresh state and provider credentials remain unchanged.
  • Non-blocking warning — CWE-362: collision validation at crates/openshell-server/src/grpc/provider.rs:2167-2186 is separate from persistence at :2235-2266. Concurrent configuration of two providers attached to the same sandbox can let both reserve the same primary or additional key, leaving conflicting unusable refreshes even though later mint validation prevents injection. Please make reservation atomic or serialize configuration by affected sandbox/key and add a concurrent-configure regression test.

No local tests, builds, clippy, pre-commit, or E2E were run as part of this code-only review.

Docs: the user-facing behavior is covered by existing Fern pages under docs/; those pages are already in navigation, so no docs/index.yml change is needed.

CI: test:e2e remains appropriate. Required Branch Checks, Helm Lint, E2E, and DCO are pending on this new head; author changes are required before pipeline monitoring can advance.

Next state: gator:in-review

@russellb

Copy link
Copy Markdown
Contributor Author

Addressed both concurrency items from the latest re-check (head 623512e), each in its own commit.

Blocking — CWE-362: crates/openshell-server/src/provider_refresh.rs:300-308,341-386 reads refresh state, awaits the STS request, writes credentials, and then persists that stale state. Meanwhile, crates/openshell-server/src/grpc/provider.rs:2369-2405 can delete the refresh and return success. If deletion completes while STS is in flight, the rotation can subsequently mint credentials and recreate the deleted refresh, including stored source-credential material. Add a store-backed generation/lease or tombstone protocol shared by rotate/configure/delete so the minted result atomically verifies that its generation still exists before modifying the provider and cannot recreate deleted state. Please add a deterministic regression test that pauses mock STS, deletes the refresh, releases STS, and verifies that both refresh state and provider credentials remain unchanged.

Fixed in f462775. refresh_provider_credential now captures the refresh's generation (metadata.resource_version) at the start, and every terminal persist goes through persist_refresh_state_if_current, which writes with WriteCondition::MatchResourceVersion. That is an UPDATE-only write: it never inserts, so a refresh deleted (or superseded) while the mint request is in flight is detected and cannot be recreated, and its stored source-credential material is not resurrected. After minting, the rotation re-checks that the refresh still exists before writing credentials into the provider, so a removed refresh is not re-minted; on a detected deletion it returns aborted without touching the provider.

The generation acts as the store-backed check you described (a delete bumps the row out of existence, so the version-matched write fails closed). I did not introduce a separate lease/tombstone object — the existing per-row resource_version already gives the compare-and-swap needed to make the persist atomic with respect to delete, and keeping the mechanism in one primitive avoids a second source of truth to keep in sync across rotate/configure/delete. Happy to switch to an explicit tombstone if you'd prefer the stronger audit trail.

Deterministic regression test rotation_does_not_resurrect_refresh_deleted_mid_flight: a gated wiremock STS responder blocks until released; the test waits until the rotation is inside the STS call, deletes the refresh, releases STS, then asserts the rotation aborts, the refresh state is not recreated, and no AWS credentials were written to the provider.

Residual window I want to be transparent about: a delete that lands between the post-mint existence re-check and the provider credential write can still leave the freshly minted credential values in the provider. That is consistent with delete semantics — handle_delete_provider_refresh clears refresh-owned expiries but intentionally does not purge credential values — and the security-critical invariant (never recreate the deleted refresh state / its source material) holds unconditionally via the version-matched persist.

Non-blocking warning — CWE-362: collision validation at crates/openshell-server/src/grpc/provider.rs:2167-2186 is separate from persistence at :2235-2266. Concurrent configuration of two providers attached to the same sandbox can let both reserve the same primary or additional key, leaving conflicting unusable refreshes even though later mint validation prevents injection. Please make reservation atomic or serialize configuration by affected sandbox/key and add a concurrent-configure regression test.

Fixed in bf68e0b. handle_configure_provider_refresh now holds the process-wide sandbox sync guard across the validate-and-persist sequence — the same guard sandbox create/attach and profile changes already take. Because the collision validation consults each attached provider's active env keys (which include configured-but-unminted refresh reservations), serializing configuration means the second configure observes the first's persisted reservation and is rejected instead of double-reserving. Regression test concurrent_configure_aws_sts_reserves_key_exactly_once runs two configures for providers on the same sandbox concurrently and asserts exactly one succeeds and the other fails with a collision.

Scope note: this guard serializes within a gateway process, matching the existing concurrency model for sandbox-affecting mutations. It is not cross-replica; a store-backed reservation would be needed for multi-writer HA, which would be a broader change than this profile-scoped PR.

Verification: openshell-server (915) and openshell-prover (26) unit tests pass; clippy clean with -D warnings. The test:e2e gate still needs a CI run against this head.

@pimlock

pimlock commented Jul 14, 2026

Copy link
Copy Markdown
Collaborator

gator-agent

Re-check After Author Update

Thanks @russellb. I reviewed current head bf68e0b29482ff983b162138ebe0c9a0a35403ac after your July 14 response describing the resource-version guard for in-flight refresh deletion and the sandbox sync guard for concurrent key reservation.

What I checked: the full current diff, the two commits since 623512ef, refresh configure/mint/delete ordering, the shared-store HA contract, regression coverage, AWS source credential handling, Fern docs, and current CI state. The update prevents a deleted refresh row from being reinserted and serializes same-process configures, but it does not make the provider credential write atomic with refresh-generation ownership or serialize reservations across gateway replicas.

Disposition: partially resolved; author changes are still required.

Remaining items:

  • Blocking — CWE-362: crates/openshell-server/src/provider_refresh.rs:383-458 checks only that a refresh with the same name exists, writes minted values to the provider, and only afterward performs the version-matched refresh-state write. A refresh deleted and recreated during mint, or a losing concurrent rotation, can therefore write credentials minted from a stale generation before returning Aborted. Make refresh-generation ownership and the provider credential update one atomic datastore operation: require the refresh row to retain expected_version, and if that condition loses, leave both the refresh state and provider unchanged. Add deterministic delete/reconfigure-between-check-and-write and concurrent-losing-rotation tests.
  • Blocking — CWE-362/CWE-269: crates/openshell-server/src/grpc/provider.rs:2161-2274 holds sandbox_sync_guard() across validation and persistence, but that guard is process-local. OpenShell explicitly supports multiple gateway replicas sharing Postgres, so two replicas can still reserve the same sandbox credential keys. Use a shared-store lock/transaction or durable atomic reservations keyed by sandbox and env key across configure, sandbox attach/create, provider/profile mutation, and release. Cover this with two independently guarded server states sharing one store.
  • Warning: provider_refresh.rs:704-719 accepts explicit access/secret keys but always passes no AWS session token. Temporary source credentials from SSO or a prior AssumeRole will fail. Add optional secret aws_session_token, validate it requires the access/secret pair, pass it to the AWS SDK, and document it.
  • Warning: provider_refresh.rs:73-79 classifies a missing conditional-write target by matching database error text for "not found". Replace this backend-dependent string match with a typed condition-not-met/not-found persistence result.

No local tests, builds, clippy, pre-commit, or E2E were run as part of this code-only review.

Docs: the user-facing behavior is covered in existing Fern pages; no docs/index.yml navigation change is needed.

CI: DCO is green. Required Branch Checks, Helm Lint, and E2E are waiting for the copy-PR mirror on this head. test:e2e remains appropriate, but review changes are required before pipeline monitoring can advance.

Next state: gator:in-review

@pimlock

pimlock commented Jul 14, 2026

Copy link
Copy Markdown
Collaborator

gator-agent

PR Review Status

Validation: This PR is project-valid because it implements the linked provider-v2 request #1576 and completes the AWS credential path enabled by merged PR #1638.

Head SHA: b0365a117f3872c17d7b9f967c1977b7d1d63981

Thanks @russellb. I checked the three commits since the prior bf68e0b2 disposition, including typed absent-row conflicts, generation-gated credential writes, and temporary AWS source-session-token support. The typed conflict and session-token changes resolve those prior warnings, but the independent code review found two concurrency blockers:

  • Blocking — CWE-362: crates/openshell-server/src/provider_refresh.rs:419-435 claims the refresh generation and writes provider credentials in two separate datastore operations. A delete or reconfiguration can commit between them, allowing the old generation to write credentials; a crash in that window leaves refresh state marked successful while provider credentials remain stale. Commit the refresh-version check, refresh-state update, and provider credential update in one SQLite/Postgres transaction, and add deterministic delete/recreate and reconfigure tests for the post-verification window.
  • Blocking — CWE-362/CWE-269: credential-key reservation still relies on the process-local sandbox_sync_guard() (crates/openshell-server/src/grpc/provider.rs:2178; crates/openshell-server/src/compute/mod.rs:332) and ordinary provider updates at grpc/provider.rs:1959-1976 do not take it. Multiple gateway replicas sharing Postgres, or a provider update racing configuration, can therefore reserve colliding keys. Use durable reservation rows or datastore-scoped locking/transactions across every relevant mutation, then test two independently guarded server states sharing Postgres plus provider-update-versus-configure.

Non-blocking follow-ups:

  • docs/sandboxes/providers-v2.mdx:314 should list optional aws_session_token and its complete-pair requirement.
  • Add direct SQLite and Postgres coverage for Conflict { current_resource_version: None } on an absent-row conditional write.

No local tests, builds, clippy, pre-commit, or E2E were run as part of this code-only review.

Docs: the direct UX is covered in existing Fern pages and needs no navigation change, subject to the reference correction above.

CI: test:e2e remains appropriate. Branch Checks, Helm Lint, and E2E are currently waiting for the new-head mirror, but author changes are required before pipeline monitoring can advance.

Next state: gator:in-review

@russellb

Copy link
Copy Markdown
Contributor Author

Addressed the two warnings and the first blocking item (head b0365a1). On the second blocking item I'm pushing back — reasoning below.

Blocking — CWE-362: crates/openshell-server/src/provider_refresh.rs:383-458 checks only that a refresh with the same name exists, writes minted values to the provider, and only afterward performs the version-matched refresh-state write. A refresh deleted and recreated during mint, or a losing concurrent rotation, can therefore write credentials minted from a stale generation before returning Aborted. Make refresh-generation ownership and the provider credential update one atomic datastore operation: require the refresh row to retain expected_version, and if that condition loses, leave both the refresh state and provider unchanged. Add deterministic delete/reconfigure-between-check-and-write and concurrent-losing-rotation tests.

Fixed in a5a89f0. The rotation now claims the generation before touching the provider: it performs the version-matched refresh-state write (put_if with MatchResourceVersion(expected_version)) first, and only writes credentials into the provider if that claim succeeds. On a lost claim — deleted, recreated, or superseded by a concurrent rotation — it returns Aborted and leaves both the refresh state and the provider unchanged, so no stale-generation credentials are written and a deleted refresh is never resurrected.

On literal single-operation atomicity across the two objects: the store exposes only per-object CAS — there is no multi-object transaction API — so a single atomic datastore op spanning the refresh row and the provider row isn't available. The claim-then-write ordering is the optimistic-concurrency equivalent: generation ownership is the gate, and the loser mutates nothing. (I also switched the persist to consume a typed conditional-write result rather than matching error strings — see the CWE-693 item below.)

Deterministic tests: rotation_superseded_mid_flight_discards_credentials (a concurrent write bumps the generation while mock STS is paused; the losing rotation aborts, writes no credentials, and leaves the winner's state intact) and the existing rotation_does_not_resurrect_refresh_deleted_mid_flight (delete while STS is paused; no resurrection, no credentials written).

Blocking — CWE-362/CWE-269: crates/openshell-server/src/grpc/provider.rs:2161-2274 holds sandbox_sync_guard() across validation and persistence, but that guard is process-local. OpenShell explicitly supports multiple gateway replicas sharing Postgres, so two replicas can still reserve the same sandbox credential keys...

You're right that the gateway is active-active (I checked: replicaCount > 1 on shared Postgres, a plain Deployment, no leader election; even single-replica runs briefly overlap two pods during a RollingUpdate), so the cross-replica race is real. I'm pushing back on treating it as a blocker for this PR, for two reasons:

  1. It's pre-existing and subsystem-wide, not introduced here. sandbox_sync_guard() is the established concurrency model for every sandbox-mutating operation — create_sandbox, provider attach/detach, and provider-profile create/update/delete all rely on the same process-local guard (e.g. grpc/sandbox.rs:140,322,422, grpc/policy.rs:1863,2087, grpc/provider.rs:1264,1347,1447). Refresh configuration just follows that pattern. None of these are cross-replica safe today.
  2. A correct fix is cross-cutting and disproportionate to bolt onto an STS-refresh PR: durable store-backed reservations (or Postgres advisory locks) would have to span configure and attach/create and profile mutation and release. Adding it to configure alone wouldn't actually close the hole, since a concurrent attach on another replica re-creates the collision.

The guard added in this PR is still a strict improvement — it closes the same-process race (the common case, including single-replica rollouts). I'd propose the cross-replica reservation be tracked as its own follow-up rather than gating this feature. Happy to file that issue / open an RFC if you agree with the scoping.

Warning: provider_refresh.rs:704-719 accepts explicit access/secret keys but always passes no AWS session token...

Fixed in b0365a1. The explicit source-credential path now accepts an optional secret aws_session_token and passes it to the AWS SDK, so temporary credentials from SSO or a prior AssumeRole work. It requires the access/secret pair, enforced at both the configure boundary and the mint path. Declared as optional secret material on the aws and aws-s3 profiles and documented in the off-AWS example.

Warning: provider_refresh.rs:73-79 classifies a missing conditional-write target by matching database error text for "not found". Replace this backend-dependent string match with a typed condition-not-met/not-found persistence result.

Fixed in 31c339e. put_if with MatchResourceVersion now reports an absent row as a typed Conflict { current_resource_version: None } (a version mismatch carries the current version), in both the SQLite and Postgres backends. persist_refresh_state_if_current matches the typed Conflict and no longer inspects error strings.

Verification: openshell-server (919) and openshell-prover (26) unit tests pass; clippy clean with -D warnings. The test:e2e gate still needs a CI run against this head.

@pimlock

pimlock commented Jul 15, 2026

Copy link
Copy Markdown
Collaborator

/ok to test b0365a1

@pimlock pimlock added gator:blocked Gator is blocked by process or repository gates and removed gator:in-review Gator is reviewing or awaiting PR review feedback gator:blocked Gator is blocked by process or repository gates labels Jul 15, 2026
@pimlock

pimlock commented Jul 15, 2026

Copy link
Copy Markdown
Collaborator

Hi @russellb, thanks for addressing all the comments. I think this is ready

For the context, the gator-agent that I used to help reviewing this is now part of the repo: https://github.com/NVIDIA/OpenShell/blob/main/.agents/skills/launch-openshell-gator/SKILL.md

I agree on postponing this item:

Blocking — CWE-362/CWE-269: crates/openshell-server/src/grpc/provider.rs:2161-2274 holds sandbox_sync_guard() across validation and persistence, but that guard is process-local. OpenShell explicitly supports multiple gateway replicas sharing Postgres, so two replicas can still reserve the same sandbox credential keys...

IMO this is now ready, if you could please resolve conflicts, I will approve and we can get this merged.

Add gateway-managed AWS STS credential refresh (provider-v2, NVIDIA#1576). The
gateway calls sts:AssumeRole and writes three short-lived credentials
(AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_SESSION_TOKEN) to the
provider record; the proxy re-signs requests with SigV4. Adds the aws and
aws-s3 provider profiles and a declarative multi-output refresh model
(additional_outputs) so one AssumeRole co-mints all three credentials.

Hardening (from review):
- Reject caller-supplied STS endpoint overrides; the endpoint is always
  region-resolved in production, test-only otherwise (CWE-918).
- Validate a single canonical STS profile binding at configure
  (matching strategy, canonical AWS_ACCESS_KEY_ID primary, both declared
  outputs); remove the mint-path fallback to hardcoded sibling keys.
- Require aws_access_key_id/aws_secret_access_key as a complete pair, with
  optional aws_session_token for temporary source credentials.
- Enforce the providers_v2 gate on every mint (worker and manual rotate),
  not just at configure time.
- Clear refresh-owned expiries for the primary and every pinned additional
  output inside the provider CAS closure, preserving independent changes.
- Gate the provider credential write on refresh-generation ownership so a
  rotation cannot resurrect a deleted refresh or write stale-generation
  credentials (CWE-362).
- Serialize refresh configuration against sandbox mutations to reserve
  credential keys atomically within a gateway process.
- Return a typed Conflict for absent-row conditional writes instead of a
  backend-dependent error string.
- Mirror runtime whole-middle-label host wildcard semantics in the prover.

Signed-off-by: Russell Bryant <rbryant@redhat.com>
@russellb
russellb force-pushed the feat/1576-aws-sts-with-sigv4 branch from b0365a1 to e3ad817 Compare July 15, 2026 15:47
@russellb

Copy link
Copy Markdown
Contributor Author

Conflicts are resolved, but I'm running through the full example scenario to make sure it's good, as the resolved conflicts were not trivial edits.

@russellb

Copy link
Copy Markdown
Contributor Author

Conflicts are resolved, but I'm running through the full example scenario to make sure it's good, as the resolved conflicts were not trivial edits.

Testing complete. Still functional.

@pimlock

pimlock commented Jul 15, 2026

Copy link
Copy Markdown
Collaborator

/ok to test e3ad817

@russellb

Copy link
Copy Markdown
Contributor Author

@pimlock do you have more visibility into what failed in CI? or the ability to retrigger it if it's an environment thing?

@pimlock

pimlock commented Jul 16, 2026

Copy link
Copy Markdown
Collaborator

@pimlock do you have more visibility into what failed in CI? or the ability to retrigger it if it's an environment thing?

@russellb Looks like a flake, I will create an issue, saw it pop up on main as well. I restarted it and it will hopefully pass this time.

EDIT: issue for the flake: #2326

@pimlock
pimlock merged commit aa483ec into NVIDIA:main Jul 16, 2026
131 of 138 checks passed
rodbutters added a commit to iamaible/OpenShell that referenced this pull request Aug 24, 2026
* refactor(server): normalize compute driver config acquisition (#1974)

* refactor(server): remove unused compute runtime constructor parameter

Signed-off-by: Evan Lezar <elezar@nvidia.com>

* refactor(server): normalize compute driver type imports

Signed-off-by: Evan Lezar <elezar@nvidia.com>

* refactor(server): key driver config tables by name

Signed-off-by: Evan Lezar <elezar@nvidia.com>

* refactor(server): normalize compute driver config acquisition

Signed-off-by: Evan Lezar <elezar@nvidia.com>

---------

Signed-off-by: Evan Lezar <elezar@nvidia.com>

* test(e2e): run gpu workloads from manifest (#1709)

* test(e2e): add workload manifest build flow

Signed-off-by: Evan Lezar <elezar@nvidia.com>

* test(e2e): add gpu workload validation tests

Signed-off-by: Evan Lezar <elezar@nvidia.com>

* ci(e2e): build gpu workloads before gpu e2e

Signed-off-by: Evan Lezar <elezar@nvidia.com>

---------

Signed-off-by: Evan Lezar <elezar@nvidia.com>

* fix(providers): reserve credential placeholder revisions (#2049)

* fix(providers): reserve credential placeholder revisions

Signed-off-by: John Myers <johntmyers@users.noreply.github.com>

* fix(providers): share placeholder namespace parser

Signed-off-by: John Myers <johntmyers@users.noreply.github.com>

* test(providers): cover non-revision env key

Signed-off-by: John Myers <johntmyers@users.noreply.github.com>

---------

Signed-off-by: John Myers <johntmyers@users.noreply.github.com>
Co-authored-by: John Myers <johntmyers@users.noreply.github.com>

* fix(CONTRIBUTING): update label format for good first issues (#2056)

* fix(helm): generate namespace-aware SANs in certgen and cert-manager templates (#2062)

The certgen hook and cert-manager Certificate template hardcoded
openshell.openshell.svc.cluster.local in server certificate SANs,
breaking deployments in any namespace other than openshell. Use
.Release.Namespace in the templates so the SANs match the actual
service FQDN regardless of the target namespace.

Closes #2060

Signed-off-by: Akram <akram.benaissi@gmail.com>

* refactor(core): remove unused extra bind addresses (#2059)

Signed-off-by: Evan Lezar <elezar@nvidia.com>

* docs(mcp): fix granular policy lifecycle examples (#2066)

Signed-off-by: Shiju <shiju@nvidia.com>

* feat(kubernetes): add combined topology config surface (#2074)

* feat(kubernetes): add combined topology config surface

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* docs(kubernetes): clarify topology defaults

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

---------

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* fix(drivers): reject whitespace in mount fields (#2086)

Signed-off-by: Evan Lezar <elezar@nvidia.com>

* refactor(api): remove SandboxTemplate.volume_claim_templates (#2088)

The field was added during the Kubernetes driver extraction refactor
(#817) as a pass-through mechanism, but was never wired up to a CLI
flag, Python SDK helper, or any documentation. The only reachable
user path was raw gRPC construction.

The Kubernetes driver now always injects the default workspace PVC,
removing the branching logic that checked for a user-supplied VCT.
Field number 9 is reserved in the proto to prevent reuse.

Signed-off-by: Evan Lezar <elezar@nvidia.com>

* feat(helm): add TLS termination for Envoy Gateway ingress (#2015)

The chart's optional Gateway API ingress only rendered a plaintext HTTP
listener, so the gateway could not be exposed over TLS. Add an HTTPS
listener option that terminates TLS at the Envoy Gateway and forwards
plaintext gRPC to the gateway pod.

- gateway.yaml renders an HTTPS listener with `tls.mode: Terminate` and
  `certificateRefs` when `grpcRoute.gateway.listener.protocol=HTTPS`,
  keeping the default HTTP listener unchanged. Guards fail the render when
  `certificateRefs` is empty or `server.disableTls` is not true (the chart
  does not render a BackendTLSPolicy for re-encryption).
- values.yaml adds `grpcRoute.gateway.listener.tls.certificateRefs`.
- ci/values-gateway-tls.yaml exercises the HTTPS branch in lint/render.
- docs/kubernetes/ingress.mdx documents HTTPS setup and clarifies that
  Envoy Gateway only terminates TLS (no OIDC SecurityPolicy); client
  identity uses OIDC bearer tokens, with the client-credentials grant for
  headless agents.
- debug-openshell-cluster skill gains HTTPS-ingress troubleshooting rows.
- Regenerated the chart README values table.

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* feat(agents): add manifest-driven gator agent (#1826)

* chore(gator): add gator gate skill

* chore(gator): add sandbox launcher scaffold

* chore(gator): add codex image and docs checks

* chore(gator): fold approved provider policy rules

* chore(gator): add deterministic reviewer runner

* chore(gator): clarify ok-to-test comments

* chore(gator): structure launcher harnesses

* chore(gator): require e2e for dependabot

* chore(gator): add codex refresh profile

* chore(gator): wip manifest agent launcher

* feat(agents): supervise watch cycles in sandbox

* fix(agents): preserve gateway refresh state

* fix(gator): continue human response threads

* fix(agents): keep watch supervisor retrying

* fix(agents): use refreshed Codex credential aliases

* fix(gator): avoid misleading gh auth checks

* docs(agents): remove architecture build update

* fix(gator): use REST-backed GitHub writes

* fix(agents): bake immutable agent payloads

* fix(agents): upload writable agent workspace

* fix(agents): surface gator watch progress

* fix(agents): prevent codex stdin hang

* fix(agents): align codex subagent input

* fix(agents): heartbeat during active cycles

* fix(agents): clean up heartbeat sleep

* fix(agents): disable gh telemetry in codex harness

* fix(agents): reconcile closed gator PRs

* fix(agents): query closed gator PR labels separately

* fix(agents): tolerate rotated credential placeholders

* fix(agents): enforce gator same-sha comment guard

Signed-off-by: John Myers <johntmyers@users.noreply.github.com>

* docs(agents): scope gator trusted commentary

Signed-off-by: John Myers <johntmyers@users.noreply.github.com>

* fix(gator): treat reviewer failures as transient

Signed-off-by: Evan Lezar <elezar@nvidia.com>

* feat(agents): refine gator supervised workflow

Signed-off-by: John Myers <johntmyers@users.noreply.github.com>

* fix(agents): stream codex prompts via stdin

Signed-off-by: John Myers <johntmyers@users.noreply.github.com>

* docs(agents): clarify trusted gator responses

Signed-off-by: John Myers <johntmyers@users.noreply.github.com>

* refactor(agents): scope gator PR to scripts

Signed-off-by: John Myers <johntmyers@users.noreply.github.com>

---------

Signed-off-by: John Myers <johntmyers@users.noreply.github.com>
Signed-off-by: Evan Lezar <elezar@nvidia.com>
Co-authored-by: John Myers <johntmyers@users.noreply.github.com>
Co-authored-by: Evan Lezar <elezar@nvidia.com>

* feat(docker,podman): add SELinux label support for bind mounts (#2092)

* feat(docker,podman): add SELinux label support for bind mounts

The Docker Engine structured Mount API does not support SELinux
relabelling (:z / :Z). Move user-supplied bind mounts from the
structured `mounts` field to the legacy string-format `binds` field,
which does support these options.

Add a shared `SelinuxLabel` enum (shared/private) to openshell-core so
both Docker and Podman drivers accept an optional `selinux_label` field
on bind mount configs. For Docker, labels are appended to the bind
string; for Podman, they are pushed to the mount options vec.

Signed-off-by: Florian Bergmann <fbergman@redhat.com>

* fix(docker): reject missing bind source paths on legacy binds

Moving user bind mounts from the structured Mount API to the legacy
Binds field changed Docker's behavior for missing source directories:
the legacy path silently creates them as empty root-owned dirs instead
of erroring. Add an explicit Path::exists() check to preserve the
fail-fast behavior operators expect.

Signed-off-by: Florian Bergmann <fbergman@redhat.com>

---------

Signed-off-by: Florian Bergmann <fbergman@redhat.com>

* test(e2e): run rootless podman on ubuntu host (#2119)

* test(e2e): run rootless podman on ubuntu host

Signed-off-by: Evan Lezar <elezar@nvidia.com>

* test(e2e): probe rootless capability behavior

Signed-off-by: Evan Lezar <elezar@nvidia.com>

* test(e2e): make capability probe observational

Signed-off-by: Evan Lezar <elezar@nvidia.com>

---------

Signed-off-by: Evan Lezar <elezar@nvidia.com>

* feat(policy): accept numeric UIDs for sandbox process identity (#1973)

* feat(policy): accept numeric UIDs in sandbox process identity validation

Allow run_as_user and run_as_group to be either the literal 'sandbox'
or a numeric UID/GID within [1000, 2_000_000_000]. This removes the
hard dependency on a baked-in 'sandbox' user in container images,
enabling compute drivers to inject resolved UIDs at sandbox creation.

Phase 1 of #1959.

Signed-off-by: Seth Jennings <sjenning@redhat.com>

* feat(supervisor): accept numeric UIDs for process identity dropping

Allow run_as_user and run_as_group to be numeric UIDs/GIDs, removing
the hard dependency on a baked-in 'sandbox' user in container images.

Changes:
- validate_sandbox_user(): accepts numeric UIDs without passwd lookup
  (logs OCSF event); keeps passwd check for "sandbox" name; rejects
  non-numeric non-sandbox strings that fail passwd lookup
- prepare_filesystem(): passes numeric UIDs/GIDs directly to chown()
  instead of requiring a passwd entry
- drop_privileges(): resolves numeric UIDs/GIDs directly via UID::from_raw
  / Gid::from_raw; skips initgroups when target uid matches current euid;
  uses guard conditions before setgid/setuid calls
- session_user_and_home(): falls back to ("{uid}", "/sandbox") for
  numeric UIDs, avoiding a passwd lookup that will fail

Re-exports MIN_SANDBOX_UID and MAX_SANDBOX_UID from openshell-policy
so callers have consistent range constants.

Phase 2 of #1959.

Signed-off-by: Seth Jennings <sjenning@redhat.com>

* feat(driver-kubernetes): resolve sandbox UID/GID from config or OpenShift SCC annotations

Phase 3 of the numeric-UID plan: allow operators to specify explicit
sandbox_uid/sandbox_gid in Kubernetes driver config, auto-detect from
OpenShift SCC namespace annotations, and propagate resolved values to
supervisor container env vars and PVC init container securityContext.

Changes:
- Add sandbox_uid/sandbox_gid fields to KubernetesComputeConfig
- Add SANDBOX_UID/SANDBOX_GID env var constants to openshell-core
- Implement resolve_sandbox_identity() to fetch namespace annotations
  and auto-detect OpenShift SCC UID ranges (sa.scc.uid-range)
- Pass resolved UID/GID through SandboxPodParams to pod spec builder
- Inject SANDBOX_UID/SANDBOX_GID env vars into supervisor container
- Update PVC init container securityContext with resolved UID/GID
  instead of hard-coded root
- Add comprehensive unit tests for resolution logic and annotation
  parsing (resolve_sandbox_uid, resolve_sandbox_gid, OpenShift SCC
  annotation parsing)

Signed-off-by: Seth Jennings <sjenning@redhat.com>

* feat(driver-vm): add configurable sandbox UID/GID and update docs/examples

Phase 4 of the numeric-UID plan: replace hardcoded SANDBOX_UID (10001)
in VM rootfs preparation with configurable sandbox_uid/sandbox_gid fields.

Changes:
- Add sandbox_uid/sandbox_gid to VmDriverConfig with serde derives
- Pass resolved UID/GID through prepare_sandbox_rootfs_from_image_root
  to ensure_sandbox_guest_user which writes /etc/passwd/group/gshadow
- Update BYOC Dockerfile: remove groupadd/useradd, document runtime UID
  injection and the ability to skip baked-in sandbox user
- Update gateway-config.mdx: document sandbox_uid/sandbox_gid for both
  Kubernetes (with OpenShift SCC autodetection) and VM drivers
- Update sandbox-compute-drivers.mdx: add Sandbox User Identity section
  explaining numeric UID support across all compute drivers
- Update rootfs tests to use non-default UIDs, verify config passthrough

Signed-off-by: Seth Jennings <sjenning@redhat.com>

* code review changes

* fix(supervisor): harden tests for restricted CI container environments

Guard tests against CI-specific constraints: root without CAP_SETPCAP,
UIDs with no /etc/passwd entry, and restricted /proc access.

Signed-off-by: Seth Jennings <sjennings@nvidia.com>
Signed-off-by: Seth Jennings <sjenning@redhat.com>

---------

Signed-off-by: Seth Jennings <sjenning@redhat.com>
Signed-off-by: Seth Jennings <sjennings@nvidia.com>

* docs: add Hermes Agent to supported agents table (#2131)

* rfc-0006: add driver config passthrough proposal (#1589)

* docs(rfc): add driver config passthrough proposal

Signed-off-by: Evan Lezar <elezar@nvidia.com>

* docs(rfc): link driver config proposal PR

Signed-off-by: Evan Lezar <elezar@nvidia.com>

* docs(rfc): clarify driver config scope

Signed-off-by: Evan Lezar <elezar@nvidia.com>

* docs(rfc): clarify driver-local config schemas

Signed-off-by: Evan Lezar <elezar@nvidia.com>

* docs(rfc): clarify driver config extension path

* docs(rfc): update driver config baseline

* docs(drivers): document bind-mount selinux_label and whitespace rules

---------

Signed-off-by: Evan Lezar <elezar@nvidia.com>

* chore(deps): bump docker/login-action from 4.2.0 to 4.4.0 (#2146)

* docs: fix STYLEGUIDE heading to match filename (#2134)

* docs(kubernetes): bump cert-manager to v1.20.3 (#2129)

* docs: fix article before OpenShell in sync-files (#2133)

* docs: warn to redact credentials from log output before sharing (#2124)

Add a reminder to the bug report template's Logs field and a new row in
the security best-practices Common Mistakes table advising reporters to
redact credentials, API keys, and tokens from stack traces before pasting.

Signed-off-by: Evan Lezar <elezar@nvidia.com>

* fix(podman): deliver sandbox JWTs as secrets (#2156)

Signed-off-by: Adam Miller <admiller@redhat.com>

* chore: remove deprecated --keep flag from docs, scripts, and e2e tests (#2126)

* docs: remove deprecated --keep flag from tutorials and examples

The --keep flag is deprecated, hidden, and a no-op since sandboxes
are kept by default. Remove references from tutorial docs and example
READMEs that explain it as a real feature.

- Remove --keep from sandbox create commands
- Remove --keep explanation text
- Clarify that sandboxes are kept by default

Signed-off-by: Ignas Baranauskas <ibaranau@redhat.com>

* chore: remove deprecated --keep usage from scripts and e2e tests

The --keep flag is a deprecated no-op since sandboxes are kept by
default. Stop passing it in internal scripts, e2e test scripts,
and example demo scripts.

Signed-off-by: Ignas Baranauskas <ibaranau@redhat.com>

---------

Signed-off-by: Ignas Baranauskas <ibaranau@redhat.com>

* chore(deps): bump astral-sh/setup-uv from 8.2.0 to 8.3.0 (#2160)

Bumps [astral-sh/setup-uv](https://github.com/astral-sh/setup-uv) from 8.2.0 to 8.3.0.
- [Release notes](https://github.com/astral-sh/setup-uv/releases)
- [Commits](https://github.com/astral-sh/setup-uv/compare/fac544c07dec837d0ccb6301d7b5580bf5edae39...d31148d669074a8d0a63714ba94f3201e7020bc3)

---
updated-dependencies:
- dependency-name: astral-sh/setup-uv
  dependency-version: 8.3.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* fix(driver-podman): gate Linux-only Path import (#2188)

Apply the Linux cfg to the Path import so native macOS lint runs do not report
it as unused when the only call site is compiled out.

This fixes `mise run rust:lint` on macOS.

Signed-off-by: Kris Hicks <khicks@nvidia.com>

* docs: fix Docker version format from 28.04 to 28.0 (#2136)

* docs: update man page date to 2026 (#2135)

* fix(sandbox): acknowledge initial policy revision; expose SDK labels/selectors (#2170)

* fix(sandbox): acknowledge initial policy revision

The supervisor loaded and enforced a sandbox-scoped policy but never told
the gateway which revision it loaded. The policy poll loop seeded itself
with the initial revision's hash on its first poll, so `policy_changed`
was never true for that revision and `ReportPolicyStatus(LOADED)` — which
only ran in the hot-reload branch — was never called. The revision stayed
`Pending` and `current_policy_version` stayed 0 even though the sandbox was
`Ready` and the policy was effective. This was most visible with sparse
policies that get baseline-enriched into a new revision during startup.

After the OPA engine is constructed, report the exact sandbox revision the
supervisor loaded as LOADED, and seed the poll loop from that revision so
it is not re-reported. Report FAILED with the original construction error
if engine construction or conversion fails. Only sandbox-sourced revisions
(version > 0) whose canonical content matches the loaded policy are
acknowledged; global and local-file policies are untouched. Delivery uses
the shared bounded retry, is non-fatal on transient failure, and a pending
initial acknowledgement is delivered before any newer revision so policy
history is never reordered.

Signed-off-by: Kyle Zheng <kyzheng@nvidia.com>

* feat(python): expose sandbox labels and selectors

The gateway protobuf and CLI already support request-level sandbox labels
(`CreateSandboxRequest.name`/`labels`) and selector-based listing
(`ListSandboxesRequest.label_selector`), but the public Python SDK dropped
them, so Python-created sandboxes could not be found via
`openshell sandbox list --selector ...`.

Add optional, source-compatible `name`/`labels` to `SandboxClient.create`,
`create_session`, and the high-level `Sandbox`, and `label_selector` to
`list`/`list_ids`. `SandboxRef` now carries the gateway labels as an
immutable mapping (default empty, so `SandboxRef(id, name, status)` still
works). Caller-provided label mappings are copied. Attaching the high-level
`Sandbox` to an existing sandbox rejects `name`/`labels` since creation
metadata cannot change on attach. Template labels remain a separate concept.
No protobuf changes are required.

Signed-off-by: Kyle Zheng <kyzheng@nvidia.com>

* fix(python): keep SandboxRef hashable and copy high-level labels

Excluding the new immutable `labels` field from SandboxRef equality/hash
(`compare=False`) preserves the original (id, name, status) identity and keeps
the frozen dataclass hashable — a MappingProxyType field would otherwise make
`hash(SandboxRef(...))` raise. Also defensively copy caller-provided labels in
the high-level `Sandbox` so later caller mutation cannot change what is sent.

Signed-off-by: Kyle Zheng <kyzheng@nvidia.com>

* fix(sandbox): bound initial-policy-ack retries

The poll loop retried a pending initial acknowledgement before processing any
newer revision, but retried unconditionally forever. A permanently
undeliverable ack (e.g. the revision was superseded before it could be
reported) would then stall all later policy hot-reloads and provider-env
refreshes. Cap the retries; after the bound, give up and resume normal polling
so the loop cannot livelock on a stuck acknowledgement.

Signed-off-by: Kyle Zheng <kyzheng@nvidia.com>

* test(sandbox): add sparse-policy revision-2 acknowledgement e2e

Regression for #2159: create a sandbox with the network-only policy-advisor
fixture, which the supervisor enriches with baseline filesystem paths during
startup (creating revision 2, superseding revision 1). Assert the effective
policy reaches revision 2 and no revision remains Pending once the supervisor
acknowledges the load. Adds SandboxGuard::create_keep_with_args to create a
kept sandbox with an initial --policy.

Signed-off-by: Kyle Zheng <kyzheng@nvidia.com>

* fix(ci): correct sandbox checks

Signed-off-by: Kyle Zheng <kyzheng@nvidia.com>

* test(cli): serialize mTLS environment access

Signed-off-by: Kyle Zheng <kyzheng@nvidia.com>

* fix(sandbox): address policy review feedback

Signed-off-by: Kyle Zheng <kyzheng@nvidia.com>

* fix(sandbox): preserve exact policy acknowledgements

Signed-off-by: Kyle Zheng <kyzheng@nvidia.com>

* fix(sandbox): preserve local policy overrides

Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com>

---------

Signed-off-by: Kyle Zheng <kyzheng@nvidia.com>
Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com>
Co-authored-by: John Myers <9696606+johntmyers@users.noreply.github.com>

* chore(deps): bump astral-sh/setup-uv from 8.3.0 to 8.3.1 (#2191)

Bumps [astral-sh/setup-uv](https://github.com/astral-sh/setup-uv) from 8.3.0 to 8.3.1.
- [Release notes](https://github.com/astral-sh/setup-uv/releases)
- [Commits](https://github.com/astral-sh/setup-uv/compare/d31148d669074a8d0a63714ba94f3201e7020bc3...f98e06938123ccabd21905ea5d0069192241f9f1)

---
updated-dependencies:
- dependency-name: astral-sh/setup-uv
  dependency-version: 8.3.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* feat(cli): add --secret-material-env to provider refresh configure (#2178)

* feat(cli): add --secret-material-env to provider refresh configure

* feat(cli): reject duplicate secret material keys

Signed-off-by: Hung Le <hple@nvidia.com>

---------

Signed-off-by: Hung Le <hple@nvidia.com>

* docs(telemetry): Added first telemetry report for the community (#2190)

* docs(telemetry): add community telemetry reports page

Add telemetry/README.md to publish aggregate usage trends every two
weeks, and link to it from the Telemetry section of the main README.
First report covers the July 8, 2026 window.

Signed-off-by: Kirit Thadaka <kthadaka@nvidia.com>

* docs(telemetry): note telemetry start date (June 1, 2026)

Clarify that all-time figures are cumulative from #1433, so readers
know when the all-time counts begin.

Signed-off-by: Kirit Thadaka <kthadaka@nvidia.com>

---------

Signed-off-by: Kirit Thadaka <kthadaka@nvidia.com>

* change packit target to new correct copr project (#2185)

Signed-off-by: Adam Miller <admiller@redhat.com>

* test(supervisor-network): add proxy hostname parser regression tests (#2197)

Add regression coverage for parser differentials in the egress proxy's
CONNECT hostname handling and OPA wildcard policy matching.

Signed-off-by: Shane Utt <shaneutt@linux.com>

* fix(tui): route warning logs to status bar instead of stderr (#2210)

* fix(tui): route warning logs to status bar instead of stderr

tracing::warn/info/debug calls in the TUI crate write to stderr via
the global tracing subscriber. In ratatui's alternate-screen/raw-mode,
stderr writes corrupt the terminal layout. Error-state sandboxes
amplify this as background gRPC polls fail every 2s tick.

Replace all 25 tracing calls with app.status_text assignments for
direct-access sites, and Vec<String> accumulation for the spawned
start_port_forwards task. Add ForwardWarnings event variant to
decouple forward warning delivery from the sandbox name in
CreateResult, preventing downstream gRPC lookup failures.

Closes #2120

Signed-off-by: Ian Miller <milleryan2003@gmail.com>

* feat(tui): add ForwardWarnings event variant

New event type for non-fatal port-forward warnings during sandbox
creation. Keeps warning delivery separate from the sandbox name
in CreateResult to avoid corrupting downstream gRPC lookups.

Signed-off-by: Ian Miller <milleryan2003@gmail.com>

* chore(tui): remove tracing dependency

Compile-time guard against reintroducing stderr-writing tracing
calls in the TUI crate. 14 other crates retain the dependency.

Signed-off-by: Ian Miller <milleryan2003@gmail.com>

---------

Signed-off-by: Ian Miller <milleryan2003@gmail.com>

* fix(mcp): include tool names in policy logs (#2189)

Signed-off-by: Kirit93 <kthadaka@nvidia.com>

* chore(deps): bump astral-sh/setup-uv from 8.3.1 to 8.3.2 (#2206)

Bumps [astral-sh/setup-uv](https://github.com/astral-sh/setup-uv) from 8.3.1 to 8.3.2.
- [Release notes](https://github.com/astral-sh/setup-uv/releases)
- [Commits](https://github.com/astral-sh/setup-uv/compare/f98e06938123ccabd21905ea5d0069192241f9f1...11f9893b081a58869d3b5fccaea48c9e9e46f990)

---
updated-dependencies:
- dependency-name: astral-sh/setup-uv
  dependency-version: 8.3.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* docs(openshift): simplify install steps and add Helm README entries for OpenShift overrides (#2125)

Signed-off-by: ChristianZaccaria <christian.zaccaria.cz@gmail.com>

* docs(issues): require release and duplicate checks (#2214)

Signed-off-by: Evan Lezar <elezar@nvidia.com>

* fix(core): pin supervisor image tag to gateway version for all drivers (#2070)

* fix(core): pin supervisor image tag to gateway version for all drivers

The Podman and Kubernetes drivers defaulted the supervisor image to
`:latest` via DEFAULT_SUPERVISOR_IMAGE, while the Docker driver already
resolved a version-pinned tag. Extract the tag resolution logic into
openshell-core so all three drivers use the same
OPENSHELL_IMAGE_TAG > IMAGE_TAG > CARGO_PKG_VERSION priority chain.

Closes #2068

Signed-off-by: Florent Benoit <fbenoit@redhat.com>

* refactor(core): simplify supervisor image tag resolver to slice-based API

Remove the Docker driver's wrapper functions and call
openshell_core::config::default_supervisor_image() directly.
Simplify resolve_supervisor_image_tag to accept &[&str] instead
of three separate parameters.

Signed-off-by: Florent Benoit <fbenoit@nvidia.com>
Signed-off-by: Florent Benoit <fbenoit@redhat.com>

---------

Signed-off-by: Florent Benoit <fbenoit@redhat.com>
Signed-off-by: Florent Benoit <fbenoit@nvidia.com>

* fix(helm): propagate supervisor image overrides (#2216)

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* feat(kubernetes): add sidecar supervisor topology (#2076)

* feat(kubernetes): add sidecar supervisor topology

Add the Kubernetes sidecar supervisor topology, its Helm/Skaffold configuration, topology documentation, and sidecar e2e matrix coverage. Skip root-only sandbox identity rewriting when process enforcement is network-only so the low-permission sidecar process container can start successfully.

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* fix(supervisor): avoid similar process id names

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* fix(supervisor): avoid similar process id names

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* fix(sandbox): avoid similar proxy id names

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* docs(kubernetes): clarify sidecar topology limits

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* fix(kubernetes): keep sidecar process leaf capless

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* fix(kubernetes): refresh sidecar provider env snapshots

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* test(supervisor): align hot-swap identity regression

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* fix(kubernetes): stage sidecar mtls files before proxy chown

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* fix(kubernetes): simplify sidecar supervisor topology

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* chore(helm): reuse sidecar skaffold values

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* fix(supervisor): avoid similar iptables helper names

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* fix(e2e): harden kube gateway wrapper setup

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* fix(supervisor): avoid nft batch rollback on OCP

Run nftables setup as individual commands so optional conntrack and log expressions can fail without rolling back required table, chain, and reject rules.

Signed-off-by: Seth Jennings <sjenning@redhat.com>
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* fix(kubernetes): preserve process identity in sidecar topology

Render sidecar pods with a shared process namespace, keep binary-aware network policy enabled, and move Kubernetes sidecar settings under the nested sidecar config table.

Also apply unprivileged Landlock/seccomp setup in NetworkOnly supervisor mode so sidecar topology keeps sandbox child hardening without privileged process setup.

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* refactor(kubernetes): replace sidecar snapshots with control socket

Coordinate sidecar policy and provider bootstrap over a local Unix socket so the process leaf no longer reads policy/provider snapshot files.

Report entrypoint startup through the control channel and keep gateway credentials confined to the network sidecar.

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* feat(kubernetes): support relaxed sidecar network identity

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* fix(sandbox): satisfy sidecar clippy lint

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* refactor(kubernetes): standardize topology naming

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* fix(sandbox): satisfy linux clippy timeout import

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* fix(kubernetes): support kata sidecar on ipv4 pods

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* fix(kubernetes): satisfy linux clippy for sidecar fallback

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* chore(kubernetes): remove stale supervisor topology references

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* fix(kubernetes): enable sidecar binary policy inspection

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* fix(kubernetes): harden sidecar control boundary

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* fix(kubernetes): couple sidecar supervisor lifecycles

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

---------

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
Signed-off-by: Seth Jennings <sjenning@redhat.com>
Co-authored-by: Seth Jennings <sjenning@redhat.com>

* feat(kubernetes): support PVC subPath driver config (#2034)

* feat(kubernetes): support PVC subPath driver config

Signed-off-by: mjamiv <michael.commack@gmail.com>

* test(kubernetes): cover writable PVC driver config

Signed-off-by: mjamiv <michael.commack@gmail.com>

* fix(kubernetes): address PVC subPath review feedback

Signed-off-by: mjamiv <michael.commack@gmail.com>

* fix(kubernetes): address PVC config review follow-up

Signed-off-by: mjamiv <michael.commack@gmail.com>

* fix(kubernetes): address PVC review follow-ups

---------

Signed-off-by: mjamiv <michael.commack@gmail.com>

* fix(network): fail closed when credential placeholders cannot be rewritten (#2162)

* fix(network): fail closed when credential placeholders cannot be rewritten

When the credential rewriter degrades internally, the proxy forwarded the
literal `openshell:resolve:env:<NAME>` placeholder (or its provider alias
marker) to the upstream instead of the resolved secret, leaking the reserved
token on the wire and causing upstream auth failures (#2161).

Two fail-open paths are closed:

- secrets: `rewrite_http_header_block` returned the header block verbatim when
  no `SecretResolver` was available, so the fail-closed marker scan (which ran
  only on the resolved path) never saw the placeholder. It now scans the
  header region for reserved markers even with no resolver and returns
  `UnresolvedPlaceholderError` when one is present. Marker-free traffic still
  passes through unchanged.

- proxy: when TLS was detected on a CONNECT but `tls_state` was `None`
  (ephemeral CA generation or CA file write failed at startup), the handler
  fell back to a raw `copy_bidirectional` tunnel, bypassing credential
  rewrite. Inside the proxy handler `tls_state` is `None` only on CA-init
  failure (`mode != Proxy` never starts the handler, and `tls: skip` is
  handled earlier), so it now refuses the connection with a 503 and a
  High-severity denial event instead of tunneling. The two startup CA-failure
  logs are raised from Medium to High.

Tests: resolver=None with a placeholder in the request line, a header value,
and the provider-alias form now fail closed; marker-free passthrough is
unchanged; the relay integration test asserts the request is rejected before
any byte reaches upstream; and the 503 fail-closed response contract is
locked.

Signed-off-by: Tony Luo <xialuo@nvidia.com>

* fix(network): refuse CONNECT before 200 when TLS termination is unavailable

The fail-closed refusal for a terminating CONNECT with no TLS termination
state (ephemeral CA init failed) was written after the 200 Connection
Established response. Because a CONNECT client only sends its TLS
ClientHello after reading the 200, the peek-based TLS detection is
inherently post-200, so the 503 landed inside the established tunnel and
surfaced to the client as a TLS protocol error rather than a readable
status. An 'allowed CONNECT' event was also logged first.

Move the decision to a pre-200 gate: query_tls_mode resolves purely from
the policy decision + host/port (no peeked bytes), so the route's TLS
treatment is known before the tunnel is acknowledged. When TLS state is
absent and the route is not tls: skip, write the 503 as the first bytes
on the socket, emit the High-severity Denied event, and close. tls: skip
routes tunnel raw exactly as before, and no allowed-CONNECT event is
emitted on the refusal path.

The now-unreachable post-200 branch is kept as defense in depth but no
longer writes an in-tunnel 503; it fails closed by dropping the
connection instead.

Add connection-level regression tests over a real loopback socket: the
gate refuses with HTTP/1.1 503 as the first bytes for a terminating
route, and writes nothing when TLS termination is present or the route is
tls: skip.

Signed-off-by: Tony Luo <xialuo@nvidia.com>

* fix(network): order the CONNECT TLS-unavailable refusal after SSRF

Addresses the gator re-check on #2162.

Ordering: the pre-200 fail-closed refusal ran before SSRF/allowed_ips
validation, so during CA-init failure an internal-address CONNECT got a
503 tls_termination_unavailable instead of the normal 403 ssrf_denied,
weakening operator visibility in degraded state. The SSRF branches now
return validated addresses; the refusal runs after that validation (an
internal address has already been denied with 403) but still before the
upstream connect and before 200 Connection Established. effective_tls_skip
is still resolved up front since the refusal consumes it.

Tests: add connection-level regressions through the real
handle_tcp_connection, driving a CONNECT from a child /bin/bash copy so
the /proc process-identity binding resolves it against a permissive
policy (the hot-swap test's identity pattern). They assert the first
bytes are HTTP/1.1 503 for a terminating route with no TLS state, a 403
(not 503) for an internal address, and no refusal for a tls: skip route.
These are gated to Linux at runtime (evaluate_opa_tcp needs /proc); a
companion test verifies the OPA policy shape (glob allow, tls mode) on
every platform so the precondition is locked where /proc is unavailable.

rest.rs: tighten the fail-closed relay test to assert the forwarded
buffer is_empty() rather than merely lacking the placeholder/secret.

Signed-off-by: Tony Luo <xialuo@nvidia.com>

* test(network): keep the CONNECT handler test client fork-free

The handler regression tests forked cat to read the proxy reply, so the
client socket fd was inherited by a second process with a different
binary. The identity resolver correctly denies that as ambiguous
shared-socket ownership (the same invariant
resolve_process_identity_denies_fork_exec_shared_socket_ambiguity pins),
so the tests exercised the deny path instead of the allow path — and on
busy CI runners the deny-path /proc fallback scan exceeded the test
budget and looked like a hang. The client script now uses only bash
builtins (exec, printf, read -d '') so exactly one process owns the
socket, and the child is left to exit on EOF instead of being killed
mid-read.

Signed-off-by: Tony Luo <xialuo@nvidia.com>

* test(network): drive the CONNECT handler tests with an in-process client

The child-process client (even fork-free) made the handler tests
environment-sensitive: on CI runners with a busy or restricted /proc,
resolving the child's socket ownership degraded into the whole-/proc
fallback scan and a deny, which surfaced as a hang. The client is now an
in-process TcpStream and the test policy allows current_exe(), so
identity resolution binds the socket to the test process itself in the
descendant scan — the same in-process pattern the passing
resolve_process_identity tests rely on.

The tls: skip test additionally asserts that the handler emitted no
DenialEvent at any stage, so it can no longer pass vacuously on a
policy or identity deny. Refusal budgets widened to 30s as a belt for
slow runners; the refusals themselves return in milliseconds.

Signed-off-by: Tony Luo <xialuo@nvidia.com>

* test(core): pin the percent-encoded marker no-resolver fail-closed path

The no-resolver scan already catches the percent-encoded canonical
marker through its decoded pass; this regression pins it: a request
line carrying openshell%3Aresolve%3Aenv%3AKEY with no resolver must
fail closed with UnresolvedPlaceholderError { location: header }.

Signed-off-by: Tony Luo <xialuo@nvidia.com>

---------

Signed-off-by: Tony Luo <xialuo@nvidia.com>

* fix(server): allow newlines in exec command arguments (#1965)

* chore(deps): bump actions/stale from 10.3.0 to 10.4.0 (#2234)

* fix(tui): redraw after sandbox shell exits (#2230)

* fix(tui): redraw after sandbox shell exits

Closes #2229

Render the restored terminal before synchronous gateway refreshes and propagate terminal lifecycle failures from both SSH handoff paths.

Signed-off-by: John T. Myers <9696606+johntmyers@users.noreply.github.com>

* fix(tui): restore input after sandbox shell

Discard stale events accumulated around the suspended TUI and let the normal periodic tick refresh state after resuming.

Signed-off-by: John T. Myers <9696606+johntmyers@users.noreply.github.com>

---------

Signed-off-by: John T. Myers <9696606+johntmyers@users.noreply.github.com>

* fix(agents): add confirmation gate to triage-issue batch mode (#2239)

Batch triage previously processed all state:triage-needed issues
immediately without any confirmation. This led to accidental
mass-commenting (29 triage comments on a public repo) when the
skill was invoked by mistake.

Add a mandatory preview-and-confirm step: the agent must show the
issue count and titles, then ask for explicit user confirmation
before posting any comments.

Single-issue mode is unchanged.

Assisted-By: 🤖 Claude Code

Signed-off-by: Roland Huß <rhuss@redhat.com>

* fix(gator): retry review after draft blocker clears (#2200)

Signed-off-by: John Myers <johntmyers@users.noreply.github.com>
Co-authored-by: John Myers <johntmyers@users.noreply.github.com>

* fix(certgen): stage temp dir inside output dir to fix cross-device rename (#2241)

generate-certs stages temp files beside --output-dir using
dir.with_file_name(). When --output-dir is a container or WSL volume
mount, the staging dir lands on a different filesystem and
std::fs::rename fails with EXDEV.

Move staging inside the output dir so the rename always stays on the
same filesystem, preserving atomic replacement.

Fixes #2173

Signed-off-by: Grace Smith <grasmith@redhat.com>

* refactor(jsonrpc): carry typed inspection errors (#2244)

Signed-off-by: Shiju <shiju@nvidia.com>

* docs(agents): add gator launch skill (#2203)

* docs(agents): add gator launch skill

Signed-off-by: John Myers <johntmyers@users.noreply.github.com>

* docs(agents): narrow gator launch trigger

Signed-off-by: John Myers <johntmyers@users.noreply.github.com>

* docs(agents): trim gator launch skill intro

Signed-off-by: John Myers <johntmyers@users.noreply.github.com>

* docs(agents): harden gator launch examples

Signed-off-by: John Myers <johntmyers@users.noreply.github.com>

* docs(agents): avoid assuming gator gateway

Signed-off-by: John Myers <johntmyers@users.noreply.github.com>

* docs(agents): trim gator launch rules

Signed-off-by: John Myers <johntmyers@users.noreply.github.com>

---------

Signed-off-by: John Myers <johntmyers@users.noreply.github.com>
Co-authored-by: John Myers <johntmyers@users.noreply.github.com>

* chore(python): lower minimum supported Python to 3.11 (#2247)

Debian 12 stable and other LTS environments ship Python 3.11 as the
system interpreter. The SDK is pure Python with a bundled native
binary, uses no 3.12-only syntax or stdlib APIs, and all dependencies
support 3.11. Lower the floor so `pip install openshell` / `uv add
openshell` works on 3.11 (security-supported until Oct 2027).

Verified with py_compile on CPython 3.11.15 plus ruff and ty.

Signed-off-by: Max Dubrinsky <mdubrinsky@nvidia.com>

* fix(policy): keep approved chunk when a mechanistic denial resubmits its endpoint (#2242)

* fix(policy): keep approved chunk when a mechanistic denial resubmits its endpoint

A mechanistic denial flush for an endpoint already covered by an
auto-approved mechanistic chunk flipped that chunk approved -> rejected
with no human action. The dedup upsert in put_draft_chunk returns the
existing row's id, which aliases onto the approved chunk; the self-reject
scan then matched the row against itself and rejected it, while the merged
rule stayed enforced — the governance ledger disagreed with the live policy.

Guard self_reject_mechanistic_if_already_covered to act only on a still
pending effective chunk, and exclude the incoming id from the covering
scan. Add a regression test and document the dedup/self-reject invariant.

Fixes #2165
Refs NVIDIA/NemoClaw#6329

Signed-off-by: Tinson Lai <tinsonl@nvidia.com>

* fix(policy): reject mechanistic chunk via atomic pending compare-and-set

The self-reject-when-covered path read a chunk, confirmed it was pending,
then issued an unconditional status update to rejected. An approval that
committed between the read and the write flipped an already-approved chunk
to rejected while its rule stayed merged, recreating the ledger/enforcement
mismatch through a concurrent path.

Add conditionally_reject_draft_chunk to the policy store: the pending->rejected
transition carries a status = 'pending' predicate on the final write and
reports whether a row changed. Zero changed rows is a benign no-op, meaning
another operation already decided the chunk. Implemented for both SQLite and
PostgreSQL. The pending pre-read and the self-exclusion guard stay as
defense-in-depth. Adds persistence-level regression tests proving an approved
row cannot be conditionally rejected and that an approval racing the reject
wins.

Signed-off-by: Tinson Lai <tinsonl@nvidia.com>

---------

Signed-off-by: Tinson Lai <tinsonl@nvidia.com>

* fix(tasks): format all Rust workspaces (#2268)

Run cargo fmt for both the root workspace and the standalone e2e Rust workspace
from the rust format and format-check tasks. Apply rustfmt to the e2e sources
so the expanded formatting check passes.

Signed-off-by: Kris Hicks <khicks@nvidia.com>

* docs: fix stray bracket in provider create command example (#2275)

Key Commands table showed --type [type]] --from-existing with an
extra closing bracket.

* rfc-0010: gateway interceptors (#1927)

* docs(rfc): add gateway interceptors RFC

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* docs(rfc): clarify gateway interceptors proposal

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* docs(rfc): clarify interceptor source of truth

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* docs(rfc): refine gateway interceptor proposal

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* wip

* docs(rfc): document interceptor order example

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* docs(rfc): renumber gateway interceptors RFC

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* docs(rfc): clarify gateway interceptor service

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* docs(rfc): clarify gateway interceptor limits

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* docs(rfc): align gateway interceptor config

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* docs(rfc): refine gateway interceptor contract

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* docs(rfc): clarify gateway interceptor payload contract

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* docs(rfc): add gateway interceptor describe request

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* docs(rfc): remove interceptor modifies flag

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* docs(rfc): shape interceptor evaluation by phase

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* docs(rfc): simplify interceptor mutation phase

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* docs(rfc): update interceptor post-commit failure mode

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* docs(rfc): accept gateway interceptors

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

---------

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* fix(release-dev): update azure/setup-helm to v5.0.1 (#2274)

Dependabot updated the Helm setup action in release-canary but missed the same
reference in the release composite action.

Signed-off-by: Kris Hicks <khicks@nvidia.com>

* feat(snap): vendor ssh in openshell snap and remove ssh-keys interface (#2280)

Previously, the openshell snap used the ssh-keys interface to get access
to the host's ssh binary, which is used for sandbox connect/exec/forward.
However, ssh-keys is a privileged interface which also grants access to
the public and private ssh keys on the host. As such, it required manual
connection in order to be used.

This weakened the security sandbox of the snap, and hurt the UX of
installing it.

This commit changes this by removing the `ssh-keys` interface and
instead vendoring the `ssh` binary within the snap.

This is safe because OpenShell always invokes the `ssh` binary with
`StrictHostKeyChecking=no`, `UserKnownHostsFile=/dev/null`, and
`GlobalKnownHostsFile=/dev/null`, and never uses any host credentials or
ssh configuration. Openshell only ever access to `~/.ssh/config` to
write OpenShell-managed aliases, and this can safely live within the
snap sandbox, rather than leaking into the host environment.

Signed-off-by: Oliver Calder <oliver.calder@canonical.com>

* feat(interceptors): initial gateway interceptor implementation and reference example (#2005)

* feat(gateway): add descriptor-driven interceptors

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* feat(gateway): add service-reflected interceptors

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* wip

* fix(gateway): harden interceptor evaluation

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* feat(interceptors): label metrics and harden governance smoke

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* remove on_error: ignore

* feat(gateway-interceptors): emit log annotations

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* feat(examples): govern provider profiles in interceptor

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* fix(gateway): preserve update config annotations

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* feat(providers): support interceptor profile catalogs

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* wip

* feat(governance-interceptor): sign provider profiles

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* fix(providers): use configured profile sources for refresh updates

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* feat(gateway-interceptors): add phase-specific evaluation payloads

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* feat(providers): compose provider profile sources

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* fix(gateway-interceptors): preserve committed responses

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* fix(gateway-interceptors): reject ambiguous protobuf oneofs

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* fix(gateway-interceptors): validate patch candidates per binding

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* refactor(gateway-interceptors): use reflected protobuf codec

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* fix(governance-example): canonicalize signed protobuf hashes

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* fix(gateway): commit policy provenance atomically

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* fix(gateway): close signed governance bypasses

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* fix(gateway): isolate interceptor secrets and authority

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* fix(gateway): snapshot provider profiles per request

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* chore(gateway): resolve server clippy warnings

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* fix(server): satisfy provider source clippy lint

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* fix(server): initialize policy test annotations

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* fix(gateway-interceptors): require explicit route allowlist

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* docs(proto): clarify update annotation semantics

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

---------

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* feat(sdk): add openshell-sdk crate (#1862)

* feat(sdk): add openshell-sdk crate

Additive extraction of the shared async gRPC client core (transport, TLS,
OIDC single-flight refresh, edge tunnel, high-level sandbox surface, raw
escape hatch) as a new workspace crate. No existing consumers yet; CLI/TUI
migration follows in a separate PR.

Signed-off-by: Max Dubrinsky <mdubrinsky@nvidia.com>

* refactor(sdk,core): reuse shared JWT exp decoder for refresh deadlines

Extract the signature-unverified JWT exp decode out of
openshell-core/grpc_client.rs into openshell_core::jwt::parse_exp_secs,
and have the openshell-sdk refresh path reuse it to derive a proactive
refresh deadline from a bearer JWT when the caller does not advertise
expires_at. Addresses review feedback to reuse pre-existing logic rather
than reimplement JWT expiry handling per client.

Signed-off-by: Max Dubrinsky <mdubrinsky@nvidia.com>

* fix(sdk): harden refresh single-flight and redact tokens in Debug

Addresses review feedback on the openshell-sdk refresh path.

- Make the single-flight cleanup cancellation-safe. The in-flight slot is
  now cleared by the shared refresh computation itself (epoch-guarded)
  rather than the leader's post-await code. Previously, if the leader future
  was dropped (e.g. an FFI caller cancelling its promise) after a follower
  drove the refresh to completion, the completed future was stranded in the
  slot and later refresh_now() calls re-joined it, pinning the client to a
  stale or already-rejected token. Adds a regression test that cancels the
  leader and asserts the next refresh starts a fresh attempt.

- Redact bearer secrets from Debug. RefreshedToken and the oidc
  RefreshTokenInput/RefreshTokenOutput now use manual Debug impls that omit
  the access/refresh token fields via finish_non_exhaustive, matching the
  house style (e.g. SecretResolver, SandboxJwtIssuer). Prevents a stray
  {:?} or a containing struct's derived Debug from writing tokens to logs.

Signed-off-by: Max Dubrinsky <mdubrinsky@nvidia.com>

* fix(sdk): fail refresh when the new token can't be encoded as metadata

store_bearer now returns an error instead of silently keeping the previous
bearer value. The TokenSource commits the refreshed token to its state
before the client writes it into the interceptor slot, so a silent drop
left the interceptor on the old (expiring) token with no path back to a
refresh. Surfacing the error fails the call loudly instead. Adds a unit
test covering a token that can't be encoded as gRPC metadata.

Signed-off-by: Max Dubrinsky <mdubrinsky@nvidia.com>

* fix(sdk): refresh OIDC tokens on raw routes and harden rotation

Raw gRPC access never triggered OIDC refresh: a client that only used
raw_grpc/raw_inference kept sending the initial bearer until it expired,
with no proactive or reactive refresh. Add raw_grpc_fresh and
raw_inference_fresh accessors that refresh before returning the client,
plus force_refresh for reactive recovery after an Unauthenticated raw
RPC.

Guard the single-flight refresh commit against a concurrent replace().
The in-flight attempt now records the generation it started from and
skips its write when an external replace() has advanced it, so timer or
callback driven rotation is no longer clobbered by a slower refresh.

Remove TokenSource::snapshot(): it returned an empty string under write
contention and had no consumer on the CLI/TUI path. Tests read committed
state directly instead.

Signed-off-by: Max Dubrinsky <mdubrinsky@nvidia.com>

* docs(sdk): rewrite crate README for consumers

Recast the openshell-sdk README as a usable crate README rather than an
RFC excerpt. Drop the Responsibilities/Non-responsibilities/Consumers
scope-boundary sections and the mTLS migration rationale, folding the
useful facts (explicit token, no disk/name resolution, Refresh trait,
SdkError mapping) into the intro, a new Auth and refresh section, and
Public surface. Remove the dead relative RFC link and status-label
prose so the doc renders cleanly wherever it is published.

Signed-off-by: Max Dubrinsky <mdubrinsky@nvidia.com>

* fix(sdk): address review feedback on OIDC refresh and transport

Apply Drew's review notes on PR #1862:

- Drop unused `rustls-pemfile` dependency and move `tokio-stream` to
  dev-dependencies (only used by tests).
- Guard OIDC `expires_at` against u64 overflow with `saturating_add`.
- Fix stale `#[non_exhaustive]` rationale in `AuthConfig` (the struct
  `Oidc` variant it described as future already ships).
- Stop double-wrapping refresh errors: store the bare refresh-error text
  so the single `SdkError::auth` wrap happens once at await.
- Strip stale CLI porting breadcrumbs from `build_channel` docs, keeping
  the branch table.
- Treat proactive token refresh as best-effort: a transient failure falls
  through to the request instead of failing an RPC whose current token is
  still valid, with a regression test.
- Collapse `exec`'s inline auth retry into the shared `unary` helper.

Signed-off-by: Max Dubrinsky <mdubrinsky@nvidia.com>

* fix(sdk): preserve transient/terminal distinction in refresh errors

The refresh single-flight collapsed both `RefreshError::Transient` and
`RefreshError::Terminal` into a stringified `SdkError::Auth`, so consumers
(CLI, TUI, future language bindings) had no machine-readable way to tell a
retryable IdP blip from a dead session that needs re-authentication.

Carry the `RefreshError` through the shared outcome (kept `Clone` for
`Shared`) instead of its rendered text, and map it at the await site to a
new `retryable` flag on `SdkError::Auth`. Add `SdkError::auth_retryable`
and a `SdkError::retryable()` accessor; transient refresh failures report
`true`, every other error `false`. Add classification tests.

Signed-off-by: Max Dubrinsky <mdubrinsky@nvidia.com>

---------

Signed-off-by: Max Dubrinsky <mdubrinsky@nvidia.com>

* fix(sdk): initialize sandbox annotations (#2296)

The SDK merged after CreateSandboxRequest and ObjectMeta gained annotations on main, leaving stale struct initializers that prevented the crate and its tests from compiling. Initialize the curated request and mock metadata with empty annotation maps.

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* fix(driver-vm): run sandbox supervisor as guest pid 1 (#2299)

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* feat(ci): introduce merge queue (#2024)

* feat(ci): introduce merge queue

Closes #1946

Signed-off-by: Evan Lezar <elezar@nvidia.com>

* fix(ci): run GPU E2E for merge groups

Signed-off-by: Evan Lezar <elezar@nvidia.com>

* docs(ci): clarify merge queue GPU gate

Signed-off-by: Evan Lezar <elezar@nvidia.com>

---------

Signed-off-by: Evan Lezar <elezar@nvidia.com>

* feat(gateway): add elevated gateway info (#2202)

* feat(cli)!: fold gateway metadata into list

BREAKING CHANGE: openshell gateway info no longer shows local gateway registration metadata. Use openshell gateway list or openshell gateway list -o json for local registration details.

Signed-off-by: Evan Lezar <elezar@nvidia.com>

* feat(gateway): add elevated gateway info

Signed-off-by: Evan Lezar <elezar@nvidia.com>

---------

Signed-off-by: Evan Lezar <elezar@nvidia.com>

* fix(ci): prune snap assets from dev release (#2302)

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

* feat!(openshell-cli): remove openshell policy prove command and z3 dependency (#2318)

Remove Z3 from the openshell CLI. Proving is handled by the gateway, so bundling the solver in the client duplicates functionality and complicates portable CLI builds and packaging.

Signed-off-by: Simon Scatton <sscatton@nvidia.com>

* fix(server): persist sandbox labels on create (#2306)

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>

* fix: remove mentions of bundled-z3 in CI and wheel builds (#2322)

* fix(gateway): probe Docker socket during driver auto-detection (#2303)

Previously, Docker was auto-detected when the CLI was installed or a candidate
Unix socket existed. Neither check verified that the Docker API was responsive.
A similar check was done when auto-detecting Podman in the past, but was
replaced in 1f07bf04 with a probe of candidate Podman sockets instead.

This change applies the functional API probing approach introduced for Podman
in 1f07bf04 to Docker. It also makes Docker driver initialization use the same
socket-selection mechanism as Docker auto-detection instead of Bollard’s local
defaults. This means the previously auto-detectable Docker socket paths
$HOME/.docker/run/docker.sock and $XDG_RUNTIME_DIR/docker.sock will actually be
usable.

When no working compute driver can be auto-detected, the gateway exits early
with a message saying as much:

> configuration error: no compute driver configured and auto-detection found no
> suitable driver; set --drivers or OPENSHELL_DRIVERS to kubernetes, podman,
> docker, or vm

This makes for a better user experience when installing OpenShell without an
available supported compute driver.

Signed-off-by: Kris Hicks <khicks@nvidia.com>

* chore(deps): bump actions/setup-node from 6.4.0 to 7.0.0 (#2289)

Bumps [actions/setup-node](https://github.com/actions/setup-node) from 6.4.0 to 7.0.0.
- [Release notes](https://github.com/actions/setup-node/releases)
- [Commits](https://github.com/actions/setup-node/compare/48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e...820762786026740c76f36085b0efc47a31fe5020)

---
updated-dependencies:
- dependency-name: actions/setup-node
  dependency-version: 7.0.0
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Evan Lezar <elezar@nvidia.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump softprops/action-gh-release from 3.0.1 to 3.0.2 (#2288)

Bumps [softprops/action-gh-release](https://github.com/softprops/action-gh-release) from 3.0.1 to 3.0.2.
- [Release notes](https://github.com/softprops/action-gh-release/releases)
- [Changelog](https://github.com/softprops/action-gh-release/blob/master/CHANGELOG.md)
- [Commits](https://github.com/softprops/action-gh-release/compare/718ea10b132b3b2eba29c1007bb80653f286566b...3d0d9888cb7fd7b750713d6e236d1fcb99157228)

---
updated-dependencies:
- dependency-name: softprops/action-gh-release
  dependency-version: 3.0.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* feat(tui): navigate panels via Up/Down arrow overflow at list boundaries (#2287)

* feat(tui): navigate panels via Up/Down arrow overflow at list boundaries

When at the bottom of a panel's item list, pressing Down/j now moves
focus to the next panel instead of being a silent no-op. Likewise,
pressing Up/k at the top moves to the previous panel with the cursor
on its last item. Empty panels are skipped and the ring wraps around.

Closes #2273

Signed-off-by: Varsha Prasad Narsing <vnarsing@nvidia.com>
Signed-off-by: Varsha Prasad Narsing <varshaprasad96@gmail.com>

* fix(tui): guard Up handlers against stale cursor in empty panels

The Down handlers already check whether the list is non-empty before
incrementing the cursor, but the Up handlers only checked cursor > 0.
When a list becomes empty after a refresh with a nonzero cursor, Up
would decrement the stale cursor instead of overflowing to the
previous panel. Add the same non-empty guard to all four Up arms.

Signed-off-by: Varsha Prasad Narsing <varshaprasad96@gmail.com>

* docs(sandboxes): add dashboard keyboard navigation to manage-sandboxes

Describe Tab/Shift+Tab panel cycling, Up/Down and j/k boundary
overflow, and middle-pane tab switching in the OpenShell Terminal
section.

Signed-off-by: Varsha Prasad Narsing <varshaprasad96@gmail.com>

---------

Signed-off-by: Varsha Prasad Narsing <vnarsing@nvidia.com>
Signed-off-by: Varsha Prasad Narsing <varshaprasad96@gmail.com>

* feat(providers): AWS STS AssumeRole refresh strategy and aws-s3 profile (#1782)

Add gateway-managed AWS STS credential refresh (provider-v2, #1576). The
gateway calls sts:AssumeRole and writes three short-lived credentials
(AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_SESSION_TOKEN) to the
provider record; the proxy re-signs requests with SigV4. Adds the aws and
aws-s3 provider profiles and a declarative multi-output refresh model
(additional_outputs) so one AssumeRole co-mints all three credentials.

Signed-off-by: Russell Bryant <rbryant@redhat.com>

* rfc-0009: supervisor middleware (#1738)

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

* test(e2e): run VM suite in CI (#2305)

* test(e2e): run VM suite in CI

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* fix(ci): configure KVM permissions directly

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* fix(e2e): flush VM overlay before restart

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* docs: simplify VM test documentation

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* test(e2e): include gateway resume in VM run

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

---------

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* fix(vm-driver): fixes BYOC sandbox creation failing with ext4-fs write access unavailable (#2150)

* feat(supervisor-middleware): add network egress middleware (#2027)

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

* docs(gator): require inline review comments (#2346)

Signed-off-by: John Myers <johntmyers@users.noreply.github.com>
Co-authored-by: John Myers <johntmyers@users.noreply.github.com>

* fix(cli): preserve symlinks in sandbox upload (#2319)

Signed-off-by: lr90 <qiuweimin@matrixorigin.cn>

* fix(kubernetes): validate sandbox names against RFC 1123 requirements (#2295)

Signed-off-by: Krzysztof Malczuk <kmalczuk@redhat.com>

* docs: bump stated Rust MSRV from 1.88 to 1.90 (#2276)

* docs: bump stated Rust MSRV from 1.88 to 1.90

Cargo.toml sets rust-version = "1.90" (rust-toolchain.toml pins
1.95.0), so building with the previously documented 1.88 fails
Cargo's MSRV check.

* docs: bump e2e/rust MSRV to 1.90

* fix: align remaining Rust version fields to 1.90

examples/governance-interceptor/Cargo.toml still had rust-version
1.88. Also bump e2e/rust's prost dependency to 0.14 to match the
workspace, since it was on 0.13 in an otherwise standalone crate.

* ci: pin docker actions to commit SHA (#2328)

login-action and setup-buildx-action used a mutable version tag while
every other action in the repo is pinned to a commit SHA. Pin both,
and align login-action to the same v4 SHA already used in ci-image.yml.

Use the full resolved version in the trailing comment (v3.12.0) to
match the more common convention used elsewhere in .github/.

* ci(e2e): reuse prebuilt CLI and gateway artifacts (#2311)

* ci(e2e): reuse prebuilt CLI artifacts

Signed-off-by: Evan Lezar <elezar@nvidia.com>

* ci(e2e): reuse prebuilt gateway artifacts

Signed-off-by: Evan Lezar <elezar@nvidia.com>

* ci(e2e): reuse prebuilt VM driver artifact

Signed-off-by: Evan Lezar <elezar@nvidia.com>

---------

Signed-off-by: Evan Lezar <elezar@nvidia.com>

* docs: fix broken links and small inconsistencies (#2329)

- README: fix github-sandbox tutorial link missing get-started segment
- README: replace dead community-sandboxes doc link with the actual repo
- README: match supported host list to support-matrix.mdx
- architecture/README: list the missing google-vertex-ai-provider doc
- SECU…
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:gateway Gateway server and control-plane work area:providers test:e2e Requires end-to-end coverage

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants