Initial sush integration - #11034
Open
plotnick wants to merge 11 commits into
Open
Conversation
The Support Shell server in the global zone serves its API here. See RFD 620. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Callers had only a snapshot of the currently available datasets, so noticing that one appeared meant polling. The debug datasets are encrypted and do not mount until the keys are available, which makes waiting for one a useful signal. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Run the Support Shell server in the global zone as a set of Tokio tasks, so that Oxide support can run authorized jobs on a sled when the control plane is down. The job manager starts during bootstrap, as soon as we know our baseboard; the HTTP API starts later, once this sled has been told its underlay address. Job output is recorded on the ramdisk at first, with a small size limit, and moves to an encrypted U.2 debug dataset with a larger limit once one is mounted. Such a dataset cannot mount before trust quorum is established, so waiting for one avoids having to know anything about trust quorum. Jobs and sessions gossip between sleds over sprockets on the bootstrap network, using the sled's own sprockets identity and reference measurements. Peer addresses come from the same DDM poller that feeds the bootstore and trust quorum. A sled whose gossip listener cannot start still serves local jobs. sush does not run unless it is configured, and the new [sush] section is absent from every shipped config. Session state is not yet persisted, so a restart forgets the current session. sush declares sled-hardware-types as a git dependency on this repo, so without the patch directive the graph holds two copies of the crate and its BaseboardId is a different type from the one sled-agent derives. We ask for sush's embedded feature, which refuses to compile if anything in the build also turns on its test-support feature. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Give every shipped config a [sush] section trusting the Support Shell sandbox root, so development racks can run signed jobs end to end. Job execution requires a permission-slip signature chaining to this root, so possession of the certificate alone grants nothing. The sandbox key is "UNTRUSTED Support Shell Prototype", held by the permslip development instance: permslip --url=https://permslip.inickles.0xeng.dev \ get-cert "UNTRUSTED Support Shell Prototype" This must not reach a customer rack: a trusted root cannot be revoked, only replaced by an OS update. TODO: replace with the production root before release.
plotnick
force-pushed
the
sush-integration
branch
from
August 10, 2026 06:01
20528c3 to
d2f73c5
Compare
plotnick
commented
Aug 10, 2026
| /// Returns the sled addresses behind advertised underlay prefixes. | ||
| /// These are candidates that callers must probe: not every underlay | ||
| /// prefix is a sled subnet (RFD 63). | ||
| pub async fn derive_sled_addrs_from_prefixes( |
Contributor
Author
There was a problem hiding this comment.
If someone from the networking team could please check the prefix math here, that would be great.
plotnick
force-pushed
the
sush-integration
branch
2 times, most recently
from
August 10, 2026 06:46
e7ebe73 to
f7ec06c
Compare
A proxy-only sush service in each switch zone (RFD 620). Sleds are discovered by probing the addresses behind the bootstrap and underlay prefixes DDM advertises, and the cubby numbering comes from MGS. Sleds answer on their bootstrap addresses from startup, so the proxy reaches them before RSS runs. The proxy serves TLS with the sled's platform identity: at zone startup, sled-agent mints an ephemeral key, has the RoT sign its certificate once over IPCC, and writes the key and chain into the zone for the proxy to serve with. A failed mint fails the proxy's SMF service alone, not the switch zone. Note for review: someone from the networking team should check the prefix math in derive_sled_addrs_from_prefixes, which assumes every /64 that is not a bootstrap prefix is a sled subnet.
Cubby-targeted jobs need the cubby map on the sled that decides whether to run them, not just at the proxy. Each sled-agent polls MGS for the map and feeds it to the embedded sush server through a watch channel: candidate switch zone addresses are derived from the underlay subnets DDM advertises, and whatever answers is an MGS. Per RFD 63 the derived subnets are candidates only (sleds also advertise internal DNS subnets, and a services prefix is reserved), so callers must probe, as both this loop and the sush proxy's sled discovery do.
plotnick
force-pushed
the
sush-integration
branch
from
August 10, 2026 13:34
f7ec06c to
a3148b2
Compare
This was referenced Aug 10, 2026
sprockets now gates IPCC behind a default-on cargo feature. Take the workspace dependency off the default, and give sled-agent the feature explicitly: it parses the ipcc config variants and mints the sush proxy identity. Advance the sush pin, which opts out so the support shell client no longer links libipcc.
plotnick
force-pushed
the
sush-integration
branch
from
August 10, 2026 22:29
55a98ce to
78dde18
Compare
plotnick
marked this pull request as ready for review
August 12, 2026 14:15
Pins by rev now that the branch is about to merge. Picks up the codephrase wire format, the client overhaul, and warm gossip connections. Old clients cannot authenticate across this change.
plotnick
force-pushed
the
sush-integration
branch
2 times, most recently
from
August 14, 2026 06:25
ef3cec1 to
1dc8b25
Compare
plotnick
force-pushed
the
sush-integration
branch
from
August 14, 2026 06:43
1dc8b25 to
43d1d4a
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This embeds the Oxide Support Shell (
sush, RFD 620) in sled-agent and adds its switch zone proxy. It pairs with sush#31. Opened as early draft so CI builds TUF repos for racklette testing, and must stay as draft until the pin moves from a branch to a rev and we have a production cert. Nexus integration is forthcoming, and not addressed at all by this PR.The proxy TLS key handling could use a review from a security person, please. Basic idea is that we mint a fresh key at switch zone startup and have the RoT sign a certificate for it. The client then verifies that the leaf chains to the production or staging platform identity roots.
🤖 Disclaimer: written with LLM assistance.