Repository navigation
fix(runs): reach the platform's internal RFT router for hosted runs - #895
Merged
Merged
Conversation
Hosted managed RFT runs start their orchestrator with `PRIME_API_BASE=<platform>/api/internal/rft` and a per-run token; that router authenticates the token from the `x-api-key` header and serves the monitoring routes (metrics, presign, confirm, finalize) only. `PlatformClient` addressed everything under `<base>/api/v1` with a Bearer header alone, so an attached hosted run would 404 every upload and go silently unmonitored. `PlatformClient` now recognises the internal root (with or without the trailing `/rft`): requests go under `<base>/api/internal` and the token is repeated in `x-api-key`. The public root is unchanged, and a `/api/v1/rft` root is accepted the same way. Headers are also sent per request, so a caller-supplied `httpx.Client` authenticates like one the SDK opened. Docs: `init()`, `RftBackend.attach`, README. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EYdQmKAVnGteT1tkimJb3g
kcoopermiller
requested review from
DamianB-BitFlipper,
JannikSt,
JohannesHa,
burnpiro,
kennethnym and
willccbb
as code owners
September 4, 2026 04:18
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c9e1a2eaf3
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
`split_api_root` stripped `/rft` from the whole string, so a platform host that is itself named `rft` (`http://rft`, a plain service name) collapsed to `http:/` and every request went to a host called `api`; the same slip existed for `/api/v1` on a host named `api` before this branch. Suffixes are now matched on the parsed URL's path only, longest first, and `/rft` is recognised only after an API path. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EYdQmKAVnGteT1tkimJb3g
kennethnym
approved these changes
Sep 4, 2026
kcoopermiller
added a commit
that referenced
this pull request
Sep 4, 2026
* fix(runs): reach the platform's internal RFT router for hosted runs Hosted managed RFT runs start their orchestrator with `PRIME_API_BASE=<platform>/api/internal/rft` and a per-run token; that router authenticates the token from the `x-api-key` header and serves the monitoring routes (metrics, presign, confirm, finalize) only. `PlatformClient` addressed everything under `<base>/api/v1` with a Bearer header alone, so an attached hosted run would 404 every upload and go silently unmonitored. `PlatformClient` now recognises the internal root (with or without the trailing `/rft`): requests go under `<base>/api/internal` and the token is repeated in `x-api-key`. The public root is unchanged, and a `/api/v1/rft` root is accepted the same way. Headers are also sent per request, so a caller-supplied `httpx.Client` authenticates like one the SDK opened. Docs: `init()`, `RftBackend.attach`, README. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EYdQmKAVnGteT1tkimJb3g * release: prime-runs 0.1.1 Bumps prime-runs to 0.1.1 so release-runs.yml publishes the one change since 0.1.0: #895 (PlatformClient reaches the platform's internal RFT router for hosted runs, with the run's token in x-api-key). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EYdQmKAVnGteT1tkimJb3g * fix(runs): normalise only the URL path when recognising an API root `split_api_root` stripped `/rft` from the whole string, so a platform host that is itself named `rft` (`http://rft`, a plain service name) collapsed to `http:/` and every request went to a host called `api`; the same slip existed for `/api/v1` on a host named `api` before this branch. Suffixes are now matched on the parsed URL's path only, longest first, and `/rft` is recognised only after an API path. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EYdQmKAVnGteT1tkimJb3g --------- Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
This was referenced Sep 4, 2026
kcoopermiller
added a commit
to PrimeIntellect-ai/prime-rl
that referenced
this pull request
Sep 9, 2026
Replaces `monitors/prime.py`'s`TrainRun` with `pr.init(kind="train", ...)`. ## What moved where | was in prime-rl | now | | --- | --- | | `TrainRun.create` / `$RUN_ID` attach | `pr.init(kind="train", model=, environments=, training=pr.TrainingSpec(...), config=, ...)` / `pr.init(kind="train", id=$RUN_ID)` | | `TrainRun.log_metrics` (fire-and-forget task per POST) | `run.log_metrics(values, step=)` — a queue put; the SDK's uploader owns retries, `Retry-After`, backpressure | | `episodes_to_parquet_bytes` + `SAMPLE_SCHEMA` + presign→PUT→confirm | `prime_runs.projection` + the SDK's training samples sink (same schema, same flow) | | the `step % 10` cadence in `log_episodes` | the sink (`step_interval=10`), keyed on each episode's `run.work.step` — the `TrainRunInfo` the dispatcher stamps | | `finalize` → `POST /finalize`, fallback `set_status` | `run.finish()` — drain, then an idempotent finalize; 409 on the failure path is treated as already-closed | | atexit `_mark_failed` | the SDK's atexit hook reports **crashed** (with a message); an attached run's failure marking stays with the launcher, as before | The `kind == "train" and subset == "effective"` cohort filter stays here. The monitor keeps `sanitize()` so dropped non-finite metric paths are still named in the log (the SDK would drop them silently). `$PRIME_API_BASE` still works — it historically points at `…/api/v1/rft`, so the monitor strips what the SDK re-adds. ## Decisions - **Step keying.** The samples sink keys each episode by the step it was *dispatched* at (`run.work.step`), not the batch step; the platform keeps one Parquet object per upload and the viewer unions every object under a step, so an off-policy episode landing in a later batch adds an object rather than replacing one, and each object numbers its `sample_id`s in its own range (SDK side, `aba26790`). "Step N" in the sample viewer therefore means dispatched at N, while the metrics at N describe the batch trained at N. - **Run identity.** The orchestrator keeps the launcher's `PRL_RUN_ID` (or a local uuid) as the run identity for episodes, W&B and sandbox labels, exactly as on `main`. The SDK keys every uploaded record to the platform run (`run.id` / `run.type` on the uploaded copy, `work` kept), so Prime Traces documents carry the id the dashboard shows without the orchestrator learning it. ## Dependency `prime-runs[train]>=0.1.1` from PyPI ([0.1.1](https://pypi.org/project/prime-runs/0.1.1/), PrimeIntellect-ai/prime#896); the lock carries `prime-runs 0.1.1` + `prime-traces 0.0.3` from the registry. The monitor's own unit tests run in CI; there is no GPU-free end-to-end path for the training seam, so the RFT contract was verified by reading the platform's `/api/v1/rft/*` and `/api/internal/rft/*` routes against the SDK's calls. ## Hosted managed runs Since PrimeIntellect-ai/platform#4885, hosted managed runs start the orchestrator with `--monitors.prime.name`, `RUN_ID`, a per-run token in `PRIME_API_KEY` and `PRIME_API_BASE=<platform>/api/internal/rft`. prime-runs 0.1.1 (PrimeIntellect-ai/prime#895) addresses that internal router and sends the run's token in `x-api-key`, the header it authenticates on; 0.1.0 only spoke `/api/v1` with a Bearer header and would have left those runs silently unmonitored. This monitor's `_base_url()` passes `$PRIME_API_BASE` minus the trailing `/rft`, which 0.1.1 accepts, so the floor bump is the whole change. External runs (prime-rl launched outside the platform) were never affected. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01Fk5fBAAaTiPfJ5UpNyqxXR <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Medium Risk** > Changes how training runs register and upload to the Prime platform (including hosted `$RUN_ID` / internal RFT paths); misconfiguration now fails launch instead of silently skipping monitoring, but orchestrator run identity for W&B stays unchanged. > > **Overview** > **Replaces** the in-repo `TrainRun` HTTP/Parquet implementation in `PrimeMonitor` with **`prime-runs[train]>=0.1.1`**, so run registration (or `$RUN_ID` attach), metrics streaming, episode sample uploads, finalize, and crash reporting are owned by the SDK uploader instead of custom `httpx` + fire-and-forget tasks. > > The monitor layer is now a thin adapter: `pr.init(kind="train", …)` on startup, `run.log_metrics` / `run.log_episodes` via `asyncio.to_thread`, and `run.finish()` with a **60s** drain timeout. Local behavior kept where it matters: non-finite metrics are still **sanitized with named warnings**, only **train / effective** episodes are forwarded, and **`$PRIME_API_BASE`** is normalized for the SDK. Episode **every-10th-step** upload cadence moves into the SDK sink (keyed on dispatch step on `run.work`). > > **Docs** in `training.md` describe the SDK, team allowlisting, **`PRIME_RUNS_MODE=disabled`**, and **`RUN_ID`** attach. **Lock/pyproject** add `prime-runs` and transitive **`prime-traces`**, with fast-track resolver entries. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit aed9bbf. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY --> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Since Sep 1, hosted managed RFT runs start their orchestrator with
--monitors.prime.name,RUN_ID, a per-run token inPRIME_API_KEYandPRIME_API_BASE=<platform>/api/internal/rft. That internal router authenticates the run's token from thex-api-keyheader and serves the monitoring routes only (/rft/metrics,/rft/samples/presign,/rft/samples/confirm,/rft/finalize). The old prime-rlTrainRunused that root verbatim and sent bothAuthorizationandx-api-key.PlatformClientaddressed everything under<base>/api/v1with a Bearer header alone, so once prime-rl's monitor moves to this SDK (PrimeIntellect-ai/prime-rl#3459) an attached hosted run would 404 every upload and go silently unmonitored — the failure #4885 had just fixed. Against prod, unauthenticated, for the record:What
PlatformClientrecognises the internal root (…/api/internal, with or without the trailing/rft): requests go under<base>/api/internaland the token is repeated inx-api-key. The public root is unchanged; a…/api/v1/rftroot (the RFT client's historical default) is normalised the same way.client.internalsays which router is in use.httpx.Clientauthenticates like one the SDK opened.init()andRftBackend.attachdocstrings, README (training bullet + configuration). Registering a run or setting its status is not available on the internal root — attach only — which is exactly the hosted case.No change for external runs or evals: same URLs, same headers.
Tests
tests/test_http.py: internal roots →/api/internalprefix +x-api-key(parametrised over the three spellings), public roots →/api/v1and nox-api-key(four spellings), injected client gets the headers.tests/test_rft_backend.py: attach + finalize through the internal router with the run's token. 254 passed; ruff clean.Release
Ships as prime-runs 0.1.1 (release PR follows, stacked on this one). prime-rl#3459 then bumps to
prime-runs[train]>=0.1.1; its_base_url()already passes$PRIME_API_BASEminus/rft, which this accepts unchanged.🤖 Generated with Claude Code
https://claude.ai/code/session_01EYdQmKAVnGteT1tkimJb3g
Note
Medium Risk
Changes core HTTP routing and auth for all platform calls when
base_urlincludes an internal path; mistakes could break hosted monitoring or mis-route external runs, though public URLs are explicitly preserved and heavily tested.Overview
PlatformClientnow supports the internal RFT base URL that hosted training launchers inject (…/api/internalor…/api/internal/rft). When that root is detected via newsplit_api_root(), requests target/api/internal(not a doubled/api/v1path) and the run token is sent inx-api-keyin addition to Bearer auth;client.internalflags this mode. Public origins and historical…/api/v1/rftURLs keep the previous behavior.Each HTTP call merges auth headers on the request so a caller-injected
httpx.Clientstill authenticates like the SDK-owned pool.README and
init()/RftBackend.attachdocs explain attaching hosted runs with$PRIME_API_BASEand that register/status APIs stay on the public API only. Tests cover internal vs public URL spellings, hostnames that look like path suffixes, per-request headers, and finalize through the internal router.Reviewed by Cursor Bugbot for commit b1a26e6. Bugbot is set up for automated code reviews on this repo. Configure here.