Skip to content

feat(api): add generic System One decision inference and model discovery - #14667

Open
PixmaNts wants to merge 8 commits into
diegosouzapw:release/v3.8.52from
PixmaNts:feat/systemone-models
Open

PixmaNts wants to merge 8 commits into
diegosouzapw:release/v3.8.52from
PixmaNts:feat/systemone-models

Conversation

@PixmaNts

@PixmaNts PixmaNts commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Make System One a provider/model capability rather than an OpenRouter-only proxy.
Typed noul, choice and score questions use the same public
POST /v1/systemone operation with native TypeSafe, OpenRouter or local Ollama.

The model identity follows the ordinary gateway convention:

Requested model Connection provider Upstream model
typesafe/jev-latest Native TypeSafe jev-latest
openrouter/typesafe/jev-1.13 OpenRouter typesafe/jev-1.13
openrouter/inception/mercury-decide:free OpenRouter inception/mercury-decide:free
ollama-local/clef-flash Configured local Ollama clef-flash

An unqualified SDK id such as jev-latest retains its OpenRouter default.
An unavailable explicit direct provider does not silently fall back to a gateway.
The legacy alias spelling ~typesafe/jev-latest also retains its OpenRouter route.

Scope

  • Shared decision request validation plus backend-specific limits and transport.
  • Existing provider credential selection, key policy, accounting, outbound guard/proxy
    policy and resilience mechanisms, attributed to the selected connection provider.
  • Native TypeSafe API-key provider and authenticated, zero-inference credential check.
  • decision provider capability and systemone model endpoint. Decision-only models
    cannot execute through chat; mixed chat/decision models keep their chat capability.
  • GET /v1/systemone/models aggregates eligible configured backends. Returned ids
    round-trip into the POST route; prices retain provenance, and a partial backend
    outage does not hide the healthy backends.
  • Dashboard endpoints reuse the existing compact cards with live model counts.
    No provider/status badges, model lists, explanations or request-example panels.

Migration

The original OpenRouter-only version accepted typesafe/jev-1.13 as a vendor id.
That qualified id now selects TypeSafe directly. Use
openrouter/typesafe/jev-1.13 to retain the OpenRouter connection and billing.
The dedicated catalog advertises the new qualified identities.

Related work

Refs #13987. Native TypeSafe work in #14498 and #14031, and the local Ollama adapter
and decision-discovery work in #15410, informed this integration. #15278/#15279's
internal routing clients and Jev-driven combo strategy remain separate concerns;
this PR does not add an internal classifier or a new combo strategy.

Verification

Review corrections are complete. Final regression set: 60 passes; dashboard card/decision-label tests: four passes. Typecheck, changed-file ESLint, provider consistency, complexity, file-size, OpenAPI routes and generated skills checks pass. Translated documentation is complete in follow-up commit 89dc1f45d8: README, API reference and provider reference in all 66 translated locales (198 files), plus 66 provider-count mirrors. Source/target receipts match after formatting and commit hooks; an incremental dry-run reports no outstanding translations for these sources.

Production verification (2026-10-05)

Built the local combined image with this PR and the existing local-only features,
took a data-volume backup (gzip integrity verified), and deployed it with the
previous image retained for rollback. Container healthy, /api/health 200, zero restarts.

  • GET /v1/systemone/models: 200, 13 decision rows, ids prefixed openrouter/,
    owned_by: openrouter, supported_endpoints: [systemone], catalog status complete.
  • POST /v1/systemone, model: openrouter/inception/mercury-decide:free: 200,
    typed Noul answer and exact cost zero (X-OmniRoute-Cost-Status: known).
  • model: typesafe/jev-latest, with no native key configured: rejected instead of
    silently using the available OpenRouter key.
  • Legacy ~typesafe/jev-latest: 200, upstream versioned Jev response and cost returned.
  • General /v1/models: no decision-only rows; GPT-6.1 Sol, Claude Opus 5.5 and ClinePass
    quota still work. Combined branch typecheck, 60 focused tests and 493 full MCP/cache
    Vitest tests pass.
    Previously completed checks: focused decision/provider tests (103 passes), adjacent
    module tests (495 passes), full MCP/autoCombo/cache Vitest suite (493 passes),
    typecheck, changed-file lint, provider consistency, OpenAPI routes, size, complexity,
    canonical documentation gates, generated skills, assets and changelog integrity.

The full Node unit run is not green: 45,232 passes, 11 failures, 53 cancellations
and 27 skips. The added TypeSafe provider's missing golden snapshot was corrected
and its three tests now pass. Four locale-sensitive failures pass in an English
test environment (15 focused tests); four file failures are missing installed
@agentclientprotocol/sdk / istanbul-lib-instrument dependencies. Two assertions
expect the spec reporter but receive TAP. Pending-promise cancellations occur in
untouched timeout/semaphore/auth tests. These files are unchanged from the base;
none of this is being hidden by relaxing assertions or running with no tests.

The full UI configuration is also not entirely green locally (400 files / 2,593 tests
passed; 32 files failed, 17 test failures and two unhandled errors). Backend suites
under jsdom cannot bundle node:sqlite; locale-sensitive formatting assertions pass
when rerun in English (five focused files / 28 tests). FlowCanvas still reports two
existing delayed-fitView teardown errors. The System One compact-card tests pass.

Native TypeSafe and Ollama adapters are contract-tested with mocks. This deployment
has no native TypeSafe credentials and its local Ollama is below the minimum System
One version, so no real native-inference verification is claimed. No credentials,
provider onboarding or Ollama upgrade are part of this change.

Review corrections include native TypeSafe model-only 404 lockout (without claiming
chat/passthrough support), operator pricing overrides, known Ollama image-capability
checks, missing upstream model fallback for accounting, and rejecting empty/array
answer payloads. Image modalities for OpenRouter remain enforced by the upstream
model contract; no public-catalog fetch is added to every inference request. Catalog
enumeration must not mutate account-selection/affinity state per model, and local
credential error markers must be rejected before choosing any upstream host.

Human source contributions acknowledged: native TypeSafe provider/credential-check
material from Sean P. Ford (#14498), and decision-only guarding/Ollama capability
mapping from Praveen K Palaniswamy (#15410). Their PRs remain open; no foreign branch
or worktree is modified by this integration.

⚠️ base-red inherited: #15306. The release branch has documented full-suite failures.
Do not confuse those with this PR's own actionable gate failures; all remaining
limitations will be named in the final verification record.

Remaining global documentation-translation drift is docs/architecture/QUALITY_GATES.md only. Its content and stale translation receipt also exist on the untouched base. This follow-up does not adopt its hash or claim that unrelated drift is resolved.

@PixmaNts

Copy link
Copy Markdown
Contributor Author

Follow-up 95b550fb36 after an internal review of the first commit. These are the fixes for API-key policy, usage accounting and resilience:

  • Connection restriction: credential selection now receives the key's allowedConnections, so a key limited to specific connections cannot draw on another OpenRouter account.
  • Endpoint restriction: new systemone endpoint category. A key limited to models now gets a 403 on both /v1/systemone and /api/v1/systemone (verified live).
  • Model policy: jev-latest and ~typesafe/jev-latest are canonicalized to typesafe/<id> before the policy check, so a single allow/deny rule covers every spelling.
  • Usage and budget: successful calls now write to usage_history (verified live: openrouter, typesafe/jev-1.13-20260917, 282/20 tokens, endpoint=/v1/systemone). The exact usage.cost is charged to the key's budget through recordCost.
  • Cooldown: upstream 401/403/429/5xx now go through markAccountUnavailable, so the connection is put in cooldown. A 422 is the caller's request error and does not penalize the connection.
  • Injection guard: extractMessageContents now includes state and each question's instructions. Before this, the guard scanned nothing on this route.
  • Invalid 200: an empty or non-JSON 200 body now returns a sanitized 502.
  • OpenAPI: request body and BearerAuth are documented.

Tests: tests/unit/systemone-openrouter.test.ts passes 9/9. The endpoint-categories, endpoint-restrictions, api-key-policy, injection-guard and input-sanitizer suites give 161/162. The one failure is the chatCore.ts executor count in the lease inventory, which fails the same way on the base (⚠️ base-red inherited: #14547).

@PixmaNts

Copy link
Copy Markdown
Contributor Author

Follow-up 7ba2294236 — clears the CI failures this PR introduced itself:

  • API / open-sse typecheck: the SystemOneCredentials narrowing in the route (same approach as /v1/rerank) and the typing of questions in extractMessageContents remove the 2 new errors.
  • Complexity: the handler is split into small helpers (parseUpstreamBody, readUsage, upstreamFailure, recordSuccess, successResponse). check:complexity-ratchets now reports 0 new violations. Behavior is unchanged; the 9/9 tests still pass.
  • File size: openapi.generated.ts 1347 → 1357, which is the one generated entry for POST /api/v1/systemone. The rebaseline is documented in file-size-baseline.json, as feat(typesafe): add Jev System One API support #14498 did.
  • Merge integrity: skills/omni-inference/SKILL.md is regenerated by generate-agent-skills --apply. ⚠️ This is an agent-instruction file (AGENTS.md → "Review focus"). The only change is a generated entry documenting POST /api/v1/systemone, derived from docs/openapi.yaml, with no hand-written instruction. Operator approval is required before merging.

Remaining failures are inherited from the base (⚠️ base-red inherited: #14547) and do not touch this PR's files: typecheck errors in auggie.ts / projectCombo.ts / cliproxyAccountHealth.ts, deps (@opencode/plugin), stryker tap.testFiles, dashboard-typecheck, .env.example out of sync, and the unit tests that also fail on #14676.

@diegosouzapw

Copy link
Copy Markdown
Owner

Good, minimal-footprint approach — reusing the existing openrouter connection (same
pattern as the /v1/classify Jina proxy) means zero setup friction for anyone who already
has OpenRouter configured, and the live dev-server test against a real OpenRouter response in
the PR body is strong evidence. I double-checked the err.message interpolation in the
catch block at systemOne.ts:224 against the repo's error-sanitization rule — it's fine,
errorResponse() already routes it through sanitizeErrorMessage(). The real blocker here
isn't code quality, it's that #14498 and #14031 also implement POST /v1/systemone, with
a different credential model (a dedicated TypeSafe provider key vs. this PR's OpenRouter
passthrough). All three write the same route file. Your own "second credential source behind
the same route" idea in the PR description sounds like a reasonable way to combine them —
flagging this for the maintainer to pick a direction across all three PRs.

@diegosouzapw diegosouzapw added the deferred-v3.8.52 Grande demais / suspeito para o lote atual; precisa de sessão dedicada no ciclo v3.8.52 label Sep 28, 2026
@diegosouzapw diegosouzapw changed the title feat(api): proxy System One models (TypeSafe Jev) through OpenRouter [defer] feat(api): proxy System One models (TypeSafe Jev) through OpenRouter Sep 28, 2026
@PixmaNts
PixmaNts force-pushed the feat/systemone-models branch from 7ba2294 to 657c49a Compare September 28, 2026 21:28
@diegosouzapw
diegosouzapw changed the base branch from release/v3.8.51 to release/v3.8.52 September 29, 2026 11:20
@diegosouzapw

Copy link
Copy Markdown
Owner

Re-homed to release/v3.8.52: v3.8.51 entered its release freeze, so the branch now belongs to the release captain and development continues on the next cycle. Nothing is wrong with this PR — it just needed a live base. No action needed from you; CI will re-run against the new base.

@PixmaNts
PixmaNts force-pushed the feat/systemone-models branch from 7427104 to f307c1d Compare September 30, 2026 11:19
@diegosouzapw diegosouzapw removed the deferred-v3.8.52 Grande demais / suspeito para o lote atual; precisa de sessão dedicada no ciclo v3.8.52 label Oct 1, 2026
@diegosouzapw diegosouzapw changed the title [defer] feat(api): proxy System One models (TypeSafe Jev) through OpenRouter feat(api): proxy System One models (TypeSafe Jev) through OpenRouter Oct 1, 2026
System One models (TypeSafe Jev) answer typed noul/choice/score questions
about a state with calibrated probabilities instead of generating text.
OpenRouter serves them at /api/v1/systemone with the TypeSafe request and
response shape, so POST /v1/systemone forwards the body unchanged using the
existing dashboard openrouter credentials: no new provider or key, and the
TypeSafe SDKs work by pointing their base URL at OmniRoute.

The call log records input/output tokens and the exact USD cost OpenRouter
returns in usage.cost; upstream status and retry-after are kept on errors.
Review of the System One proxy found that API-key restrictions did not
apply to it: credential selection ignored the key's allowed connections,
/v1/systemone had no endpoint category (so a key limited to "models" could
still call it), and bare model ids such as jev-latest bypassed deny rules
written for the typesafe/ namespace OpenRouter routes them to.

- register a "systemone" endpoint category and pass allowedConnections plus
  the canonical typesafe/<id> model to credential selection and policy
- record successful calls in usage_history and charge the exact
  usage.cost to the API-key budget ledger
- put the connection into cooldown on upstream 401/403/429/5xx, not on 422
- scan state and question instructions in the prompt-injection guard
- reject an empty or non-JSON 200 body as a sanitized 502
- document the request body and bearer security in the OpenAPI spec
- type the credential union and the question map so the strict api and
  open-sse typechecks no longer report new errors in the systemone route
  and the injection-guard extractor
- split the System One handler into small helpers (parse, usage, failure,
  success) so it adds no new cyclomatic or cognitive complexity
- raise the frozen size of the generated openapi.generated.ts by the one
  endpoint entry the spec now emits (1347 -> 1357)
- regenerate skills/omni-inference/SKILL.md, which the generator derives
  from docs/openapi.yaml and now lists POST /api/v1/systemone
@PixmaNts
PixmaNts force-pushed the feat/systemone-models branch from f307c1d to c489758 Compare October 3, 2026 09:23
PixmaNts and others added 4 commits October 5, 2026 11:00
… the dashboard

GET /v1/systemone/models returns the live OpenRouter decisions catalog
(output_modalities=decisions) with upstream ids, names and pricing, since
those models are absent from the default OpenRouter list and only work
through /v1/systemone. The Endpoints page gets a System One section for the
POST and GET routes, with the 13 new strings translated in all 66 locales.
Route native TypeSafe, OpenRouter and local Ollama decision models through the configured provider connection. Keep gateway-prefixed IDs unambiguous, preserve bare Jev compatibility, and never fall back from a missing direct provider to a gateway. Share typed validation while respecting backend limits, native pricing overrides, read-only catalog eligibility and model-scoped failures.

Co-authored-by: Sean P. Ford <sean@seanford.com>
Co-authored-by: Praveen K Palaniswamy <Praveen.Palaniswamy@gmail.com>
@PixmaNts PixmaNts changed the title feat(api): proxy System One models (TypeSafe Jev) through OpenRouter feat(api): add generic System One decision inference and model discovery Oct 5, 2026
@PixmaNts

PixmaNts commented Oct 5, 2026

Copy link
Copy Markdown
Contributor Author

Combining the TypeSafe, OpenRouter and Ollama work

@diegosouzapw — this PR now addresses the overlap you pointed out between the different System One PRs.

Instead of choosing between TypeSafe and OpenRouter, it lets the same endpoint use either one, or a configured local Ollama server. The model name tells OmniRoute which connection to use:

  • typesafe/jev-latest → TypeSafe directly, with a TypeSafe key.
  • openrouter/typesafe/jev-1.13 → through OpenRouter, with an OpenRouter key.
  • ollama-local/<model> → the configured local Ollama server.

If the requested connection is missing, the request fails rather than being sent somewhere else. Existing calls using jev-latest or ~typesafe/jev-latest still go through OpenRouter.

Credit for the work reused:

I adapted parts of their code; I did not copy their complete commits or merge their PRs. Both authors are credited in commit be2c232e2d. Their PRs remain open and untouched.

The proposal is to support decision models like we support embeddings or image models: providers can offer them alongside chat, or offer only decisions. The model list shows which connection serves each model. Existing key restrictions, cost tracking and error handling still apply. The optional work that uses Jev to choose a chat model is left for separate PRs.

The targeted tests pass, and the OpenRouter path has been tested on a running instance. TypeSafe and Ollama were tested with simulated responses only; I do not have the setup to claim real requests to those two passed. The full test runs still have failures, described in the PR, and translated docs are being finished.

Could this be the shared starting point for the TypeSafe and Ollama PRs? I'd like to agree on the remaining work with their authors and keep everyone credited.

Synchronize README, API and provider references in all 66 translated locales, plus the provider-count mirrors. Preserve source receipts and update hashes after formatting. No production code changes.
@yourspraveen

Copy link
Copy Markdown
Contributor

Thanks @PixmaNts, and thanks for the credit. I'm happy for #14667 to be the shared base. The Ollama pieces from #15410 are all here.

I ran your branch at 89dc1f45d8 against a real Ollama 0.35.1 server, using an ollama-local connection whose baseUrl points at a LAN host (not the default localhost).

Worked as expected (live)

  • ollama-local/nimble and ollama-local/tev1:4b: noul, choice and score questions in one request all return 200. The nimble answers are identical to calling Ollama directly, and cost is recorded as $0.
  • A model that isn't pulled (clef-flash): Ollama's 404 model_not_found passes through, and the lockout is per model. nimble on the same connection keeps working.
  • Validation: data-URL image → 400, stream → 400, more than 64 KiB → 413, invalid keep_alive → 400 ("5m" is accepted), fewer than 2 choice candidates → 400. Ollama's own 400s (for example a question without instructions) pass through.
  • A caller abort doesn't mark the connection. An unreachable server puts the connection into cooldown (network_error), and retries get 429 with retry_after.
  • Chat with a normal model on the same connection (qwen3.5:4b-mlx) still works.

Small points

  • When the model is locked or the connection is cooling down, the client gets 400 "No credentials for provider: ollama-local". /v1/rerank does the same, so this may belong in a separate change, but a 429/503 with the reason would be clearer. I also hit the same 400 once on a request sent about 100 ms after creating the connection; the next request worked.
  • The response's model is the upstream id (nimble), not the requested ollama-local/nimble.

Unit tests only (not live): model discovery, GET /v1/systemone/models, type: "decision" in /v1/models, and the decision-only chat guard. Discovery wouldn't run in my test environment because of a local sandbox limit (spawn EBADF), not the PR. Without it, a chat request to nimble went upstream as expected. The PR's own tests for these parts pass: 57/57 across systemone-{generic,models,routing,review-fixes,openrouter} and ollama-local-capabilities-routing, including "decision-only chat rejects while mixed capability stays selectable".

@diegosouzapw — if you agree with this direction, I'll close #15410 once #14667 lands.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants