Repository navigation
test(rail): boundary coupling measurement script and multi-format provider-journey contract (rail 3.8.55) - #16205
Merged
diegosouzapw merged 2 commits intoOct 10, 2026
Conversation
…cript scripts/quality/measure-boundary.mjs scans src/**/*.ts(x) and open-sse/**/*.ts(x) (skipping node_modules, .next, tests/, __tests__/ and *.test/*.spec files), lexes out comments/strings/regex literals, extracts static, re-export, side-effect, dynamic import() and require() specifiers, resolves @/, @omniroute/open-sse and relative paths, and counts unique cross-boundary file edges in both directions, grouped by subsystem of origin. Emits --json or --md, deterministically ordered, plus the full open-sse -> src edge list as the v4 remediation input. Exposed as `npm run quality:boundary` and registered in the gate manifest as separately-invoked (report only, not a gate). Rail 3.8.55, Task 13.
…ites The #8330 journey covers one OpenAI-compatible provider and stops at /v1/models. Two companion suites extend it to other wire formats and add the legs that only break across module boundaries, against a fake upstream on an ephemeral 127.0.0.1 port (no external network): - provider-journey-claude: anthropic-compatible node -> connection -> REAL model sync (sync route -> /models route -> upstream /models) -> /v1/models publishes <prefix>/<model> with owned_by = prefix -> /v1/messages (JSON and SSE) reaches the upstream as an Anthropic call for the raw model id with the connection key, and the client gets a Claude message / Anthropic event stream. - provider-journey-gemini: openai-compatible node, same sync/catalog legs, then /v1beta/models/<prefix>/<model>:generateContent is translated to an OpenAI Chat Completions upstream call and back to candidates[]. A TODO step records a gap found on the way: /v1beta/models still names custom-provider models by the raw provider-node UUID (#8327 residue on the Gemini catalog surface). Rail 3.8.55, Task 13.
This was referenced Oct 10, 2026
Closed
This was referenced Oct 10, 2026
Contributor
CI Coverage Report
Coverage artifact was not available for this run. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Rail 3.8.55, Task 13: boundary remeasurement and an expanded provider-journey E2E, ahead of the v4 split of
open-sse/(engine) fromsrc/lib/*(modules). No production code changes.scripts/quality/measure-boundary.mjs(npm run quality:boundary): a reproducible, deterministic measurement of src ⇄ open-sse coupling, replacing the hand-built tables from the characterization tasks.provider-journey.contract.test.ts(test(e2e): contract test for the full provider journey #8330). One covers the Claude format, one covers the Gemini client format. Both run against a fake upstream on an ephemeral 127.0.0.1 port, with no external network.What changed
scripts/quality/measure-boundary.mjssrc/**/*.ts(x)andopen-sse/**/*.ts(x)and skipsnode_modules,.next,tests/,__tests__/and*.test/*.specfiles. A small lexer strips comments, string contents, template text and regex literals. The script then extracts static imports, re-exports, side-effect imports,import()calls andrequire()calls. It resolves@/,@omniroute/open-sse[/…]and relative paths, including../../src/...escapes. Edges are counted as unique (importing file → imported file) pairs, grouped by subsystem of origin (src/lib/<name>,src/app/api/<route>,src/<dir>,open-sse/<dir>). Output is--jsonor--md. The JSON also includesopenSseToSrcEdges, the full engine→app list, which is the v4 remediation input. Onlynode:fs,node:pathandnode:urlare used; there is no child process and no shell.tests/unit/measure-boundary.test.ts@omniroute/open-sseimport, a relative escape,import(),require(), type-only imports, commented-out imports, test/__tests__/node_modules/.nextexclusion, dedupe, subsystem aggregation, determinism, Markdown output, and the CLI.package.json,config/quality/gate-manifest.jsonquality:boundaryalias, registered asseparately-invoked. It is a report only, not a gate.tests/integration/_providerJourneyHarness.ts/modelsroute, so the real sync path runs.tests/integration/provider-journey-claude.contract.test.ts/modelsreceivesx-api-key) →/v1/modelspublishes<prefix>/<model>withowned_by= prefix and never the UUID (#8327) →POST /v1/messages, both JSON and SSE. The upstream receives/v1/messageswith the raw model id, the connection key, and the system and user turns. The client receives a Claudemessageor an Anthropic event stream, with no OpenAI shapes.tests/integration/provider-journey-gemini.contract.test.ts/v1/modelsowned_by= prefix →/v1beta/models/<prefix>/<model>:generateContent. The upstream receives/v1/chat/completionswith the raw id and the system and user messages, and nocontents. The client receivescandidates[0].content.parts[0].textwithfinishReason: STOP./v1beta/modelslists the model.Both suites are collected by the existing
tests/integration/*.test.tsglob (test:integration,test:integration:ci).Measurement at the release tip (
cc84e9426a)The engine→app direction is far from zero. The source subsystems are
open-sse/services(284),handlers(258),executors(63),mcp-server(59),utils(50),config(16) andtranslator(15). The top targets aresrc/shared(266),src/lib/db(146),src/lib/usageDb(45),src/lib/usage(37),src/lib/providers(33),src/domain(26) andsrc/sse(21). The full per-edge list is the v4 remediation list.Cross-check against the Task 12 hand tables: files importing open-sse match exactly (memory 21, guardrails 7, skills 7, a2a 3, oauth 18). Distinct open-sse consumers also match exactly (memory 5, guardrails 3, skills 12, oauth 6). The only difference is a2a: the script finds 0 open-sse consumers against Task 12's 1, because that consumer is a vitest file, and test files are excluded here by design.
Findings
/v1beta/modelsnames custom-provider modelsmodels/<raw provider-node UUID>/<model>, which is the identifier that fix(models): /v1/models exposes internal provider UUID as namespace, ignores configured prefix, multiplies effort variants #8327 removed from/v1/models. The Gemini suite records this as atodostep, so it shows up in every run without turning CI red. It needs a follow-up fix insrc/app/api/v1beta/models/route.ts.GET /v1beta/modelshas no auth. The handler takes no request, sorequireAuthForModelsis not applied to the Gemini catalog the way it is to/v1/models. Reported here for triage; no test pins it.geminiexecutor ignores a per-connection base URL (case "gemini"inopen-sse/executors/default.ts), and provider nodes are only openai- or anthropic-compatible. A fake Gemini-format upstream therefore cannot be wired without mocking fetch, so the Gemini journey uses a Gemini client against an OpenAI-compatible upstream.Validation
node --import tsx/esm --test tests/unit/measure-boundary.test.tsnode --import tsx/esm --import ./open-sse/utils/setupPolyfill.ts --import ./tests/_setup/isolateDataDir.ts --test --test-force-exit --test-concurrency=1 tests/integration/provider-journey{,-claude,-gemini}.contract.test.ts tests/unit/measure-boundary.test.tsRUN_CONTRACT_INT), 1 todo/v1betagenerate routenpx eslint <changed files>npm run typecheck:corenpm run check:dead-codenpm run check:test-discoverynode scripts/quality/gate-manifest.mjs