Skip to content

feat(providers): onboard Morph as a catalog provider - #1648

Merged
murdore merged 1 commit into
feat/novita-catalogfrom
feat/morph-catalog
Sep 19, 2026
Merged

murdore merged 1 commit into
feat/novita-catalogfrom
feat/morph-catalog

Conversation

@murdore

@murdore murdore commented Sep 6, 2026

Copy link
Copy Markdown
Contributor

Summary

Morph's payment wall cleared. Its recorded wire contract gives it plain chat, SSE streaming and genuinely strong structured output, so it earns a catalog entry — with tool calling deliberately declared false.

Tool calling is false on purpose

Morph answers 200 when sent tools, which is exactly the trap that makes an optimistic flag easy to write. It never returns a tool_calls array:

morph-v3-large  ->  content: "<tool_call>\n{\"name\": \"get_time\", ...}\n</tool_call>"   (literal text)
morph-v3-fast   ->  content: Python source; the tool is ignored

Verified across both models at two token budgets, all four runs finish_reason=stop with no tool_calls. An OpenAI-compatible client would surface that as assistant prose. An earlier campaign note claimed Morph tools worked; it cited models that are no longer in the 22-id roster, so it was stale.

Message content must be a plain string

Array content parts return HTTP 500 Request processing failed: text.charCodeAt is not a function — the same class of quirk cloudflare.json already handles, and expressed the same way. The vision probe fails identically, so vision is not claimed.

A live call proves the fix: it succeeds with no 500, which is only possible if our client sent string content.

Capability flags and their evidence

Flag Value Why
tools / toolsWithStreaming false 200 but no tool_calls, on both models, at two budgets.
structuredOutput true json_schema returned valid schema-conformant JSON; json_object accepted.
structuredOutputWithTools false Meaningless when tools do not work; not claimed on the strength of a 200.
vision not claimed Fails with the array-content 500.

Only the two models the vendor's own error text lists as accepted by chat/completions are catalogued. The other 18 roster ids were never live-verified against chat, so they are omitted rather than invented.

Live proof against the built package

Generate returns ready, streaming yields 1 2 3, a schema call returns {"color":"blue"} with structuredData parsed, and the declared default morph-v3-large is confirmed to be the model that actually resolves.

MORPH_API_KEY is wired into the nightly live matrix. That was missing from the first cut and would have silently excluded Morph from nightly verification.

Gates

check, lint (0 errors), build, catalog codegen drift check (16 providers), test:providers-mocked 98/98, test:provider-structure 3/3, test:openai-compat-catalog 43/43 including the new Morph alias cases, test:error-classifier-contract 44/44, verify:provider-onboarding 9/9, docs:api regenerated. Break-one ritual on the new case: inverted reports a failure with a non-zero exit, restored passes.

The catalog alias suite is hand-maintained rather than derived from the catalog, so Morph was silently absent from it at first; the cases were added and the suite re-run.

@coderabbitai

coderabbitai Bot commented Sep 6, 2026 •

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository: juspay/neurolink/.coderabbit.yaml

Review profile: CHILL

Plan: Advanced

Run ID: 014ea6cf-6ca3-4da4-bac1-33e10d76edb2

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Sep 6, 2026 •

Copy link
Copy Markdown
Contributor

✅ Single Commit Policy - COMPLIANT

Status: Policy requirements met • 1 commit • Valid format • Ready for merge

📊 View validation details

📝 Commit Details

  • Hash: 6c2595e73036a48766e5c9c36886b6ca34022724
  • Message: feat(providers): onboard Morph as a catalog provider
  • Author: Sachin Sharma

✅ Validation Results

  • Single commit requirement met
  • No merge commits in branch
  • Semantic commit message format verified
  • Ready for squash merge to release branch

🤖 Automated validation by NeuroLink Single Commit Enforcement

@github-actions

github-actions Bot commented Sep 6, 2026 •

Copy link
Copy Markdown
Contributor

Documentation Validation Results

🚀 Documentation validation passed!

Check Status Result
Frontmatter Validation ✅ Passed
TypeScript Check ✅ Passed
Build ✅ Passed
Link Validation ✅ Passed

📦 Build artifact uploaded successfully. Ready for deployment preview.

Commit: 7549fd1b2460d89ef11142ce81ae424083b9c99f | Workflow: View logs

@Tara-ag Tara-ag left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review: Morph catalog onboarding

Verdict: APPROVE — clean catalog-only onboarding; the two "traps" (tool calling 200-with-no-tool_calls, array content → 500) are both correctly reflected as data flags that the existing catalog machinery genuinely enforces (supportsTools off keeps tools off the wire; messageContentFormat:"string" is consumed by the loader + ConfiguredOpenAICompatProvider.adjustRequestBody), with test coverage and nightly env wiring included.

Severity File:line Finding
MINOR src/lib/providers/catalog/morph.json:12 morph-v3-fast is a code-edit specialist that the PR's own evidence says returns unrelated text when used for chat; it is the secondary fallback/fallbackModelName

Checked and clean

  • messageContentFormat:"string" quirk → loader.ts reads entry.quirks.messageContentFormat → ConfiguredOpenAICompatProvider.adjustRequestBody (src/lib/providers/configuredOpenAICompat.ts). Same path as Cloudflare. ✓
  • tools:false / toolsWithStreaming:false → loader.ts maps capabilities.tools → supportsTools; executeStream/generate gate on supportsTools() so no tools array ever reaches the Morph wire. ✓
  • errorRules 401/400 with {model}/{apiKeyEnvVar}/{setupUrl} placeholders → buildErrorRules interpolates them. ✓
  • Credential/env resolution: catalogEnvVar("morph", …) → MORPH_API_KEY/MORPH_BASE_URL/MORPH_MODEL, matching the nightly MORPH_API_KEY wiring. ✓
  • AIProviderName.MORPH + NeurolinkCredentials.morph? + CatalogProviderName/CatalogCredentialKey are all additive (rule 5 backward-compatible); no hand registration needed beyond catalog + registry loop.
  • addedInPR:"pending" matches the convention for pre-merge catalog entries (mancer, gmicloud, io-intelligence …). Not a finding.
  • Generated index.generated.ts / providerCatalog.generated.ts + docs/api/** are codegen/CI-owned (skipped per repo conventions).
  • Alias tests for morph/morphllm in the openai-compat catalog suite match the catalog (URL api.morphllm.com/v1, default morph-v3-large).

Impact-analysis note: I could not run the code-review-graph blast-radius tools in this environment (the local checkout is not indexed and the sandbox blocked filesystem access outside .yama/). Blast radius was instead assessed from the diff plus remote source inspection via the GitHub API. The change is a purely additive catalog/enum registration touching only the generic catalog+registry machinery, so no out-of-diff callers are expected to break.

Minor doc nit (not blocking): the PR body states all four tool probes ended finish_reason=stop, but morph.json's evidence.liveMatrix.result records morph-v3-fast returning finish_reason=length. Worth reconciling so the evidence stays internally consistent.

@Tara-ag Tara-ag left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor design note on the Morph fallback chain; onboarding otherwise looks correct (verified quirks/capabilities/error-rules wiring against the catalog loader and OpenAI-compat provider code).

Comment thread src/lib/providers/catalog/morph.json

@Tara-ag Tara-ag left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving per the full review in the summary comment (<!-- yama:summary -->): clean catalog-only onboarding with the two mechanics flags correctly enforced and test/CI coverage included. The single non-blocking MINOR finding (morph-v3-fast as the chat fallback in morph.json:12) is tracked in the inline thread and does not block merge.

@Tara-ag Tara-ag left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Recurring review — Morph catalog onboarding

Verdict: APPROVE — re-confirmed on current head 364c6a6. The head commit SHA moved since the last review (cbee555 → 364c6a6) but the diff content is identical; the catalog-only onboarding remains clean, additive, and the two "trap" flags (tools:false, messageContentFormat:"string") are genuinely enforced by the catalog loader + ConfiguredOpenAICompatProvider machinery. No new findings.

Severity File:line Finding Status
MINOR src/lib/providers/catalog/morph.json:12 morph-v3-fast is a code-edit specialist used as the secondary chat fallback — a general-chat request can get unrelated output when the primary model is unavailable Retained (inline thread)
MINOR PR body / morph.json evidence.liveMatrix.result PR body says all four tool probes ended finish_reason=stop, but the evidence records morph-v3-fast at finish_reason=length; reconcile for internal consistency New (doc nit, non-blocking)

On the retained finding: the inline thread (marker yama:morph-fallback) has no author reply, and the evidence in morph.json still confirms morph-v3-fast "ignores the tools parameter and returns unrelated generated text" and is "purpose-built for applying code edits rather than general chat". So it is kept as a MINOR design note, not elevated — the fallback only engages on invalid-model / discovery-empty paths, defaults resolve to morph-v3-large.

Checked and re-verified clean on this pass

  • tools:false / toolsWithStreaming:false → supportsTools() gate keeps tools off the Morph wire. ✓
  • messageContentFormat:"string" → adjustRequestBody string-content path (same quirk as Cloudflare). ✓
  • Additive surface only: AIProviderName.MORPH, NeurolinkCredentials.morph?, CatalogProviderName/CatalogCredentialKey, MorphModels — no exhaustive switch / hand-registration breakage; detect_changes risk 0.00, no affected flows. ✓
  • errorRules 401/400 placeholders + credential env resolution → MORPH_API_KEY/MORPH_BASE_URL/MORPH_MODEL. ✓
  • Alias tests for morph/morphllm match the catalog (URL api.morphllm.com/v1, default morph-v3-large); nightly MORPH_API_KEY wired. ✓
  • docs/api/**, llms-full.txt, index.generated.ts, providerCatalog.generated.ts are codegen/CI-owned (skipped per repo conventions). ✓
  • Check runs on the current head include the review agent; prior gates (test:providers-mocked, test:provider-structure, catalog codegen drift, 43/43 alias suite) plus Single Commit Policy and Generated artifacts are current pass.

@murdore

murdore commented Sep 11, 2026

Copy link
Copy Markdown
Contributor Author

Live acceptance run — SDK and CLI, real vendor

Run from this PR's worktree at head 364c6a64d, dist/ built from that head (0 source files newer), public surface only.

provider=morph  model=morph-v3-large
node=v24.14.1

SDK generate  -> 1547ms  content="HELLO"
              provider=morph model=morph-v3-large usage={"input":104,"output":3,"total":107}
SDK stream    -> 6028ms  chunks=5  text="1\n2\n3"
CLI generate  -> 1947ms  exit=0  stdout="HELLO"
CLI stream    -> 6864ms  exit=0  stdout="HELLO"

SUMMARY sdk.generate=true sdk.stream=true

All four surfaces pass.

And a correction to my own earlier comment on the fallback thread

While running this I also probed the no-model path, and it contradicts what I claimed in #1648 (comment). Details are on that thread; the short version is that the default model resolves to morph-v3-large, not morph-v3-fast:

NO-MODEL generate -> resolved model=morph-v3-large
                    content="HELLO"

I had read loader.ts's fallbackModelName ?? fallbacks[1] ?? fallbacks[0] and concluded that an ordinary call lands on the code-edit specialist. It does not — that line feeds getFallbackModelName(), which is a different accessor from getDefaultModel(), and morph.json sets "default": "morph-v3-large". Reading the derivation was not the same as running it.

murdore added a commit that referenced this pull request Sep 12, 2026
`loader.ts` derives a provider's fallback positionally —
`fallbackModelName ?? fallbacks[1] ?? fallbacks[0]` — and nothing ever
checked the result. When that derivation lands on the provider's own
default, the retry path is silently dead: the primary fails, the SDK
retries the identical model, and it fails again for the same reason.

Auditing all 16 shipped catalogs found three in that state, so this is
not a hypothetical:

| catalog        | models | fallback == default | avoidable |
| -------------- | ------ | ------------------- | --------- |
| gmicloud       | 1      | yes                 | no        |
| inception-labs | 1      | yes                 | no        |
| mistral        | 32     | yes                 | YES       |

`mistral.json` ships 32 models and explicitly pinned `fallbackModelName`
to its own default, overriding a perfectly serviceable `fallbacks[1]`
(`mistral-large-latest`). Removing that one line restores a real
fallback. The two single-model catalogs have no alternative to name and
are exempt by construction — the check only fires when the catalog
declares more than one model, so it reports a mistake rather than an
unavoidable fact.

The rule lives in the schema's existing `superRefine`, beside the checks
that already constrain `default` and `fallbacks` to catalog keys, so it
runs wherever catalogs are parsed. That is both required CI jobs:
`test` via `codegen:catalog -- --check` (ci.yml:221) and
`provider-safety-net` via `verify:provider-onboarding` (ci.yml:440),
which imports `parseProviderCatalogJson` directly.

Before, on the unmodified tree:

    Error: Invalid provider catalog file mistral.json:
    models.fallbackModelName: the fallback model resolves to
    "mistral-small-2506", which is also models.default — a fallback
    identical to the primary cannot fall back.
    exit 1

After:

    codegen:catalog complete — 15 providers
    verify:provider-onboarding — 8/8 new providers fully onboarded
    test:openai-compat-catalog — 41 passed · 0 failed
    exit 0

No generated artifacts change: the fallback is resolved at load time,
not emitted into a generated file.

Found while investigating a review thread on #1648 which suspected
Morph's fallback. Morph turned out to be fine — its fallback differs
from its default — and the real defect was in a provider nobody was
looking at, reached by checking the derivation itself instead.
murdore added a commit that referenced this pull request Sep 12, 2026
`loader.ts` derives a provider's fallback positionally —
`fallbackModelName ?? fallbacks[1] ?? fallbacks[0]` — and nothing ever
checked the result. When that derivation lands on the provider's own
default, the retry path is silently dead: the primary fails, the SDK
retries the identical model, and it fails again for the same reason.

Auditing all 16 shipped catalogs found three in that state, so this is
not a hypothetical:

| catalog        | models | fallback == default | avoidable |
| -------------- | ------ | ------------------- | --------- |
| gmicloud       | 1      | yes                 | no        |
| inception-labs | 1      | yes                 | no        |
| mistral        | 32     | yes                 | YES       |

`mistral.json` ships 32 models and explicitly pinned `fallbackModelName`
to its own default, overriding a perfectly serviceable `fallbacks[1]`
(`mistral-large-latest`). Removing that one line restores a real
fallback. The two single-model catalogs have no alternative to name and
are exempt by construction — the check only fires when the catalog
declares more than one model, so it reports a mistake rather than an
unavoidable fact.

The rule lives in the schema's existing `superRefine`, beside the checks
that already constrain `default` and `fallbacks` to catalog keys, so it
runs wherever catalogs are parsed. That is both required CI jobs:
`test` via `codegen:catalog -- --check` (ci.yml:221) and
`provider-safety-net` via `verify:provider-onboarding` (ci.yml:440),
which imports `parseProviderCatalogJson` directly.

Before, on the unmodified tree:

    Error: Invalid provider catalog file mistral.json:
    models.fallbackModelName: the fallback model resolves to
    "mistral-small-2506", which is also models.default — a fallback
    identical to the primary cannot fall back.
    exit 1

After:

    codegen:catalog complete — 15 providers
    verify:provider-onboarding — 8/8 new providers fully onboarded
    test:openai-compat-catalog — 41 passed · 0 failed
    exit 0

No generated artifacts change: the fallback is resolved at load time,
not emitted into a generated file.

Found while investigating a review thread on #1648 which suspected
Morph's fallback. Morph turned out to be fine — its fallback differs
from its default — and the real defect was in a provider nobody was
looking at, reached by checking the derivation itself instead.

@Tara-ag Tara-ag left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed Morph catalog onboarding. Clean PR; one MINOR follow-up on the test suite's env-var neutralization. Details in the summary comment.

Comment thread test/continuous-test-suite-openai-compat-catalog.ts
@Tara-ag

Tara-ag commented Sep 12, 2026

Copy link
Copy Markdown
Contributor

Verdict: APPROVE

Morph is a well-evidenced catalog onboarding. Capability flags are grounded in live wire probes (tools deliberately false for both models, structuredOutput true, vision not claimed), the catalog reuses the existing messageContentFormat: "string" quirk exactly as cloudflare.json does, and all generated surfaces (index.generated.ts, providerCatalog.generated.ts, MorphModels, AIProviderName.MORPH) are regenerated and drift-consistent. Provider count gate, lint, build and all listed test suites are green.

Severity File:line Finding
MINOR test/continuous-test-suite-openai-compat-catalog.ts:1109 MORPH_API_KEY added to the alias checks but missing from CATALOG_ENV_VARS neutralize list — breaks the suite's documented "no real creds in mocked tests" invariant

Existing thread — accepted (resolved)

  • yama:morph-fallback (morph.json:12): the author's follow-up investigation is thorough and conclusive. No catalog-only fix exists (trimming fallbacks to [morph-v3-large] fails the new fix(catalog): reject a fallback model that cannot fall back #1690 cross-catalog validation; marking morph-v3-fast retired would be false; deleting it removes a legitimately selectable model; and fallbackModelName: morph-v3-large would disable falling back entirely). The live probe also confirmed the default resolves to morph-v3-large, not the specialist. Leaving morph.json as-is is justified — not reposted, thread resolved.

Checked and found clean

  • morph.json structure vs cloudflare.json: identical quirks/capabilities/errorRules/setup/evidence convention; enumMember correctly omitted (model ids auto-derive to valid identifiers).
  • Generated wiring consistent: CATALOG_PROVIDER_IDS, CatalogProviderName, CatalogCredentialKey all include morph.
  • Alias routing tests for morph / morphllm added using MORPH_API_KEY and the correct host.
  • live-matrix.yml wires MORPH_API_KEY (the missing-from-first-cut item the PR calls out).
  • Blast radius: additive provider + enum; no existing SDK surface modified (rule 5 safe); providers.ts changes are line-shift churn only.

No CRITICAL / MAJOR findings.

@Tara-ag Tara-ag left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving per the full review in the summary comment (<!-- yama:summary -->). Clean additive catalog onboarding; the two non-blocking MINOR findings (morph-v3-fast as chat fallback — author-justified, thread resolved — and the MORPH_* env-var neutralize gap) do not block merge.

Comment thread test/continuous-test-suite-openai-compat-catalog.ts
@Tara-ag

Tara-ag commented Sep 12, 2026

Copy link
Copy Markdown
Contributor

Review: Morph catalog onboarding — final

Verdict: APPROVE — clean, additive catalog-only onboarding; both "trap" flags (tools:false, messageContentFormat:"string") are genuinely enforced by the existing catalog machinery, and the two findings are non-blocking (one author-justified, one MINOR).

Severity File:line Finding Status
MINOR test/continuous-test-suite-openai-compat-catalog.ts:1109 MORPH_* env vars added to the alias checks but not to the CATALOG_ENV_VARS neutralize list — latent hermeticity gap Inline thread (posted this review)
MINOR src/lib/providers/catalog/morph.json:12 morph-v3-fast is a code-edit specialist; it is the secondary fallback/fallbackModelName Retained, accepted — author proved no catalog-only fix exists (inline thread, resolved)

Checked and clean

  • morph.json follows the cloudflare.json convention (quirks.messageContentFormat:"string", same capabilities/setup/errorRules/evidence blocks); enumMember correctly omitted (identifiers auto-derive). ✓
  • Generated wiring drift-free: CATALOG_PROVIDER_IDS, CatalogProviderName, CatalogCredentialKey (morph), AIProviderName.MORPH, MorphModels. ✓
  • Rule 5 backward-compat safe: purely additive surface, no exhaustive-switch or hand-registration breakage; no out-of-diff callers affected (additive provider + enum only). ✓
  • Alias tests for morph/morphllm match the catalog (URL api.morphllm.com/v1, default morph-v3-large); nightly MORPH_API_KEY wired in live-matrix.yml. ✓
  • docs/api/**, index.generated.ts, providerCatalog.generated.ts are codegen/CI-owned (skipped per repo conventions). ✓

Validation note: blast radius assessed from the diff + remote source inspection (code-graph tools unavailable in this sandbox). Change is additive and touches only generic catalog/enum machinery, so no breakage is expected.

Note on summaries: this is the single canonical summary for the PR (issue comment). Earlier identical summaries embedded in two review bodies on older commits (cbee555, 364c6a6) are superseded by this one.

murdore added a commit that referenced this pull request Sep 13, 2026
`loader.ts` derives a provider's fallback positionally —
`fallbackModelName ?? fallbacks[1] ?? fallbacks[0]` — and nothing ever
checked the result. When that derivation lands on the provider's own
default, the retry path is silently dead: the primary fails, the SDK
retries the identical model, and it fails again for the same reason.

Auditing all 16 shipped catalogs found three in that state, so this is
not a hypothetical:

| catalog        | models | fallback == default | avoidable |
| -------------- | ------ | ------------------- | --------- |
| gmicloud       | 1      | yes                 | no        |
| inception-labs | 1      | yes                 | no        |
| mistral        | 32     | yes                 | YES       |

`mistral.json` ships 32 models and explicitly pinned `fallbackModelName`
to its own default, overriding a perfectly serviceable `fallbacks[1]`
(`mistral-large-latest`). Removing that one line restores a real
fallback. The two single-model catalogs have no alternative to name and
are exempt by construction — the check only fires when the catalog
declares more than one model, so it reports a mistake rather than an
unavoidable fact.

The rule lives in the schema's existing `superRefine`, beside the checks
that already constrain `default` and `fallbacks` to catalog keys, so it
runs wherever catalogs are parsed. That is both required CI jobs:
`test` via `codegen:catalog -- --check` (ci.yml:221) and
`provider-safety-net` via `verify:provider-onboarding` (ci.yml:440),
which imports `parseProviderCatalogJson` directly.

Before, on the unmodified tree:

    Error: Invalid provider catalog file mistral.json:
    models.fallbackModelName: the fallback model resolves to
    "mistral-small-2506", which is also models.default — a fallback
    identical to the primary cannot fall back.
    exit 1

After:

    codegen:catalog complete — 15 providers
    verify:provider-onboarding — 8/8 new providers fully onboarded
    test:openai-compat-catalog — 41 passed · 0 failed
    exit 0

No generated artifacts change: the fallback is resolved at load time,
not emitted into a generated file.

Found while investigating a review thread on #1648 which suspected
Morph's fallback. Morph turned out to be fine — its fallback differs
from its default — and the real defect was in a provider nobody was
looking at, reached by checking the derivation itself instead.
@murdore
murdore changed the base branch from release to feat/novita-catalog September 13, 2026 22:12
@murdore
murdore force-pushed the feat/novita-catalog branch from 63a8a3c to d3d6557 Compare September 19, 2026 01:20
Morph's payment wall cleared. The recorded wire contract gives it plain
chat, SSE streaming and genuinely strong structured output (json_schema
came back as valid schema-conformant JSON), so it earns a catalog entry.

Tool calling is declared false on purpose. Morph answers 200 when sent
tools but never returns a tool_calls array: morph-v3-large emits the call
as literal text in message.content and morph-v3-fast ignores tools and
returns Python. Verified across both models at two token budgets. An
earlier campaign note claiming Morph tools worked cited models that are
no longer in its roster.

Message content must be a plain string: array parts return HTTP 500
"text.charCodeAt is not a function". That is expressed the same way
cloudflare.json expresses the identical quirk, and a live call proves our
client now avoids it.

Only the two models the vendor's own error text lists as accepted by
chat/completions are catalogued. MORPH_API_KEY is wired into the nightly
live matrix.
@murdore
murdore merged commit 9c2f0c5 into feat/novita-catalog Sep 19, 2026
22 checks passed
@murdore
murdore deleted the feat/morph-catalog branch September 19, 2026 04:54
@murdore

murdore commented Sep 19, 2026

Copy link
Copy Markdown
Contributor Author

Follow-up checked and closed out — nothing to carry to #1728.

The remaining item was Tara-ag's doc-consistency note, raised twice (once in the first review, once in the "Recurring review" table as New (doc nit, non-blocking)) and then dropped from the final approval summary: the PR body says all four tool-calling probes ended finish_reason=stop, while src/lib/providers/catalog/morph.json:93 records morph-v3-fast returning unrelated generated text at finish_reason=length.

Checked both at the pinned head. They do contradict — but morph.json holds the accurate record, and it is the artifact that survives the merge; the PR body is the part that is wrong, and it disappears. capabilities.toolCalling: false is consistent with the evidence text either way, so the catalog entry needs no change.

Recording it here so the discrepancy is on file rather than silently dropped. #1728 carries only the three items that describe durable code.

murdore added a commit that referenced this pull request Sep 19, 2026
Two providers in one commit because #1648 (Morph) was merged into this
branch rather than into release, which left feat/novita-catalog carrying
both and failing the single-commit policy. The content is unchanged from
the two reviewed commits; only their messages are joined.

Novita

Novita's payment wall cleared, so its wire contract could finally be
recorded end to end: 156 models, chat, SSE streaming with a usage chunk,
real structured tool calls, json_object and json_schema, and the 401/404
shapes the error rules need. The entry is written from that recording.

The declared default is zai-org/glm-5.3-flash rather than a plain instruct
model. meta-llama/llama-3.3-70b-instruct returns HTTP 400 for BOTH
response_format json_schema ("Supported formats: json_object") and
json_object, and the catalog schema can only express structuredOutput per
provider, so making it the default would have declared a capability the
default caller could never use. It stays as a fallback for plain chat.

Vision is not claimed: the vision probe returns 200 with an empty reply.

Live proof against the built package: generate and streaming on the
default model, a schema call returning parsed structuredData, and tool
calling verified at the wire on both models (finish_reason=tool_calls).
NOVITA_API_KEY is wired into the nightly live matrix so the entry keeps
being checked.

Morph

Morph's payment wall cleared. The recorded wire contract gives it plain
chat, SSE streaming and genuinely strong structured output (json_schema
came back as valid schema-conformant JSON), so it earns a catalog entry.

Tool calling is declared false on purpose. Morph answers 200 when sent
tools but never returns a tool_calls array: morph-v3-large emits the call
as literal text in message.content and morph-v3-fast ignores tools and
returns Python. Verified across both models at two token budgets. An
earlier campaign note claiming Morph tools worked cited models that are
no longer in its roster.

Message content must be a plain string: array parts return HTTP 500
"text.charCodeAt is not a function". That is expressed the same way
cloudflare.json expresses the identical quirk, and a live call proves our
client now avoids it.

Only the two models the vendor's own error text lists as accepted by
chat/completions are catalogued. MORPH_API_KEY is wired into the nightly
live matrix.
murdore added a commit that referenced this pull request Sep 24, 2026
`loader.ts` derives a provider's fallback positionally —
`fallbackModelName ?? fallbacks[1] ?? fallbacks[0]` — and nothing ever
checked the result. When that derivation lands on the provider's own
default, the retry path is silently dead: the primary fails, the SDK
retries the identical model, and it fails again for the same reason.

Auditing all 16 shipped catalogs found three in that state, so this is
not a hypothetical:

| catalog        | models | fallback == default | avoidable |
| -------------- | ------ | ------------------- | --------- |
| gmicloud       | 1      | yes                 | no        |
| inception-labs | 1      | yes                 | no        |
| mistral        | 32     | yes                 | YES       |

`mistral.json` ships 32 models and explicitly pinned `fallbackModelName`
to its own default, overriding a perfectly serviceable `fallbacks[1]`
(`mistral-large-latest`). Removing that one line restores a real
fallback. The two single-model catalogs have no alternative to name and
are exempt by construction — the check only fires when the catalog
declares more than one model, so it reports a mistake rather than an
unavoidable fact.

The rule lives in the schema's existing `superRefine`, beside the checks
that already constrain `default` and `fallbacks` to catalog keys, so it
runs wherever catalogs are parsed. That is both required CI jobs:
`test` via `codegen:catalog -- --check` (ci.yml:221) and
`provider-safety-net` via `verify:provider-onboarding` (ci.yml:440),
which imports `parseProviderCatalogJson` directly.

Before, on the unmodified tree:

    Error: Invalid provider catalog file mistral.json:
    models.fallbackModelName: the fallback model resolves to
    "mistral-small-2506", which is also models.default — a fallback
    identical to the primary cannot fall back.
    exit 1

After:

    codegen:catalog complete — 15 providers
    verify:provider-onboarding — 8/8 new providers fully onboarded
    test:openai-compat-catalog — 41 passed · 0 failed
    exit 0

No generated artifacts change: the fallback is resolved at load time,
not emitted into a generated file.

Found while investigating a review thread on #1648 which suspected
Morph's fallback. Morph turned out to be fine — its fallback differs
from its default — and the real defect was in a provider nobody was
looking at, reached by checking the derivation itself instead.
murdore added a commit that referenced this pull request Sep 27, 2026
…e catalog docs

Three catalog providers merged without any user-facing documentation:
FriendliAI (#1657), Morph (#1648) and Novita (#1647) each have a catalog
JSON file and an AIProviderName member, but no page under
docs/getting-started/providers/ and no entry in the README or provider
index. Each now has a setup page built only from its catalog entry (base
URL, key env var, models, capabilities), wired into the provider index,
the README list and the docs-site sidebar.

The migration ledger added in #1764 still listed mistral, deepseek and
huggingface as pending, although #1781 moved all three to the JSON
catalog and removed them from HAND_DESCRIPTORS. They now sit under a
Migrated section with the schema fields each one needed.

The tier-2 authoring guide described quirks as "two escape hatches" while
the schema has five, including replayReasoningContent from #1800. Every
quirk is now documented with what it does and when to use it.

Provider counts disagreed across the docs: most pages said 40, the README
said "the other 23" above a longer list, and the enum has 44 members
(excluding auto). Counts now match the code: 44 providers, 27 in the
README's secondary list, and 31 native / 3 model-dependent / 10 no-tool
support, which sums to 44.
murdore added a commit that referenced this pull request Sep 27, 2026
…e catalog docs

Three catalog providers merged without any user-facing documentation:
FriendliAI (#1657), Morph (#1648) and Novita (#1647) each have a catalog
JSON file and an AIProviderName member, but no page under
docs/getting-started/providers/ and no entry in the README or provider
index. Each now has a setup page built only from its catalog entry (base
URL, key env var, models, capabilities), wired into the provider index,
the README list and the docs-site sidebar.

The migration ledger added in #1764 still listed mistral, deepseek and
huggingface as pending, although #1781 moved all three to the JSON
catalog and removed them from HAND_DESCRIPTORS. They now sit under a
Migrated section with the schema fields each one needed.

The tier-2 authoring guide described quirks as "two escape hatches" while
the schema has five, including replayReasoningContent from #1800. Every
quirk is now documented with what it does and when to use it.

Provider counts disagreed across the docs: most pages said 40, the README
said "the other 23" above a longer list, and the enum has 44 members
(excluding auto). Counts now match the code: 44 providers, 27 in the
README's secondary list, and 31 native / 3 model-dependent / 10 no-tool
support, which sums to 44.
murdore added a commit that referenced this pull request Sep 27, 2026
…e catalog docs

Three catalog providers merged without any user-facing documentation:
FriendliAI (#1657), Morph (#1648) and Novita (#1647) each have a catalog
JSON file and an AIProviderName member, but no page under
docs/getting-started/providers/ and no entry in the README or provider
index. Each now has a setup page built only from its catalog entry (base
URL, key env var, models, capabilities), wired into the provider index,
the README list and the docs-site sidebar.

The migration ledger added in #1764 still listed mistral, deepseek and
huggingface as pending, although #1781 moved all three to the JSON
catalog and removed them from HAND_DESCRIPTORS. They now sit under a
Migrated section with the schema fields each one needed.

The tier-2 authoring guide described quirks as "two escape hatches" while
the schema has five, including replayReasoningContent from #1800. Every
quirk is now documented with what it does and when to use it.

Provider counts disagreed across the docs: most pages said 40, the README
said "the other 23" above a longer list, and the enum has 44 members
(excluding auto). Counts now match the code: 44 providers, 27 in the
README's secondary list, and 31 native / 3 model-dependent / 10 no-tool
support, which sums to 44.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants