Skip to content

fix(cron): refuse cross-vendor model/provider pairs at create/update time - #641

Merged
Kyzcreig merged 1 commit into
mainfrom
fix/cron-vendor-consistency-admission
Aug 29, 2026
Merged

Kyzcreig merged 1 commit into
mainfrom
fix/cron-vendor-consistency-admission

Conversation

@Kyzcreig

Copy link
Copy Markdown
Collaborator

Problem

cronjob(action='create') accepts a model and provider from different vendors. The job is persisted and then fails every fire with:

HTTP 400: the model field names a different vendor model than this endpoint serves; retrying will not help

This is a recurring class, not a one-off — it hit real scheduled jobs on 2026-08-19 and again on 2026-08-29 (model="gpt-5.6-sol" + provider="claude-apr"). Each time it was only noticed after the job had burned failed fires, because nothing rejects the pair at admission time. A post-hoc lint catches it (Rule #20b), which is strictly too late for a job that is dead on arrival.

Change

Promote that post-hoc rule into a create/update-time admission gate in _creation_admission_error(), alongside the existing Rule #5 I1 (sub-hourly deliver=origin) and Rule #4a (bare platform deliver) checks.

  • Conservative and fail-open. Each side's vendor is inferred from a name-prefix map (claude/gpt-/gemini/grok/… → vendor; claude-*/openai*/codex/… → vendor). An unrecognized model or provider name yields vendor None and is not flagged. This catches obvious mis-pairs; it deliberately does not try to maintain a model catalog, so custom/local/proxied names are never blocked.
  • CREATE: the requested pin is checked before create_job() persists anything.
  • UPDATE: the check runs against the effective post-update pair. An update may set only model or only provider, so repointing an openai-pinned job at an anthropic provider is only detectable after merging the update over the stored job.
  • Both stored model shapes handled: a plain model name, and the {"model": ..., "provider": ...} object (whose inner provider is also checked against the effective top-level provider, since either can be the one that routes).
  • The error names both inferred vendors and states that retrying will not help, so the caller fixes the pin rather than re-running a dead job.

Example message:

model 'gpt-5.6-sol' (openai) cannot run on provider 'claude-apr' (anthropic) -- this pair
fails every run with HTTP 400 (the model field names a different vendor model than this
endpoint serves; retrying will not help). Pick a provider that serves this model's vendor.
(cron Rule #20b)

Tests

tests/tools/test_cronjob_tools.py — new TestModelProviderVendorConsistency:

  • predicate: mismatch detected, matching pair clean, unknown side fails open
  • create rejected on a cross-vendor pair (and nothing persisted)
  • create allowed on a matching pair
  • create allowed when the model or the provider is unrecognized
  • update rejected when only the provider changes; also when only the model changes
  • update allowed when the pair stays consistent, and for unrelated-field updates
  • dict model shape: inner provider and top-level provider both checked; consistent pair allowed
=== Summary: 1 files, 130 tests passed, 0 failed (100% complete) in 3.9s (64 workers) ===

Wider cron slice (62 files, 889 passed) has one pre-existing failure,
test_cronjob_run_background.py::test_dispatches_and_returns_handle_immediately — a 5s
threading.Event timeout that reproduces identically on unmodified fork/main under the
same parallel load and passes on both trees when run alone. Not introduced here.

…time

A cron job pinned to a model and provider from different vendors (e.g.
model='gpt-5.6-sol' + provider='claude-apr') is accepted today, persisted,
and then fails EVERY fire with HTTP 400 'the model field names a different
vendor model than this endpoint serves; retrying will not help'. It is a
recurring class — it burned real scheduled jobs on 2026-08-19 and again on
2026-08-29, each time only noticed after the job had already failed fires.

Promote the post-hoc cron-config-lint Rule #20b into a create/update-time
admission gate alongside the existing Rule #5 I1 / #4a checks:

* Infer each side's vendor from a conservative name-prefix map and refuse a
  provably cross-vendor pair. Unknown model or provider names yield vendor
  None and are NOT flagged (fail-open) — this catches obvious mis-pairs, it
  does not maintain a model catalog.
* CREATE: check the requested pin before create_job() persists anything.
* UPDATE: check the EFFECTIVE post-update pair, since an update may supply
  only 'model' or only 'provider' — repointing an openai-pinned job at an
  anthropic provider is only detectable after merging with the stored job.
* Handle both stored model shapes: a plain name, and the {model, provider}
  object (whose inner provider is checked against the effective top-level
  provider too).

The error names both inferred vendors and states retrying will not help, so
the caller can fix the pin instead of re-trying a dead job.

Tests: create rejected/allowed, fail-open on unrecognized names, update
rejected when only the provider (or only the model) changes, update allowed
when the pair stays consistent, and both model shapes.
@Kyzcreig
Kyzcreig force-pushed the fix/cron-vendor-consistency-admission branch from c8ab7d1 to 40ac822 Compare August 29, 2026 13:32
@Kyzcreig
Kyzcreig added this pull request to the merge queue Aug 29, 2026
Merged via the queue into main with commit de200eb Aug 29, 2026
44 checks passed
@Kyzcreig
Kyzcreig deleted the fix/cron-vendor-consistency-admission branch August 29, 2026 13:53
@Kyzcreig

Copy link
Copy Markdown
Collaborator Author

FleetReview

Confidence: 3/5

Findings

  • P2 tools/cronjob_tools.py:722 — Unpausable Jobs
  • P1 tools/cronjob_tools.py:626 — Vendor-prefix map misses slash-qualified model ids, so the guard is inert for the fleet's common naming
  • P1 tools/cronjob_tools.py:669 — Inner model.provider hard-block disagrees with runtime routing

FleetReview provenance · models: B=gpt-5.6-sol, C=claude-code-opus-4-8, D=grok-4.5, F=gpt-5.6-sol, G=grok-4.5 · cost: $6.14 · duration: 30m 36s · rounds: 3 · files examined: 2

@Kyzcreig
Kyzcreig restored the fix/cron-vendor-consistency-admission branch September 21, 2026 10:32
Kyzcreig added a commit that referenced this pull request Sep 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant