Repository navigation
feat(OMN-20372): add paid and local model entries to pricing manifest - #4480
Conversation
Add paid hosted entries and no-cost local entries to the pricing manifest; no existing entry was changed. Each paid entry carries evidence metadata citing provider pricing documentation, marked low confidence. A unit test covers the new entries. The manifest becomes the sole pricing authority; downstream cost accounting tables will be pointed at it in later pull requests.
Pull request was closed
…12394) * evidence(OMN-20372): author OCC companion for OmniNode-ai/omnibase_infra#4480 OCC companion by node_pr_lifecycle_fix_effect (OMN-13317 F1 / OMN-13990 / OMN-14285). Product PR head b1aa250bfde20a8ebce459531685ea2d34e248cf. * evidence(OMN-20372): self-bind OCC#12394 + rebind contract_sha256 --------- Co-authored-by: omnimarket-bot <bot@omninode.ai>
There was a problem hiding this comment.
Hostile Reviewer — adversarial findings (OMN-17492)
Models succeeded: qwen3-review, gpt-oss-review
Models failed: none
New finding threads: 0
Deduped (already posted on this PR): 0
Below quorum (one model only, reported not threaded): 8
Nit-level findings suppressed: 0
The model is the FINDER, never the gate: merge is gated only by the
deterministic Hostile Review Thread Gate, which blocks while
hostile-reviewer threads are unresolved. Resolve each thread after
addressing (or rejecting, with a reply) its finding.
Below quorum: 8 finding(s) raised by one model only (OMN-18479)
These are reported and NOT dropped, but they get no thread and do not block: a single model's finding no other model reproduced is not evidence enough to stop a merge. Read them; act on them if they are right.
- [MAJOR]
tests/unit/configs/test_pricing_manifest_fill_omn20372.py(qwen3-review) — Test asserts retrieved_at equals effective_date, but they are not semantically equivalent | The test test_paid_models_have_documented_rates asserts that raw_entry["evidence"]["retrieved_at"] == raw_en - [MAJOR]
tests/unit/configs/test_pricing_manifest_fill_omn20372.py(qwen3-review) — No test validates that local model entries have the expected evidence structure | The test test_local_models_have_zero_api_cost checks that input and output costs are zero and that the source is LOCAL - [MAJOR]
tests/unit/configs/test_pricing_manifest_fill_omn20372.py(qwen3-review) — Cost estimate test does not account for potential rounding or truncation in estimate_cost | The test test_haiku_cost_estimate_uses_the_documented_rate asserts that estimate_cost("claude-haiku-4-5", 10 - [MAJOR]
src/omnibase_infra/configs/pricing_manifest.yaml(gpt-oss-review) — Missing evidence block for local model entries | The newly added local model entries (e.g., qwen3.8, Qwen3.8-27B, Qwen3-Coder-30B, DeepSeek-R1-14B) omit the required "evidence" section that other part - [MINOR]
tests/unit/configs/test_pricing_manifest_fill_omn20372.py(qwen3-review) — No test verifies that the YAML manifest is valid YAML and parseable by the loader | The tests rely on ModelPricingTable.from_yaml() and yaml.safe_load to parse the manifest. If the YAML has a syntax e - [MINOR]
src/omnibase_infra/configs/pricing_manifest.yaml(qwen3-review) — Model IDs use inconsistent casing and naming conventions | The manifest introduces model IDs with mixed casing: qwen3.8 (lowercase), Qwen3.8-27B (PascalCase), Qwen3-Coder-30B (PascalCase), DeepSeek-R1 - [MINOR]
tests/unit/configs/test_pricing_manifest_fill_omn20372.py(qwen3-review) — No test validates that the cost estimate for a local model is zero | The test suite verifies that local model entries have zero costs in the YAML, but does not test that ModelPricingTable.estimate_cos - [MINOR]
tests/unit/configs/test_pricing_manifest_fill_omn20372.py(gpt-oss-review) — Assumption about return type of estimate_cost in test | The testtest_haiku_cost_estimate_uses_the_documented_rateaccessesestimate.estimated_cost_usd. IfModelPricingTable.estimate_costreturn
✅ Hostile Reviewer — REVIEWEDCritical findings: 0
Semantics (OMN-17492 — the model finds, thread resolution gates)
Powered by omniintelligence.review_pairing.cli_review — multi-model adversarial review: qwen3-review (Qwen3.8-27B), gpt-oss-review (gpt-oss-120b) (OMN-8468/OMN-8524/OMN-17492) |
|
Lab: host=h201 lane=pricing-one-2a21 head=b1aa250bfde20a8ebce459531685ea2d34e248cf command="uv run pytest tests/ci/test_pricing_manifest_schema.py tests/unit/configs tests/unit/models/pricing tests/unit/nodes/node_savings_estimation_compute -q -n 0" observed="157 passed"; scripts/lint_pricing_manifest.py exit 0 |
Closing and reopening the PR at 2026-10-02T18:57:58Z ran the five close-only workflows (Auto-Tag on Merge, TODO Audit on Merge, Contractor Integration Note, PR Merged Event Producer, Runtime Rebuild Trigger). They concluded skipped on b1aa250, and CI Summary (run 37050987347) fails closed on those skipped rows. None of them runs on a push, so a new head does not carry them. No file changes.
There was a problem hiding this comment.
Hostile Reviewer — adversarial findings (OMN-17492)
Models succeeded: qwen3-review, gpt-oss-review
Models failed: none
New finding threads: 4
Deduped (already posted on this PR): 0
Below quorum (one model only, reported not threaded): 6
Nit-level findings suppressed: 0
The model is the FINDER, never the gate: merge is gated only by the
deterministic Hostile Review Thread Gate, which blocks while
hostile-reviewer threads are unresolved. Resolve each thread after
addressing (or rejecting, with a reply) its finding.
Below quorum: 6 finding(s) raised by one model only (OMN-18479)
These are reported and NOT dropped, but they get no thread and do not block: a single model's finding no other model reproduced is not evidence enough to stop a merge. Read them; act on them if they are right.
- [MAJOR]
tests/unit/configs/test_pricing_manifest_fill_omn20372.py(qwen3-review) — Floating-point equality assertion on YAML-parsed values is fragile | The test assertsentry.input_cost_per_1k == pytest.approx(input_cost_per_1k)after loading viaModelPricingTable.from_yaml(). I - [MAJOR]
tests/unit/configs/test_pricing_manifest_fill_omn20372.py(qwen3-review) — Test does not verify that local models are absent from paid-model lookup paths | The test confirms local models have zero cost, but does not verify that the pricing table does not accidentally treat t - [MAJOR]
src/omnibase_infra/configs/pricing_manifest.yaml(qwen3-review) — Model ID casing inconsistency between qwen3.8 and Qwen3.8-27B | The manifest defines bothqwen3.8(lowercase) andQwen3.8-27B(uppercase Q). If the pricing table lookup is case-sensitive, these ar - [MINOR]
tests/unit/configs/test_pricing_manifest_fill_omn20372.py(qwen3-review) — No test verifies that unknown models raise or return a sentinel, not a silent default | The test suite validates that specific models are present and priced correctly, but does not test the behavior f - [MINOR]
tests/unit/configs/test_pricing_manifest_fill_omn20372.py(qwen3-review) — Test reads the manifest file twice: once via ModelPricingTable, once via raw yaml.safe_load | Each test function callsModelPricingTable.from_yaml()and then separately calls `yaml.safe_load(_DEFAUL - [MINOR]
src/omnibase_infra/configs/pricing_manifest.yaml(gpt-oss-review) — External source URLs are not validated for safety | The manifest now includes several HTTPS URLs from third‑party providers. The code that consumes these URLs does not perform any validation or saniti
Findings demoted from threads (anchor rejected)
-
[MINOR] hostile-reviewer (qwen3-review)
Duplicated evidence block structure across all entries is a maintenance hazard | Every new entry repeats the same
evidencesub-structure with identicalgenerated_at,sample_window_days,sample_count,min_samples,query, andauthoritativefields. This is a data duplication that will drift over time. If the query string or min_samples threshold changes, all 13 entries must be updated in lockstep. A schema-level default or a shared evidence template would reduce the surface area for inconsistencyResolve this thread when addressed — the
Hostile Review Thread Gateblocks while hostile-reviewer threads are unresolved (OMN-17492). -
[MINOR] hostile-reviewer (qwen3-review)
estimate_cost test uses 1000 input and 1000 output tokens, but does not verify the formula | The test asserts
estimate.estimated_cost_usd == pytest.approx(0.006)for 1000 input and 1000 output tokens at $0.001/1k input and $0.005/1k output. The expected value is 1000/1000 * 0.001 + 1000/1000 * 0.005 = 0.001 + 0.005 = 0.006. This is correct. However, the test does not verify the formula for non-1000-token counts. Ifestimate_costhas an off-by-one in the token-to-1k conversion (e.g., using integer divisiResolve this thread when addressed — the
Hostile Review Thread Gateblocks while hostile-reviewer threads are unresolved (OMN-17492). -
[MINOR] hostile-reviewer (gpt-oss-review)
Local model entries omit source_url in evidence block | The newly added local model entries provide an evidence section that lacks a "source_url" field, whereas paid model entries include this field. Downstream code that assumes the presence of a URL may raise a KeyError or skip validation, breaking the manifest processing pipeline. | Evidence: Local model entry for "qwen3.8" includes evidence with keys generated_at, sample_window_days, sample_count, min_samples, query, authoritative but no "source_url". |
Resolve this thread when addressed — the
Hostile Review Thread Gateblocks while hostile-reviewer threads are unresolved (OMN-17492). -
[MINOR] hostile-reviewer (gpt-oss-review)
Test reliance on default manifest path may be brittle | The new unit tests call ModelPricingTable.from_yaml() which reads the manifest from a hard‑coded default path. If the test runner's working directory differs from the repository root, the path resolution could fail, causing false negatives in CI environments. | Evidence: tests/unit/configs/test_pricing_manifest_fill_omn20372.py imports _DEFAULT_MANIFEST_PATH and uses it without adjusting for relative execution context. | Fix: Modify the test to locate
Resolve this thread when addressed — the
Hostile Review Thread Gateblocks while hostile-reviewer threads are unresolved (OMN-17492).
There was a problem hiding this comment.
Hostile Reviewer — adversarial findings (OMN-17492)
Models succeeded: qwen3-review, gpt-oss-review
Models failed: none
New finding threads: 0
Deduped (already posted on this PR): 0
Below quorum (one model only, reported not threaded): 9
Nit-level findings suppressed: 2
The model is the FINDER, never the gate: merge is gated only by the
deterministic Hostile Review Thread Gate, which blocks while
hostile-reviewer threads are unresolved. Resolve each thread after
addressing (or rejecting, with a reply) its finding.
Below quorum: 9 finding(s) raised by one model only (OMN-18479)
These are reported and NOT dropped, but they get no thread and do not block: a single model's finding no other model reproduced is not evidence enough to stop a merge. Read them; act on them if they are right.
- [MAJOR]
tests/unit/configs/test_pricing_manifest_fill_omn20372.py:48-49(qwen3-review) — Floating-point equality assertion on parsed YAML values | The test test_paid_models_have_documented_rates asserts entry.input_cost_per_1k == pytest.approx(input_cost_per_1k) where the expected value i - [MAJOR]
src/omnibase_infra/configs/pricing_manifest.yaml:430-460(qwen3-review) — No test for model ID case-sensitivity or normalization | The manifest contains both 'qwen3.8' (lowercase) and 'Qwen3.8-27B' (mixed case) as distinct keys. The test parametrizes both and asserts they r - [MAJOR]
tests/unit/configs/test_pricing_manifest_fill_omn20372.py:60-64(qwen3-review) — estimate_cost test does not verify input/output token attribution | test_haiku_cost_estimate_uses_the_documented_rate calls table.estimate_cost('claude-haiku-4-5', 1000, 1000) and asserts the total is - [MINOR]
src/omnibase_infra/configs/pricing_manifest.yaml:384-395(qwen3-review) — Duplicate sample_count field at two nesting levels | Each new manifest entry has sample_count: 0 at the top level of the model entry and again inside evidence.sample_count: 0. The test only checks the - [MINOR]
tests/unit/configs/test_pricing_manifest_fill_omn20372.py:33(qwen3-review) — No validation that source_url is reachable or well-formed beyond https prefix | The test asserts raw_entry['evidence']['source_url'].startswith('https://') but does not validate that the URL is a well - [MINOR]
tests/unit/configs/test_pricing_manifest_fill_omn20372.py:25-35(qwen3-review) — Test loads and parses the full YAML manifest on every parametrized invocation | Each parametrized test case calls ModelPricingTable.from_yaml() and yaml.safe_load(_DEFAULT_MANIFEST_PATH.read_text(...) - [MINOR]
src/omnibase_infra/configs/pricing_manifest.yaml(gpt-oss-review) — Low confidence entries added without validation | The diff adds multiple model pricing entries marked with confidence "LOW_CONFIDENCE" and source "FALLBACK_PROVIDER_DOCUMENTATION". Downstream componen - [MINOR]
tests/unit/configs/test_pricing_manifest_fill_omn20372.py(gpt-oss-review) — Test suite loads full manifest for each parametrized case | Each parametrized test reads the entire pricing manifest from disk, causing repeated I/O and YAML parsing, which can degrade test suite perf - [MINOR]
tests/unit/configs/test_pricing_manifest_fill_omn20372.py(gpt-oss-review) — Assumption that retrieved_at equals effective_date may be brittle | The test asserts that "retrieved_at" matches "effective_date" for each new entry. If future updates record a different retrieval dat
Summary
Ticket OMN-20372, lane step 1 of 3: fill the omnibase_infra price list
src/omnibase_infra/configs/pricing_manifest.yaml, the single pricing authority per theoperator-approved direction. Steps 2 (omniclaude
cost_accounting.py) and 3 (omnimarketcost_pricingandpricing.py) are separate PRs and wait for a released omnibase_infracarrying these entries; the newest tag, v0.38.61, does not carry them.
Changes
claude-haiku-4-5: 0.001 / 0.005claude-sonnet-4-6: 0.003 / 0.015claude-opus-4-5: 0.005 / 0.025gemini-2.5-flash: 0.0003 / 0.0025gemini-2.5-flash-lite: 0.0001 / 0.0004evidence.source_urlandretrieved_at, and is markedLOW_CONFIDENCE / FALLBACK_PROVIDER_DOCUMENTATION.
qwen3.8,Qwen3.8-27B,Qwen3-Coder-30B,DeepSeek-R1-14B— the local ids omnimarketcost_pricing.yamlprices.tests/unit/configs/test_pricing_manifest_fill_omn20372.py.Verification
Run locally:
scripts/lint_pricing_manifest.pypasses.tests/ci/test_pricing_manifest_schema.py,tests/unit/configs,tests/unit/models/pricing, andtests/unit/nodes/node_savings_estimation_compute:157 passed.
ruff checkandruff formatclean.Follow-ups
Found during this work, not changed here:
claude-opus-4-6is 0.015 / 0.075 per 1k, but the Anthropic pricing pagelists Opus 4.6 at $5 input / $25 output per MTok (0.005 / 0.025 per 1k); omniclaude
cost_accounting.pycarries the same 15/75.claude-haiku-4-5at $0.80 / $4.00 per MTok; the page says$1 / $5.
cost_pricing.yamlpricesgemini-2.5-flashat 0.10 / 0.40 per MTok andgemini-2.5-flash-liteat 0.075 / 0.30; the Google page says 0.30 / 2.50 and 0.10 / 0.40.Pointing those tables at the manifest in step 3 changes those figures.
qwen3.6-35b-a3bdiffers in case from omnimarketQwen3.6-35B-A3B;get_entryis exact-match.Not in scope: the CI guard against new price literals (OMN-19980, Lakshman).
Evidence-Ticket: OMN-20372
Evidence-Source: OCC#12394
Lab pool readback (OMN-18893)