Skip to content

feat(OMN-20372): add paid and local model entries to pricing manifest - #4480

Merged
jonahgabriel merged 2 commits into
devfrom
jonah/omn-20372-pricing-manifest-fill
Oct 2, 2026
Merged

jonahgabriel merged 2 commits into
devfrom
jonah/omn-20372-pricing-manifest-fill

Conversation

@jonahgabriel

@jonahgabriel jonahgabriel commented Oct 2, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Ticket OMN-20372, lane step 1 of 3: fill the omnibase_infra price list
src/omnibase_infra/configs/pricing_manifest.yaml, the single pricing authority per the
operator-approved direction. Steps 2 (omniclaude cost_accounting.py) and 3 (omnimarket
cost_pricing and pricing.py) are separate PRs and wait for a released omnibase_infra
carrying these entries; the newest tag, v0.38.61, does not carry them.

Changes

  • Added paid entries (USD per 1k tokens, input/output), with source and date:
  • Added zero-cost local entries: qwen3.8, Qwen3.8-27B, Qwen3-Coder-30B,
    DeepSeek-R1-14B — the local ids omnimarket cost_pricing.yaml prices.
  • No existing entry was changed.
  • New test: tests/unit/configs/test_pricing_manifest_fill_omn20372.py.

Verification

Run locally:

  • scripts/lint_pricing_manifest.py passes.
  • pytest on tests/ci/test_pricing_manifest_schema.py, tests/unit/configs,
    tests/unit/models/pricing, and tests/unit/nodes/node_savings_estimation_compute:
    157 passed.
  • ruff check and ruff format clean.
  • pre-commit passed on commit.

Follow-ups

Found during this work, not changed here:

  • (a) Existing claude-opus-4-6 is 0.015 / 0.075 per 1k, but the Anthropic pricing page
    lists Opus 4.6 at $5 input / $25 output per MTok (0.005 / 0.025 per 1k); omniclaude
    cost_accounting.py carries the same 15/75.
  • (b) omniclaude prices claude-haiku-4-5 at $0.80 / $4.00 per MTok; the page says
    $1 / $5.
  • (c) omnimarket cost_pricing.yaml prices gemini-2.5-flash at 0.10 / 0.40 per MTok and
    gemini-2.5-flash-lite at 0.075 / 0.30; the Google page says 0.30 / 2.50 and 0.10 / 0.40.
    Pointing those tables at the manifest in step 3 changes those figures.
  • (d) Manifest key qwen3.6-35b-a3b differs in case from omnimarket Qwen3.6-35B-A3B;
    get_entry is exact-match.

Not in scope: the CI guard against new price literals (OMN-19980, Lakshman).

Evidence-Ticket: OMN-20372
Evidence-Source: OCC#12394

Lab pool readback (OMN-18893)

LAB PROOF PASS: omnibase_infra#4480 head 86ca01b559 on lab-101 (192.168.86.101), isolated project omnibase-infra-local, 2026-10-02T19:59:31Z to 2026-10-02T20:11:26Z
  ok   head_matches
  ok   stack_built
  ok   image_identity
  ok   health_8085
  ok   health_8086
  ok   migration_gate_healthy
  ok   no_wiring_failures
  ok   focused_tests
  restored=yes (containers=0 volumes=0 networks=0 images=0 listeners=0 workdir=gone, snapshot-diff=0)
  method: scripts/runtime_build/prepr_runtime_pool.py (OMN-18893)
  receipt OmniNode-ai/omnibase_infra#4480@86ca01b559d7fc0075fbc1a6a926a2fc3bb8cac3:omnibase_infra.runtime_and_scripts@1 result=PASS verifier=MISSING_MANDATORY_CHECK (mandatory passing check(s) missing: ['changed_path_live', 'forwarder_refused_topic']) file=/Users/jonah/.local/state/omni/landing/lab/workers/000059/lab-proof-receipts/omnibase_infra-4480-86ca01b559d7fc0075fbc1a6a926a2fc3bb8cac3.receipt.json event=/Users/jonah/.local/state/omni/landing/lab/workers/000059/lab-proof-receipts/omnibase_infra-4480-86ca01b559d7fc0075fbc1a6a926a2fc3bb8cac3.event.json
  ledger: RELEASE re=<your HOLD id> surface=101-runtime result=PASS restored=yes

Add paid hosted entries and no-cost local entries to the pricing
manifest; no existing entry was changed. Each paid entry carries
evidence metadata citing provider pricing documentation, marked low
confidence. A unit test covers the new entries.

The manifest becomes the sole pricing authority; downstream cost
accounting tables will be pointed at it in later pull requests.
Comment thread .github/workflows/ci.yml Fixed
Comment thread .github/workflows/ci.yml Fixed
@jonahgabriel
jonahgabriel changed the base branch from main to dev October 2, 2026 18:52
@jonahgabriel
jonahgabriel enabled auto-merge October 2, 2026 18:56
auto-merge was automatically disabled October 2, 2026 18:57

Pull request was closed

@jonahgabriel jonahgabriel reopened this Oct 2, 2026
jonahgabriel pushed a commit to OmniNode-ai/onex_change_control that referenced this pull request Oct 2, 2026
…12394)

* evidence(OMN-20372): author OCC companion for OmniNode-ai/omnibase_infra#4480

OCC companion by node_pr_lifecycle_fix_effect (OMN-13317 F1 / OMN-13990 / OMN-14285). Product PR head b1aa250bfde20a8ebce459531685ea2d34e248cf.

* evidence(OMN-20372): self-bind OCC#12394 + rebind contract_sha256

---------

Co-authored-by: omnimarket-bot <bot@omninode.ai>

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hostile Reviewer — adversarial findings (OMN-17492)

Models succeeded: qwen3-review, gpt-oss-review
Models failed: none
New finding threads: 0
Deduped (already posted on this PR): 0
Below quorum (one model only, reported not threaded): 8
Nit-level findings suppressed: 0

The model is the FINDER, never the gate: merge is gated only by the
deterministic Hostile Review Thread Gate, which blocks while
hostile-reviewer threads are unresolved. Resolve each thread after
addressing (or rejecting, with a reply) its finding.

Below quorum: 8 finding(s) raised by one model only (OMN-18479)

These are reported and NOT dropped, but they get no thread and do not block: a single model's finding no other model reproduced is not evidence enough to stop a merge. Read them; act on them if they are right.

  • [MAJOR] tests/unit/configs/test_pricing_manifest_fill_omn20372.py (qwen3-review) — Test asserts retrieved_at equals effective_date, but they are not semantically equivalent | The test test_paid_models_have_documented_rates asserts that raw_entry["evidence"]["retrieved_at"] == raw_en
  • [MAJOR] tests/unit/configs/test_pricing_manifest_fill_omn20372.py (qwen3-review) — No test validates that local model entries have the expected evidence structure | The test test_local_models_have_zero_api_cost checks that input and output costs are zero and that the source is LOCAL
  • [MAJOR] tests/unit/configs/test_pricing_manifest_fill_omn20372.py (qwen3-review) — Cost estimate test does not account for potential rounding or truncation in estimate_cost | The test test_haiku_cost_estimate_uses_the_documented_rate asserts that estimate_cost("claude-haiku-4-5", 10
  • [MAJOR] src/omnibase_infra/configs/pricing_manifest.yaml (gpt-oss-review) — Missing evidence block for local model entries | The newly added local model entries (e.g., qwen3.8, Qwen3.8-27B, Qwen3-Coder-30B, DeepSeek-R1-14B) omit the required "evidence" section that other part
  • [MINOR] tests/unit/configs/test_pricing_manifest_fill_omn20372.py (qwen3-review) — No test verifies that the YAML manifest is valid YAML and parseable by the loader | The tests rely on ModelPricingTable.from_yaml() and yaml.safe_load to parse the manifest. If the YAML has a syntax e
  • [MINOR] src/omnibase_infra/configs/pricing_manifest.yaml (qwen3-review) — Model IDs use inconsistent casing and naming conventions | The manifest introduces model IDs with mixed casing: qwen3.8 (lowercase), Qwen3.8-27B (PascalCase), Qwen3-Coder-30B (PascalCase), DeepSeek-R1
  • [MINOR] tests/unit/configs/test_pricing_manifest_fill_omn20372.py (qwen3-review) — No test validates that the cost estimate for a local model is zero | The test suite verifies that local model entries have zero costs in the YAML, but does not test that ModelPricingTable.estimate_cos
  • [MINOR] tests/unit/configs/test_pricing_manifest_fill_omn20372.py (gpt-oss-review) — Assumption about return type of estimate_cost in test | The test test_haiku_cost_estimate_uses_the_documented_rate accesses estimate.estimated_cost_usd. If ModelPricingTable.estimate_cost return

@github-actions

github-actions Bot commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

✅ Hostile Reviewer — REVIEWED

Critical findings: 0
Major findings: 3
Total findings: 11
Models succeeded: qwen3-review,gpt-oss-review
Models unavailable: none

Action required: findings were posted as review threads. Address or reject each one, then resolve its thread — the Hostile Review Thread Gate fails while hostile-reviewer threads are unresolved (OMN-17492).


Semantics (OMN-17492 — the model finds, thread resolution gates)

Surface Meaning Blocks merge?
Review threads Per-finding, posted by the reviewer No (informational)
Hostile Review Thread Gate Deterministic: unresolved hostile-reviewer threads exist Fails until resolved (not yet a required context)
degraded verdict Fewer than 2 models succeeded (infra) Fails this job with a named reason

Powered by omniintelligence.review_pairing.cli_review — multi-model adversarial review: qwen3-review (Qwen3.8-27B), gpt-oss-review (gpt-oss-120b) (OMN-8468/OMN-8524/OMN-17492)

@jonahgabriel

Copy link
Copy Markdown
Collaborator Author

Lab: host=h201 lane=pricing-one-2a21 head=b1aa250bfde20a8ebce459531685ea2d34e248cf command="uv run pytest tests/ci/test_pricing_manifest_schema.py tests/unit/configs tests/unit/models/pricing tests/unit/nodes/node_savings_estimation_compute -q -n 0" observed="157 passed"; scripts/lint_pricing_manifest.py exit 0

Closing and reopening the PR at 2026-10-02T18:57:58Z ran the five
close-only workflows (Auto-Tag on Merge, TODO Audit on Merge, Contractor
Integration Note, PR Merged Event Producer, Runtime Rebuild Trigger). They
concluded skipped on b1aa250, and CI Summary (run 37050987347) fails
closed on those skipped rows. None of them runs on a push, so a new head
does not carry them. No file changes.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hostile Reviewer — adversarial findings (OMN-17492)

Models succeeded: qwen3-review, gpt-oss-review
Models failed: none
New finding threads: 4
Deduped (already posted on this PR): 0
Below quorum (one model only, reported not threaded): 6
Nit-level findings suppressed: 0

The model is the FINDER, never the gate: merge is gated only by the
deterministic Hostile Review Thread Gate, which blocks while
hostile-reviewer threads are unresolved. Resolve each thread after
addressing (or rejecting, with a reply) its finding.

Below quorum: 6 finding(s) raised by one model only (OMN-18479)

These are reported and NOT dropped, but they get no thread and do not block: a single model's finding no other model reproduced is not evidence enough to stop a merge. Read them; act on them if they are right.

  • [MAJOR] tests/unit/configs/test_pricing_manifest_fill_omn20372.py (qwen3-review) — Floating-point equality assertion on YAML-parsed values is fragile | The test asserts entry.input_cost_per_1k == pytest.approx(input_cost_per_1k) after loading via ModelPricingTable.from_yaml(). I
  • [MAJOR] tests/unit/configs/test_pricing_manifest_fill_omn20372.py (qwen3-review) — Test does not verify that local models are absent from paid-model lookup paths | The test confirms local models have zero cost, but does not verify that the pricing table does not accidentally treat t
  • [MAJOR] src/omnibase_infra/configs/pricing_manifest.yaml (qwen3-review) — Model ID casing inconsistency between qwen3.8 and Qwen3.8-27B | The manifest defines both qwen3.8 (lowercase) and Qwen3.8-27B (uppercase Q). If the pricing table lookup is case-sensitive, these ar
  • [MINOR] tests/unit/configs/test_pricing_manifest_fill_omn20372.py (qwen3-review) — No test verifies that unknown models raise or return a sentinel, not a silent default | The test suite validates that specific models are present and priced correctly, but does not test the behavior f
  • [MINOR] tests/unit/configs/test_pricing_manifest_fill_omn20372.py (qwen3-review) — Test reads the manifest file twice: once via ModelPricingTable, once via raw yaml.safe_load | Each test function calls ModelPricingTable.from_yaml() and then separately calls `yaml.safe_load(_DEFAUL
  • [MINOR] src/omnibase_infra/configs/pricing_manifest.yaml (gpt-oss-review) — External source URLs are not validated for safety | The manifest now includes several HTTPS URLs from third‑party providers. The code that consumes these URLs does not perform any validation or saniti

Findings demoted from threads (anchor rejected)

  • [MINOR] hostile-reviewer (qwen3-review)

    Duplicated evidence block structure across all entries is a maintenance hazard | Every new entry repeats the same evidence sub-structure with identical generated_at, sample_window_days, sample_count, min_samples, query, and authoritative fields. This is a data duplication that will drift over time. If the query string or min_samples threshold changes, all 13 entries must be updated in lockstep. A schema-level default or a shared evidence template would reduce the surface area for inconsistency

    Resolve this thread when addressed — the Hostile Review Thread Gate blocks while hostile-reviewer threads are unresolved (OMN-17492).

  • [MINOR] hostile-reviewer (qwen3-review)

    estimate_cost test uses 1000 input and 1000 output tokens, but does not verify the formula | The test asserts estimate.estimated_cost_usd == pytest.approx(0.006) for 1000 input and 1000 output tokens at $0.001/1k input and $0.005/1k output. The expected value is 1000/1000 * 0.001 + 1000/1000 * 0.005 = 0.001 + 0.005 = 0.006. This is correct. However, the test does not verify the formula for non-1000-token counts. If estimate_cost has an off-by-one in the token-to-1k conversion (e.g., using integer divisi

    Resolve this thread when addressed — the Hostile Review Thread Gate blocks while hostile-reviewer threads are unresolved (OMN-17492).

  • [MINOR] hostile-reviewer (gpt-oss-review)

    Local model entries omit source_url in evidence block | The newly added local model entries provide an evidence section that lacks a "source_url" field, whereas paid model entries include this field. Downstream code that assumes the presence of a URL may raise a KeyError or skip validation, breaking the manifest processing pipeline. | Evidence: Local model entry for "qwen3.8" includes evidence with keys generated_at, sample_window_days, sample_count, min_samples, query, authoritative but no "source_url". |

    Resolve this thread when addressed — the Hostile Review Thread Gate blocks while hostile-reviewer threads are unresolved (OMN-17492).

  • [MINOR] hostile-reviewer (gpt-oss-review)

    Test reliance on default manifest path may be brittle | The new unit tests call ModelPricingTable.from_yaml() which reads the manifest from a hard‑coded default path. If the test runner's working directory differs from the repository root, the path resolution could fail, causing false negatives in CI environments. | Evidence: tests/unit/configs/test_pricing_manifest_fill_omn20372.py imports _DEFAULT_MANIFEST_PATH and uses it without adjusting for relative execution context. | Fix: Modify the test to locate

    Resolve this thread when addressed — the Hostile Review Thread Gate blocks while hostile-reviewer threads are unresolved (OMN-17492).

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hostile Reviewer — adversarial findings (OMN-17492)

Models succeeded: qwen3-review, gpt-oss-review
Models failed: none
New finding threads: 0
Deduped (already posted on this PR): 0
Below quorum (one model only, reported not threaded): 9
Nit-level findings suppressed: 2

The model is the FINDER, never the gate: merge is gated only by the
deterministic Hostile Review Thread Gate, which blocks while
hostile-reviewer threads are unresolved. Resolve each thread after
addressing (or rejecting, with a reply) its finding.

Below quorum: 9 finding(s) raised by one model only (OMN-18479)

These are reported and NOT dropped, but they get no thread and do not block: a single model's finding no other model reproduced is not evidence enough to stop a merge. Read them; act on them if they are right.

  • [MAJOR] tests/unit/configs/test_pricing_manifest_fill_omn20372.py:48-49 (qwen3-review) — Floating-point equality assertion on parsed YAML values | The test test_paid_models_have_documented_rates asserts entry.input_cost_per_1k == pytest.approx(input_cost_per_1k) where the expected value i
  • [MAJOR] src/omnibase_infra/configs/pricing_manifest.yaml:430-460 (qwen3-review) — No test for model ID case-sensitivity or normalization | The manifest contains both 'qwen3.8' (lowercase) and 'Qwen3.8-27B' (mixed case) as distinct keys. The test parametrizes both and asserts they r
  • [MAJOR] tests/unit/configs/test_pricing_manifest_fill_omn20372.py:60-64 (qwen3-review) — estimate_cost test does not verify input/output token attribution | test_haiku_cost_estimate_uses_the_documented_rate calls table.estimate_cost('claude-haiku-4-5', 1000, 1000) and asserts the total is
  • [MINOR] src/omnibase_infra/configs/pricing_manifest.yaml:384-395 (qwen3-review) — Duplicate sample_count field at two nesting levels | Each new manifest entry has sample_count: 0 at the top level of the model entry and again inside evidence.sample_count: 0. The test only checks the
  • [MINOR] tests/unit/configs/test_pricing_manifest_fill_omn20372.py:33 (qwen3-review) — No validation that source_url is reachable or well-formed beyond https prefix | The test asserts raw_entry['evidence']['source_url'].startswith('https://') but does not validate that the URL is a well
  • [MINOR] tests/unit/configs/test_pricing_manifest_fill_omn20372.py:25-35 (qwen3-review) — Test loads and parses the full YAML manifest on every parametrized invocation | Each parametrized test case calls ModelPricingTable.from_yaml() and yaml.safe_load(_DEFAULT_MANIFEST_PATH.read_text(...)
  • [MINOR] src/omnibase_infra/configs/pricing_manifest.yaml (gpt-oss-review) — Low confidence entries added without validation | The diff adds multiple model pricing entries marked with confidence "LOW_CONFIDENCE" and source "FALLBACK_PROVIDER_DOCUMENTATION". Downstream componen
  • [MINOR] tests/unit/configs/test_pricing_manifest_fill_omn20372.py (gpt-oss-review) — Test suite loads full manifest for each parametrized case | Each parametrized test reads the entire pricing manifest from disk, causing repeated I/O and YAML parsing, which can degrade test suite perf
  • [MINOR] tests/unit/configs/test_pricing_manifest_fill_omn20372.py (gpt-oss-review) — Assumption that retrieved_at equals effective_date may be brittle | The test asserts that "retrieved_at" matches "effective_date" for each new entry. If future updates record a different retrieval dat

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants