test(blackbox): cover MoA attribution end to end

[Kyzcreig]

Summary

• add a real MoA execution → turn finalizer → Blackbox SQLite seam test
• prove cached advisor pricing calls are cleared before accounting can duplicate them
• pin honest partial status when a nonzero nested physical route is unpriced

Verification

• scripts/run_tests.sh on 7 MoA/Blackbox files: 32 passed
• ruff check on changed tests: passed
• mutation proofs: explicit moa/default, cache-hit reset, and nested-partial mutations each made the intended test fail

Follow-up coverage for #342; test-only, no runtime behavior change.

[greptile-apps]

Greptile Summary

This PR adds test-only coverage for the MoA cost-attribution pipeline introduced in #342: a full end-to-end seam test (MoA execution → turn finalizer → Blackbox SQLite), a cache-clearing proof for advisor pricing calls, and a new partial-status assertion for unpriced physical routes.

• test_moa_seam_integration.py wires a real MoAChatCompletions.create() call through _build_moa_pricing_calls, finalize_turn, and the Blackbox plugin into an in-process SQLite instance, asserting that the stored turn row carries the correct provider, model, token counts, and cost status.
• test_moa_aggregator_cost_slot.py extends the existing cache test to prove that a HIT iteration zeros out any pending pricing calls, preventing stale MISS data from reaching accounting on a delayed consume.
• test_cost_perclass.py adds the complementary partial-status case for a nested MoA call where one physical route returns a nonzero-token unknown result.

Confidence Score: 4/5

Safe to merge — test-only, no runtime behaviour changes.

All three test files are test-only changes with no production code impact. The seam test carries an undocumented dependency on live pricing-DB entries for specific versioned model names; if those entries are missing or renamed in a different environment, two assertions fail with misleading equality errors rather than a clear diagnostic. The cache-gap and missing perclass-keys concerns are minor but real. Nothing here would silently regress production behaviour.

tests/plugins/blackbox/test_moa_seam_integration.py — pricing-data dependency makes the "estimated" assertions environment-sensitive.

Important Files Changed

Filename	Overview
tests/agent/test_moa_aggregator_cost_slot.py	Extends an existing test with a MISS+HIT sequence to verify the cache-HIT clears pending pricing calls; the MISS step is not independently verified, leaving a small coverage gap for new-message scenarios.
tests/plugins/blackbox/test_cost_perclass.py	Adds a new partial-status test for a nonzero unknown physical MoA route; logic is correct but the perclass key-set assertion is missing, inconsistent with other tests in the file.
tests/plugins/blackbox/test_moa_seam_integration.py	New end-to-end seam test wiring MoA execution → turn finalizer → Blackbox SQLite; relies on un-mocked pricing data for specific model names to assert "estimated" status, making it fragile if pricing entries change or differ across environments.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
    participant Test
    participant MoAClient
    participant MoAChatCompletions
    participant References as Reference Models (x2)
    participant finalize_turn
    participant BlackboxPlugin as Blackbox Plugin
    participant SQLite

    Test->>MoAClient: "chat.completions.create(model=default, messages)"
    MoAClient->>MoAChatCompletions: create()
    MoAChatCompletions->>References: _run_references_parallel() cache MISS
    References-->>MoAChatCompletions: usage + pricing_calls per advisor
    MoAChatCompletions-->>MoAClient: aggregator response
    Test->>MoAClient: consume_reference_pricing_calls()
    MoAClient-->>Test: [advisor1_call, advisor2_call]
    Test->>Test: _build_moa_pricing_calls + compute_turn_cost
    Test->>finalize_turn: "finalize_turn(_Agent, _turn_calls=[turn_call])"
    finalize_turn->>BlackboxPlugin: invoke_hook on_session_end
    BlackboxPlugin->>BlackboxPlugin: "_build_record model=moa/default"
    BlackboxPlugin->>SQLite: store.insert_turn
    Test->>SQLite: store.get_turn(turn_moa_seam)
    SQLite-->>Test: row
    Test->>Test: assert row cost_usd approx expected_cost

Loading

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
    participant Test
    participant MoAClient
    participant MoAChatCompletions
    participant References as Reference Models (x2)
    participant finalize_turn
    participant BlackboxPlugin as Blackbox Plugin
    participant SQLite

    Test->>MoAClient: "chat.completions.create(model=default, messages)"
    MoAClient->>MoAChatCompletions: create()
    MoAChatCompletions->>References: _run_references_parallel() cache MISS
    References-->>MoAChatCompletions: usage + pricing_calls per advisor
    MoAChatCompletions-->>MoAClient: aggregator response
    Test->>MoAClient: consume_reference_pricing_calls()
    MoAClient-->>Test: [advisor1_call, advisor2_call]
    Test->>Test: _build_moa_pricing_calls + compute_turn_cost
    Test->>finalize_turn: "finalize_turn(_Agent, _turn_calls=[turn_call])"
    finalize_turn->>BlackboxPlugin: invoke_hook on_session_end
    BlackboxPlugin->>BlackboxPlugin: "_build_record model=moa/default"
    BlackboxPlugin->>SQLite: store.insert_turn
    Test->>SQLite: store.get_turn(turn_moa_seam)
    SQLite-->>Test: row
    Test->>Test: assert row cost_usd approx expected_cost

Loading

_{Reviews (1): Last reviewed commit: "test(blackbox): cover MoA attribution en..." | Re-trigger Greptile}