test(gateway): pin the final-send suppression contract across a behaviour matrix

[JoaoMarcos44]

Summary

Complementary to #82592 — no overlap, no competing fix. This PR ships no behaviour change: it pins the invariant that the whole final-send suppression family depends on, and reports the one exposure that remains open.

The gateway skips its own final send when the stream consumer claims the turn final already reached the user (gateway/run.py: final_response_sent / final_content_delivered, reconciled through delivered_final_matches). Every incident in that family is the same failure:

Issue	Symptom	Fixed at
#71643	successful finalize edit carried only the last preview snapshot	one got_done branch
#78541	payload-less multi-message split inherited legacy trust	delivered_final_matches split guard
#82656	frozen preview left on screen with a visible cursor	open (#82592 in flight)

Each was fixed with a scenario test pinned to one branch of GatewayStreamConsumer.run(). That got_done handler now has five sibling branches that each set the suppression flags and record a turn-final payload, and nothing checks them as a group. A new branch — or a new early return True in _send_or_edit — can reintroduce the class without failing a test.

What this adds

tests/gateway/test_suppression_contract_matrix.py pins the invariant instead of the branch:

If the consumer offers the gateway any signal it would trust, the complete final text must have reached the wire.

%%{init: {'theme':'base','themeVariables':{'primaryColor':'#0d0221','primaryTextColor':'#00f0ff','primaryBorderColor':'#ff00a0','lineColor':'#00f0ff','secondaryColor':'#1a0b2e','tertiaryColor':'#0d0221','fontFamily':'monospace','clusterBkg':'#12061f','clusterBorder':'#ff00a0','edgeLabelBackground':'#0d0221'}}}%%
flowchart TD
    subgraph MATRIX["Behaviour Matrix - 40 Scenarios"]
        E["Edit Transport<br/>always / dies / never / lies-after-2 / lies-always"]
        S["Send Transport<br/>always / never"]
        F["Fresh-Final Preference<br/>on / off"]
        I["Stream End<br/>clean / interrupted"]
    end

    MATRIX --> DRIVE["Real GatewayStreamConsumer<br/>on_delta -> on_delta -> finish / cancel"]
    DRIVE --> WIRE["Wire Ledger<br/>records ONLY rendered frames"]
    DRIVE --> CLAIM["Consumer Claim<br/>final_response_sent OR final_content_delivered<br/>not vetoed by delivered_final_matches"]

    WIRE --> CHECK{"Claim implies<br/>complete answer rendered?"}
    CLAIM --> CHECK

    CHECK -->|"Honest transport - 24 scenarios"| PASS["HARD ASSERTION<br/>24 passed"]
    CHECK -->|"Lying transport - 16 scenarios"| SPLIT{"Invariant held?"}
    SPLIT -->|"15 scenarios"| PASS2["Invariant holds"]
    SPLIT -->|"1 scenario"| GAP["EXPECTED FAILURE<br/>open exposure #82656"]

    style PASS fill:#0d0221,stroke:#00ff9f,stroke-width:2px,color:#00ff9f
    style PASS2 fill:#0d0221,stroke:#00ff9f,stroke-width:2px,color:#00ff9f
    style GAP fill:#1a0b2e,stroke:#ff00a0,stroke-width:3px,color:#ff00a0
    style CHECK fill:#12061f,stroke:#ff00a0,stroke-width:2px,color:#00f0ff
    style SPLIT fill:#12061f,stroke:#ff00a0,stroke-width:2px,color:#00f0ff
    style WIRE fill:#1a0b2e,stroke:#00f0ff,stroke-width:2px,color:#00f0ff
    style CLAIM fill:#1a0b2e,stroke:#00f0ff,stroke-width:2px,color:#00f0ff
    style DRIVE fill:#12061f,stroke:#00f0ff,stroke-width:2px,color:#00f0ff

Loading

The fake adapter records only frames that actually rendered, so an ACK the platform drops does not count as delivery. That distinction is the whole point: the consumer advances _last_sent_text and its turn-final record from a call's return value, never from evidence that the payload reached a screen.

Results

39 passed, 1 xfailed in 38.98s

• 24 honest-transport scenarios — hard assertion. This is the regression net: it covers all five got_done sibling branches at gateway/stream_consumer.py lines 973, 1073, 1103, 1117 and 1195, plus whatever branch a future scenario happens to take.
• 16 lying-transport scenarios — same invariant checked. 15 hold. The one that violates it is reported as an expected failure with the wire ledger attached, documenting the exposure rather than asserting it away.

The failing combination is edit_lies_always + send_always + clean stream:

_last_sent_text       = <complete answer>   <- advanced by the dropped ACK
_delivered_final_text = <complete answer>
delivered_final_matches(final) -> True      -> gateway suppresses
rendered wire         = ["Ack received. Starting the deploy now, |"]

That is the exact log signature in #82656: streamed=True previewed=False content_delivered=True, with only a cursor-terminated partial on screen.

Relationship to #82592

#82592 is the fix; this is the net under it. Two notes for that review, both out of scope here:

1. _mark_skip_redundant_finalize records cursor-stripped _last_sent_text instead of _accumulated. That closes the case where the transport reports failure (the relay disconnect() path fix(gateway,relay): stop frozen-preview finals and dropped idle-session delegation callbacks #82592 also fixes, where the pending future was failed so _last_sent_text never advanced). It does not close the case where the transport ACKs and drops — both the recorded payload and the acked text are poisoned identically. That is the scenario this PR marks as an expected failure.
2. _DISCONNECT_DRAIN_GRACE_S = 5.0 is prepended to RelayWSTransport.disconnect(), whose own comment budgets "three sequential awaits at 1.0s [to] stay under the runner's default 5s adapter disconnect budget". Worst case becomes 8.0s inside asyncio.wait_for(adapter.disconnect(), 5.0) (gateway/run.py:6585, _ADAPTER_DISCONNECT_TIMEOUT_SECS_DEFAULT at run.py:87), so the drain can be cancelled mid-flight — the teardown awaits and the fail-any-remaining loop never run, and the frame the drain was protecting is lost anyway. Clamping the grace to the caller's remaining budget would keep it inside the leash.

Test plan

• pytest tests/gateway/test_suppression_contract_matrix.py -q → 39 passed, 1 xfailed
• New file only; no production code touched
• CI green

Refs #82656, #82592, #71643, #78541

[Riccardo-Vecchi]

The matrix is a useful direction, but in its current form it can stay green while the known exposure expands, and it does not actually execute the gateway suppression decision it claims to pin. Those two properties weaken it as a regression net.

[Riccardo-Vecchi]

This dynamic xfail masks every lying-transport scenario that violates the invariant, not only the one known exposure. A regression could take the result from 1 xfail to all 16 scenarios xfailed and CI would remain green. Please mark the exact known parameter ID with pytest.mark.xfail(strict=True, reason=...) and keep a hard assertion for every other matrix cell. Then expansion becomes a failure, and fixing the known cell becomes an XPASS that forces removal of the marker.

[Riccardo-Vecchi]

This is a test-local reimplementation of the production suppression decision, so the matrix does not pin gateway/run.py: if the real decision adds/changes a condition while this helper stays unchanged, the test can pass while the gateway suppresses incorrectly. Since the PR's stated invariant is about what the gateway would trust, please extract/import the production predicate (including stale-final reconciliation) or drive the real boundary, then assert against that result.

[victor-kyriazakos]

@JoaoMarcos44 — thank you for this. The matrix caught a real budget-cliff in our drain fix (now clamped, 2b92ae874 on #82592) and your ACK-then-drop analysis was exactly right about where the class boundary sits. We've carried this harness onto the #82592 branch as a cherry-pick with your authorship preserved (b973b5cd8) — it runs 39 passed + 1 xfail there and is now part of the delivery-path regression set our staging fleet gates on. Once #82592 merges, this PR can close as merged-via that branch with your commit intact in history. Appreciated the falsification rigor on #82656 too — confirming the proposed mechanism is unreachable saved everyone a wrong fix.