fix(api): treat assistant refusal/annotations null-empty as omit; fail-closed otherwise - #577
fix(api): treat assistant refusal/annotations null-empty as omit; fail-closed otherwise#577seonghobae wants to merge 15 commits into
Conversation
… effort as omit
Buyer SDKs often send stream_options with only false flags, tool_choice:{},
and reasoning_effort:"" as optional defaults. Accept them as omit no-ops
while still fail-closing true stream_options flags without stream=true and
non-empty reasoning_effort. Tip honesty substrate re-ship; 849 unit pass.
SDK clients send user:null as an optional default. Treat null as omit on chat Completions, legacy Completions, Responses, embeddings, and batch embeddings. Empty/whitespace/non-string user still fail closed with invalid_user. Local full unit: 857 passed.
…l/response_format/endpoint as omit SDK clients and stringified optional controls may send empty or whitespace strings. Treat as omit no-ops on embeddings encoding_format, chat/Responses tool_choice and function_call, response_format, and batch embeddings endpoint. Non-empty unsupported values still fail closed. Local full unit: 865 passed.
SDK stringified empty controls for reasoning, Responses text, and include are treat-as-omit on chat Completions, legacy Completions, and Responses. Non-empty unsupported values still fail closed with named errors. Local full unit: 872 passed.
… as omit Legacy Completions has no tools surface. Treat SDK defaults tool_choice none/auto/empty-string/empty-object and function_call none/auto/empty-string as omit no-ops (parity with chat). Non-default controls and non-empty tools still fail closed with a chat migration path. Local full unit: 877 passed.
SDK clients may send top_logprobs:0 (no top alternatives). Treat 0 and null as omit on chat Completions and legacy Completions. Non-zero values still fail closed with invalid_top_logprobs. Local full unit: 881 passed.
…format float
Incidental whitespace around honest no-op values (" auto ", " float ")
is stripped before validation so SDK-padded strings match. Unsupported
values (flex, base64) still fail closed after strip. Local full unit:
886 passed.
…ll named honesty Treat empty-string response_format/prediction/reasoning_effort as omit on legacy Completions. Accept audio/web_search_options keys with null/empty as omit and non-empty as named invalid_* (not unknown_fields). Local full unit: 894 passed.
…ns modalities text no-op Whitespace-padded none/auto on tool_choice and function_call are omit no-ops on chat, Completions, and Responses. Completions modalities ["text"] is an honest text-only no-op; non-text modalities still fail closed. Local full unit: 901 passed.
…l names Treat empty/whitespace prediction as omit on chat and Responses. Strip modalities array items and empty-string modalities for text-only match. Strip model names on Completions/chat, Responses, and embeddings so SDK-padded pool ids resolve. Local full unit: 907 passed.
…nd as omit OpenAI reasoning_effort none disables extra reasoning — honest omit no-op on chat and Completions. Empty/whitespace store, stream, and background strings are treat-as-omit across chat Completions, Completions, and Responses store. Local full unit: 911 passed.
SDK clients may stringify omitted optionals as empty strings. Treat empty/whitespace temperature, top_p, max_tokens, max_completion_tokens, penalties, n, seed, logprobs, parallel_tool_calls, include_orchestration_trace, echo, best_of, dimensions, max_output_tokens, and responses stream as omit. Whitespace-only stop arrays are also omit. Local full unit: 917 passed.
Vision callers send content-parts arrays. Shape-check and passthrough text/image_url parts; unsupported part types fail closed. Coerce part text for agent selection so list content does not 500. Tip substrate from empty-string numeric honesty. Local full unit: 921 passed.
Reject empty/whitespace text parts and empty image_url.url; accept bare-string image_url; validate optional detail to auto/low/high (null/empty omit). HTTP tests lock named invalid_message_content errors. Local full unit: 928 passed.
…l-closed otherwise SDK clients send refusal:null and annotations:[] as optional defaults. Treat null/empty as omit. Non-empty refusal or annotations fail closed with named errors so clients never believe those planes were applied. Tip substrate from #576. Local full unit: 932 passed.
|
Important Review skippedToo many files! This PR contains 131 files, which is 31 over the limit of 100. To get a review, reduce the PR to 100 files or fewer by splitting it into smaller PRs or changing its base branch. Upgrade to a paid plan to raise the limit. This review couldn't start because sufficient usage credits or metered capacity aren't available. Add credits or update usage-based reviews in the billing tab, then retry. ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (131)
You can disable this status message by setting the Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![opencode-agent[bot]](https://avatars.githubusercontent.com/in/1549082?s=60&v=4){kind=small}
There was a problem hiding this comment.
Pull request overview
OpenCode cannot approve yet because required coverage evidence did not pass.
Review outcome
1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence
-
Problem: The required coverage-evidence job result was
failure, so OpenCode cannot establish approval sufficiency for this head. -
Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.
-
Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports
successwith required evidence or explicit no-source not-applicable evidence. -
Regression test: Keep the approval branch checking
needs.coverage-evidence.result == successbefore posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present. -
Result: REQUEST_CHANGES
-
Reason: coverage-evidence result was
failure, so required test/docstring evidence was not proven for current headb8ec73b312d6b82b9c5859bfa5dbefc2d594713f. -
Head SHA:
b8ec73b312d6b82b9c5859bfa5dbefc2d594713f -
Workflow run: 32020845339
-
Workflow attempt: 1
Coverage evidence
Coverage evidence job did not run or did not publish coverage evidence.
Changed-File Evidence Map
flowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Changed file (3 files)"]
S1 --> I1["repository behavior"]
I1 --> R1["Review risk: Changed file (3 files)"]
R1 --> V1["required checks"]
Evidence --> S2["Test (128 files)"]
S2 --> I2["regression suite"]
I2 --> R2["Review risk: Test (128 files)"]
R2 --> V2["targeted test run"]
OpenCode Review Overview
Pull request overviewOpenCode cannot approve yet because required coverage evidence did not pass. Review outcome1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence
Coverage evidenceCoverage evidence job did not run or did not publish coverage evidence. Changed-File Evidence Mapflowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Changed file (3 files)"]
S1 --> I1["repository behavior"]
I1 --> R1["Review risk: Changed file (3 files)"]
R1 --> V1["required checks"]
Evidence --> S2["Test (128 files)"]
S2 --> I2["regression suite"]
I2 --> R2["Review risk: Test (128 files)"]
R2 --> V2["targeted test run"]
|
1 participant
Summary
refusalnull/empty omit; non-empty fails closed (invalid_message_refusal).annotationsnull/empty omit; non-empty fails closed (invalid_message_annotations).Test plan
pytest tests/test_assistant_refusal_annotations_http_honesty.pypytest tests -q(932 passed) — evidence implementer/evidence/pytest-577-full.logProduct gates only: Full unit + Semgrep (Strix ignored). Independent non-author APPROVE still required.