Repository navigation
fix(OMN-12813): SEA generation output format — force fenced yaml+python blocks + exemplar, suppress reasoning - #1105
Conversation
…on blocks + exemplar, suppress reasoning
Root cause: Qwen3.6-35B-A3B returns reasoning prose ("1. Analyze the
Request...") and bare YAML instead of the required two-fenced-block format,
causing contract_passed=false + empty contract_yaml/handler_source in the
generation_events projection row.
Changes:
- Strengthen _DEFAULT_SYSTEM_PROMPT: explicit "ONLY two fenced code blocks,
nothing else" directive (no chain-of-thought, no numbered analysis steps)
- Add two inference protocol profiles in inference_protocols.v1.yaml scoped
to task_type="node_generation":
* local-qwen-generation-no-think: prepends /no_think to user prompt for
Qwen models (suppresses reasoning tokens)
* local-qwen-generation-exemplar: appends a one-shot exemplar (```yaml +
```python with the exact required output shape) to the system prompt
- Wire apply_inference_protocol into HandlerGenerationConsumer._call_llm
with task_type="node_generation" so both profiles fire for Qwen
- Non-Qwen models and non-generation task types are unaffected (task_types
scoping prevents the exemplar from bleeding into delegation/summarization)
Tests (17 new, 0 regressions out of 6338):
- _DEFAULT_SYSTEM_PROMPT contains "ONEX node generator" marker + prose suppression
- Profiles fire for Qwen + node_generation, silent for other models/task types
- apply_once prevents /no_think double-insertion
- validate_generation: well-formed response → contract_passed=True
- validate_generation: reasoning-prose response → contract_passed=False
- Handler wires apply_inference_protocol: exemplar in system_prompt, /no_think
in user_prompt for Qwen
- End-to-end: well-formed LLM response → contract_passed=True + non-empty
contract_yaml + non-empty handler_source
Reuses OMN-12738 (PR #1065) inference_protocols machinery.
Exemplar is the proven stabilizer from context_matrix_v3 findings.
dod_evidence: 17 new tests pass, 6338/0 full suite, mypy --strict clean, pre-commit clean, generation profiles scoped to node_generation task type.
|
Warning Review limit reached
More reviews will be available in 20 minutes and 38 seconds. Learn how PR review limits work. Your organization has run out of usage credits. Purchase more in the billing tab. ⌛ How to resolve this issue?After more reviews become available, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available. Please see our Fair Usage Limits Policy for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (4)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
…neration-output-format # Conflicts: # src/omnimarket/nodes/node_generation_consumer/handlers/handler_generation_consumer.py
|
| Rule | Description |
|---|---|
| ARCH-TOPIC-001 | No hardcoded Kafka topic strings in handler code |
| ARCH-DI-001 | No event_bus=None bypass |
| ARCH-DI-002 | No direct Handler instantiation outside workflow_runner/adapters |
| ARCH-DI-003 | No reinvented DI containers |
| ARCH-TOPIC-002 | contract.yaml topics follow `onex.{cmd |
ARCH-DI-002: Direct Handler instantiation (warning)\n\n\nsrc/omnimarket/nodes/node_generation_consumer/handlers/handler_generation_consumer.py:491: self._effect = HandlerLlmOpenaiCompatible(transport=_Transport())\n\n\nHandler classes should be wired only in workflow_runner.py or adapter modules. Direct instantiation elsewhere bypasses the contract system.\n\n
Powered by node_pr_review_bot
✅ PR Review Bot — CLEANFindings: 0
Verdict semantics
Powered by node_pr_review_bot — Qwen3-Coder-30B reviewer + DeepSeek-R1 judge (OMN-7963) |
|
| Verdict | Meaning | Blocks merge? |
|---|---|---|
passed |
No critical/error findings | No |
blocked |
CRITICAL or ERROR findings found | Yes |
degraded |
All models unavailable (infra) | No (pilot) |
Powered by omniintelligence.review_pairing.cli_review — node-based adversarial review via HandlerLlmCliSubprocess (OMN-8524)
…t env vars (OmniNode-ai#687) * fix(OMN-11182): resolve generation consumer endpoint via contract, not env vars Replace bespoke GENERATION_CONSUMER_ENDPOINT/MODEL_ID/API_KEY/PROVIDER/ENDPOINT_CLASS env vars with contract-first endpoint resolution. - contract.yaml: add model_routing section (endpoint_env, model_id, auth) and dependencies entries (type: environment, key:) for LLM_CODER_URL, LOCAL_LLM_SHARED_SECRET, LLM_ENDPOINT_CIDR_ALLOWLIST so the overlay resolver knows these are required and populates them at boot - handler: read endpoint_env and model_id from contract model_routing at construction time; resolve the actual URL from os.environ[self._endpoint_env] (LLM_CODER_URL) rather than GENERATION_CONSUMER_ENDPOINT at call time - tests: 4 new unit tests verify contract declares correct model_routing and dependency keys, and that the handler falls back correctly when section absent * fix(OMN-11182): remove IP examples from contract descriptions; add deploy-gate DoD contract - contract.yaml: stripped 192.168.86.201 examples from dependency descriptions to pass Leaked Literals Gate - docs/work-tracking/contracts/OMN-11182.yaml: added deploy-gate contract with dod-deploy evidence item (docker exec import check) * fix(OMN-11182): add dod-occ-pr evidence item binding to OCC PR OmniNode-ai#1105 * ci(OMN-11182): retrigger receipt gate after OCC probe_stdout fix * ci(OMN-11182): retrigger receipt gate after dod-occ-pr commit_sha fix * ci(OMN-11182): retrigger receipt gate after adding Evidence-Ticket line * fix(OMN-11182): accept OPEN or MERGED state in dod-occ-pr check (CodeRabbit)
1 participant
Evidence-Source: OCC#2317
Evidence-Ticket: OMN-12813
Summary
Fixes the last-mile defect blocking a real SEA generation proof: Qwen3.6-35B-A3B returns reasoning prose ("1. Analyze the Request...") and bare YAML instead of the required two-fenced-block output, causing
contract_passed=falsewith emptycontract_yaml/handler_sourcein thegeneration_eventsprojection row.Changes
src/omnimarket/configs/inference_protocols.v1.yamllocal-qwen-generation-no-think: scoped totask_types: [node_generation]+model_name_patterns: [*qwen*]; prepends/no_thinkto user prompt to suppress reasoning tokenslocal-qwen-generation-exemplar: same scope; appends a one-shot exemplar (yaml contract +python handler with the exact required output shape) to the system prompt viasystem_suffixdirectivetask_type="node_generation"so they do NOT bleed into delegation, summarization, or code-generation callssrc/omnimarket/nodes/node_generation_consumer/handlers/handler_generation_consumer.py_DEFAULT_SYSTEM_PROMPT: explicit "ONLY two fenced code blocks, nothing else" directiveapply_inference_protocol(..., task_type="node_generation")into_call_llmbefore building the LLM request — same pattern as OMN-12738 / PR fix(OMN-12738): suppress qwen reasoning for code delegation #1065 delegation fixEvidence
contracts/OMN-12813.yaml(OCC#2317) + in-repo deploy-evidence carrierdocs/work-tracking/contracts/OMN-12813.yamlTest results
test_omn12813_output_format_fix.py— all passmypy --strict: 0 issues across 2308 source filestest_protocol_config.py(4 tests): pass — generation profiles do not affect summarization/delegation callsArchitecture notes
inference_protocols.v1.yaml— not hardcoded in Python (perfeedback_contract_config_not_env_vars)apply_inference_protocolfrom OMN-12738 — no new machinerytask_typesscoping is critical:_profile_matchesskipssystem_prompt_containswhentask_typeis provided, so profiles that omittask_typescan leak. Both new profiles usetask_typesnotsystem_prompt_contains.generation_eventsrowRelated
project_context_matrix_v3_findingsRebase + e2e (w-integration, 2026-06-07)