Repository navigation
feat(OMN-20910): native Anthropic Messages protocol with prompt caching for the review gateway - #4209
Conversation
…ng for the review gateway - node_llm_delegation_call_effect gains inference_protocol on its request (openai_chat default, anthropic_messages new) plus typed system_blocks and messages (text blocks, optional 5m or 1h cache TTL), tool_schema, tool_choice and provider_headers. - Setting blocks refuses prompt and system_prompt; more than four cache breakpoints, a credential in provider_headers, response_format on the native protocol and native fields on the chat protocol are each refused. - The result gains a typed stop_reason (EnumAnthropicStopReason), cache_creation_input_tokens (with 5m and 1h parts) and cache_read_input_tokens; the finished observation carries stop_reason and the cache token counts. - finish_reason is mapped from stop_reason; max_tokens maps to length. - The native call posts the key in x-api-key with an anthropic-version header, returns the forced tool's input as content, and bills cache writes at 1.25x (5m) and 2x (1h) and cache reads at 0.1x of the input price. - byok_provider_backends.v1.yaml gains a second anthropic row (plan api_native, backend_id byok-anthropic-native, endpoint https://api.anthropic.com/v1/messages); the existing api row becomes default_plan true. - node_claude_review_gateway_effect resolves the api_native plan, splits the system message into cached blocks and the user message into a diff block (5m cache) and a persona block (not cached) on a boundary marker. - It turns a json_schema response_format into a forced tool, answers finish_reason length on a max_tokens stop, and returns cache usage in the reply's usage; a review with no user message answers 400. - Tests: tests/test_omn20910_native_messages_protocol.py (23 tests) and the updated gateway and catalogue tests. - Not in this change: sticky key per review run, admission before each call, declared topics, model allowed list.
…cket contract's 50 character limit
- merge origin/dev (903e33f) into omnimarket#4209 branch omn-20910-native-messages-cache (head 7332678) - conflicts in five files: contracts/OMN-20910.yaml, src/omnimarket/events/llm_delegation_call.py, the delegation call effect handler, its observation model, and the Claude review gateway handler - resolve each conflict by keeping both sides: dev's review-call cost events (review provenance on the request, per-attempt observation fields) and this branch's native Anthropic Messages protocol (inference_protocol, system and message blocks, stop_reason, cache token counts) - contract keeps dev's AC7, AC10, AC11, AC12, AC13 and this branch's AC30, AC31, AC32 with their dod_evidence entries - native reply path also fills cache_read_tokens, cache_write_5m_tokens and cache_write_1h_tokens from the reply's usage, so a native review call's cost record carries its cache usage like the OpenAI-compatible path - native call keeps skipping the served-model probe; dev's served-model refresh applies to the other calls - tests/test_omn20910_review_gateway_provenance.py reads the user text from the request's message blocks instead of prompt, since the gateway sends the native protocol - tests/test_omn20910_native_messages_protocol.py gains assertions that the finished observation's cache_read_tokens and cache write figures match the native cache counts
|
| Rule | Description |
|---|---|
| ARCH-TOPIC-001 | No hardcoded Kafka topic strings in handler code |
| ARCH-DI-001 | No event_bus=None bypass |
| ARCH-DI-002 | No direct Handler instantiation outside workflow_runner/adapters |
| ARCH-DI-003 | No reinvented DI containers |
| ARCH-TOPIC-002 | contract.yaml topics follow `onex.{cmd |
ARCH-DI-002: Direct Handler instantiation (warning)\n\n```\nsrc/omnimarket/cli/cli_secret.py:447: return HandlerSecretStore()
src/omnimarket/cli/cli_secret.py:1116: result = HandlerLocalSecretStore().handle(
src/omnimarket/cli/cli_secret.py:1176: result = HandlerLocalSecretStore().handle(
src/omnimarket/cli/cli_secret.py:1206: result = HandlerCapturedSecretStore(onex_home=Path.home() / ".onex").handle(
src/omnimarket/cli/cli_secret.py:1244: result = HandlerCapturedSecretResolve(store=reader).handle(request)
src/omnimarket/cli/cli_secret.py:1286: written = HandlerLocalSecretStore().handle(
src/omnimarket/cli/cli_secret.py:1300: return HandlerModelSetup(write_model_list=_write_model_list)
src/omnimarket/cli/cli_secret.py:1769: handler = HandlerClaudeReviewGateway(
src/omnimarket/cli/cli_secret.py:1774: call=HandlerLlmDelegationCall(),\n```\n\nHandler classes should be wired only in workflow_runner.py or adapter modules. Direct instantiation elsewhere bypasses the contract system.\n\n
Static architectural lint — no model inference (OMN-14176).
|
The branch was brought current with dev by merge commit c3964d5 (parents 7332678 and dev 903e33f); the PR body's lab line names the earlier head 7332678, and the line below is for the new head. Five files had conflicts with dev's OMN-20910 review cost events, judge flag, replay set and batch nodes; each was resolved by keeping both sides. The native Messages reply now also fills cache_read_tokens, cache_write_5m_tokens and cache_write_1h_tokens, so a native review call's cost record carries its cache usage, and tests/test_omn20910_review_gateway_provenance.py reads the user text from the message blocks instead of prompt. Lab: host=h201 lane=fill-pf-drive-omnimarket-4209-20261019T0938Z head=c3964d58094a99488d9e6bf1cae9bc54275dc55e work_unit=lw-31923eca582b4994 command= |
✅ Hostile Reviewer — PASSEDBlocking findings (critical/error): 0 Gate semantics
Powered by omniintelligence.review_pairing.cli_review — node-based adversarial review via HandlerLlmCliSubprocess (OMN-8524) |
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| @property | ||
| def tokens_out(self) -> int: ... | ||
| @property | ||
| def stop_reason(self) -> EnumAnthropicStopReason | None: ... |
| @property | ||
| def stop_reason(self) -> EnumAnthropicStopReason | None: ... | ||
| @property | ||
| def cache_creation_input_tokens(self) -> int: ... |
| @property | ||
| def cache_creation_input_tokens(self) -> int: ... | ||
| @property | ||
| def cache_read_input_tokens(self) -> int: ... |
Merge current dev c2eb05b into the existing native Messages branch, retaining both sides of the OMN-20910 evidence contract conflict. This also includes the landed dev-tag release staleness fix. Declare the native protocol row explicit-only for plan detection. Both credential intake paths keep model discovery and key refusal on the default Anthropic plan; native selection stays available explicitly. Default and prohibited detection-only rows cannot opt out of detection. Validation: 704 passed, 1 skipped; registration regressions 54 passed with skips forbidden; strict mypy passed for 5910 source files; ruff check/format, node metadata dependency scope, and evidence binding checks passed. Closes OMN-17427
This change adds a native Anthropic Messages protocol with prompt caching in the one sanctioned model caller, used by the Claude review gateway. Requests gain an
inference_protocolswitch, typed system blocks and messages with optional 5m or 1h cache TTLs, and results gain a typed stop reason with cache creation and read token counts. The BYOK catalogue gains anapi_nativeanthropic row, and the gateway resolves that plan, splits its system and user messages into cached and uncached blocks, and returns cache usage.What changed
node_llm_delegation_call_effect: the request gainsinference_protocol(openai_chatdefault,anthropic_messagesnew), typedsystem_blocksandmessages(text blocks with optional 5m or 1h cache TTL),tool_schema,tool_choice, andprovider_headers. Blocks refusepromptandsystem_prompt. More than four cache breakpoints, a credential header inprovider_headers,response_formaton the native protocol, and native fields on the chat protocol are each refused.stop_reason(EnumAnthropicStopReason),cache_creation_input_tokens(with 5m and 1h parts), andcache_read_input_tokens; the finished observation carriesstop_reasonand the cache token counts, andfinish_reasonis mapped fromstop_reason(max_tokensgiveslength).x-api-keyplus ananthropic-versionheader, returns the forced tool input as content, and prices cache writes at 1.25x (5m) and 2x (1h) and reads at 0.1x of the input price.byok_provider_backends.v1.yaml: a new anthropic row with planapi_native, backendbyok-anthropic-native, endpointhttps://api.anthropic.com/v1/messages; the existingapirow is nowdefault_plan true, so a caller naming no plan keeps the chat-completions row.node_claude_review_gateway_effect: resolvesapi_nativeexplicitly; splits the system message into cached blocks and the user message into a diff block (5m cache) and a persona block (uncached) on a boundary marker; ajson_schemaresponse_format becomes a forced tool; amax_tokensstop answersfinish_reasonlength; cache usage is returned in usage; a review with no user message answers 400.contracts/OMN-20910.yamlgains requirementreq-native-messages-protocolwith AC30, AC31, AC32, threedod_evidenceentries bound to them, and a container deploy check.Tests
tests/test_omn20910_native_messages_protocol.py(23 tests): a fake provider simulates the prompt cache from the wire payload (tools, then system, then messages, up to a block withcache_control). Two calls differing only in the persona block: the second reportscache_read_input_tokensequal to the tool, both system blocks and the diff, through the call effect and through the gateway. Amax_tokensreply reaches the compact-strict parser as the typed stop reason and the parser reports truncated. A call naming no plan resolves the compatibility row and posts the chat shape with a Bearer key. At the merge base, 18 of the 23 tests fail./v1/messagesresponse shape; they are not recordings from a live key, because the lane holds no key.Lab: host h201, lane hr-native-messages-cache-0410, command
onex-lab-run --via bus --host h201 --kind test -- uv run pytest tests/test_omn20910_native_messages_protocol.py tests/unit/nodes/node_claude_review_gateway_effect tests/test_omn17353_provider_catalogue.py tests/test_omn17372_byok_routing_overlay_bridge.py tests/test_byok_gemini_glm_plans.py tests/test_omn19205_c29_byok_throttle_re_resolve.py tests/test_omn20157_ac5_byok_gemini_receipt_names_model.py tests/test_golden_chain_node_delegate_skill_orchestrator.py tests/ciat sha 7332678, exit 0, 2003 passed and 2 skipped; tests/ci/test_ruleset_readback_omn20070.py deselected because it reads GitHub directly and a lane's raw GitHub reads are refused. No live Anthropic call was made.Not in this change
Closes OMN-20910
Ticket: OMN-20910