Skip to content
Merged
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
103 changes: 103 additions & 0 deletions tests/routing-policy-surface-parity.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
import { describe, expect, test } from "bun:test";

import { chatCompletionsToResponsesBody } from "../src/chat/inbound";
import { anthropicToResponsesTranslation } from "../src/claude/inbound";
import { evidenceFromBody } from "../src/routing/request-evidence";

const MODEL = "policy/daily";
const EXPECTED_RICH_EVIDENCE = {
toolsRequired: true,
imageInputRequired: true,
};

describe("routing policy request evidence parity", () => {
test("tools and image input produce the same evidence across Responses, Chat Completions, and Claude Messages", () => {
const responsesBody = {
model: MODEL,
input: [{
type: "message",
role: "user",
content: [
{ type: "input_text", text: "inspect this" },
{ type: "input_image", image_url: "data:image/png;base64,AA==" },
],
}],
tools: [{
type: "function",
name: "inspect",
parameters: { type: "object", properties: {} },
}],
};

const chatBody = chatCompletionsToResponsesBody({
model: MODEL,
messages: [{
role: "user",
content: [
{ type: "text", text: "inspect this" },
{ type: "image_url", image_url: { url: "data:image/png;base64,AA==" } },
],
}],
tools: [{
type: "function",
function: {
name: "inspect",
parameters: { type: "object", properties: {} },
},
}],
});

const claudeBody = anthropicToResponsesTranslation({
model: MODEL,
max_tokens: 128,
messages: [{
role: "user",
content: [
{ type: "text", text: "inspect this" },
{
type: "image",
source: {
type: "base64",
media_type: "image/png",
data: "AA==",
},
},
],
}],
tools: [{
name: "inspect",
input_schema: { type: "object", properties: {} },
}],
}).body;

expect(evidenceFromBody(responsesBody)).toEqual(EXPECTED_RICH_EVIDENCE);
expect(evidenceFromBody(chatBody)).toEqual(EXPECTED_RICH_EVIDENCE);
expect(evidenceFromBody(claudeBody)).toEqual(EXPECTED_RICH_EVIDENCE);
});

test("plain text without tools produces no hard routing evidence on every surface", () => {
const responsesBody = {
model: MODEL,
input: [{
type: "message",
role: "user",
content: [{ type: "input_text", text: "hello" }],
}],
};

const chatBody = chatCompletionsToResponsesBody({
model: MODEL,
messages: [{ role: "user", content: "hello" }],
});

const claudeBody = anthropicToResponsesTranslation({
model: MODEL,
max_tokens: 128,
messages: [{ role: "user", content: "hello" }],
}).body;

expect(evidenceFromBody(responsesBody)).toEqual({});
expect(evidenceFromBody(chatBody)).toEqual({});
expect(evidenceFromBody(claudeBody)).toEqual({});
Comment on lines +76 to +104

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | 🏗️ Heavy lift

Exercise each routing entry point.

These assertions call evidenceFromBody() directly after calling the translators. They cannot fail if a Responses, Chat Completions, or Claude Messages dev handler stops passing its normalized body to evidenceFromBody(), passes the raw body, or drops the resulting evidence before policy evaluation.

Drive one request for each surface through its routing entry point. Assert the evidence received by the routing policy. Keep these assertions as translator unit tests if they are useful.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/routing-policy-surface-parity.test.ts` around lines 73 - 101, Update
the routing policy parity tests to exercise each surface through its actual
routing entry point rather than calling evidenceFromBody directly. For
Responses, Chat Completions, and Claude Messages, send representative requests
through their dev handlers and assert the evidence received by policy
evaluation, while retaining translator unit coverage where useful.

});
});
Loading