test: split up e2e tests - #836
Conversation
|
Warning Rate limit exceeded@steebchen has exceeded the limit for the number of commits or files that can be reviewed per hour. Please wait 17 minutes and 10 seconds before requesting another review. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. 📒 Files selected for processing (12)
WalkthroughDeletes the legacy gateway monolithic e2e, adds a shared chat e2e helper module, introduces multiple focused chat-*.e2e.ts test suites, and updates an individual test to import the shared helpers (request-id/log utilities). Changes
Sequence Diagram(s)sequenceDiagram
autonumber
participant T as Vitest Test
participant A as App (/v1/chat/completions)
participant L as Log Store
rect rgba(200,230,255,0.25)
note right of T: Non-streaming request (shared helpers + focused suites)
T->>A: POST /v1/chat/completions\nAuthorization, X-Request-Id
A-->>T: 200 JSON response (choices, usage, optional reasoning)
end
A->>L: Write request/usage log (streamed: false)
T->>L: Fetch logs by requestId via validateLogByRequestId
L-->>T: Log entry
sequenceDiagram
autonumber
participant T as Vitest Test
participant A as App (/v1/chat/completions?stream=true)
participant L as Log Store
rect rgba(200,255,200,0.25)
note right of T: Streaming (SSE) tests (streaming & streaming-reasoning)
T->>A: POST with stream: true\nAuthorization, X-Request-Id
A-->>T: text/event-stream chunks (delta.content, usage, optional reasoning_content)
end
A->>L: Write request/usage log (streamed: true)
T->>L: Fetch logs by requestId via validateLogByRequestId
L-->>T: Log entry
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Possibly related PRs
Pre-merge checks and finishing touches❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 4
🧹 Nitpick comments (24)
apps/gateway/src/chat-api.e2e.ts (3)
27-30: Use non-deprecated string API for request IDssubstr is deprecated. Use slice.
-export function generateTestRequestId(): string { - return `test-${Date.now()}-${Math.random().toString(36).substr(2, 9)}`; -} +export function generateTestRequestId(): string { + return `test-${Date.now()}-${Math.random().toString(36).slice(2, 11)}`; +}
37-38: Gate noisy startup log behind LOG_MODEAvoid polluting CI logs.
-console.log("running with test options:", getTestOptions()); +if (logMode) console.log("running with test options:", getTestOptions());
19-25: Prefer alias import for consistency and resolver stabilityOther suites use "@/index". Align here to avoid resolver/order lint noise.
-import { app } from "."; +import { app } from "@/index";apps/gateway/src/chat-toolcalls.e2e.ts (1)
100-105: Be tolerant to providers that return parsed argumentsSome providers may already return an object for function.arguments. Handle both.
-const args = JSON.parse(toolCall.function.arguments); +const args = + typeof toolCall.function.arguments === "string" + ? JSON.parse(toolCall.function.arguments) + : toolCall.function.arguments;apps/gateway/src/api-individual.e2e.ts (2)
13-19: Align app import with alias used elsewhereUse "@/index" for consistency and to satisfy import resolvers.
-import { app } from "."; +import { app } from "@/index";
240-246: Gate debug output behind LOG_MODEAvoid unconditional console noise.
-const json = await res.json(); -console.log("response:", JSON.stringify(json, null, 2)); +const json = await res.json(); +if (logMode) console.log("response:", JSON.stringify(json, null, 2));apps/gateway/src/chat-full.e2e.ts (5)
23-31: Skip the suite explicitly when FULL_MODE is offWhen FULL_MODE is falsy this file defines an empty suite. Prefer an explicit skip to make CI intent obvious.
- if (fullMode) { + if (fullMode) { // tests... - } + } else { + test.skip("reasoning + tool calls (FULL_MODE disabled)", () => {}); + }
115-120: Harden parsing of tool call argumentsGuard the type and surface a clearer failure when arguments aren’t valid JSON.
- const args = JSON.parse(toolCall.function.arguments); + expect(typeof toolCall.function.arguments).toBe("string"); + let args: any; + try { + args = JSON.parse(toolCall.function.arguments as string); + } catch (e) { + throw new Error(`Invalid tool_call.arguments JSON: ${toolCall.function.arguments}`); + }
121-123: Relax finish_reason to reduce provider-specific flakinessSome providers variably report finish_reason. Matching “tool_call” is more tolerant.
- expect(json.choices[0]).toHaveProperty("finish_reason", "tool_calls"); + expect(String(json.choices[0].finish_reason)).toMatch(/tool_call/);
125-139: Avoid unnecessary type assertionNarrow via a truthy check instead of casting.
- const reasoningProvider = providers?.find( - (p: ProviderModelMapping) => p.reasoning === true, - ); - if ( - (reasoningProvider as ProviderModelMapping)?.reasoningOutput !== "omit" - ) { + const reasoningProvider = providers?.find( + (p: ProviderModelMapping) => p.reasoning === true, + ); + if (reasoningProvider && reasoningProvider.reasoningOutput !== "omit") { // assertions... }
81-83: Stabilize outputs with deterministic decodingSetting a low temperature reduces variance across providers and lowers flake rate.
tool_choice: "auto", reasoning_effort: "medium", + temperature: 0,apps/gateway/src/chat-toolcalls-result.e2e.ts (4)
90-93: Fix misleading log labelThe message says “empty content” but this test validates a tool-call result.
- "tool calls with empty content response:", + "tool calls with result response:",
49-55: Use a neutral tool_call id to avoid provider‑specific formatsHard‑coding an OpenAI‑looking id can be brittle. Use a generic id and keep it consistent.
- id: "toolu_015dgN1nk5Ay12iN8e16XPbs", + id: "tc_1", ... - tool_call_id: "toolu_015dgN1nk5Ay12iN8e16XPbs", + tool_call_id: "tc_1",Also applies to: 62-63
121-124: Optionally assert toolResults are recorded in logsIf the gateway stores tool results, assert they exist when available to catch regressions.
const log = await validateLogByRequestId(requestId); expect(log.streamed).toBe(false); +if (log.toolResults) { + expect(Array.isArray(log.toolResults)).toBe(true); + expect(log.toolResults.length).toBeGreaterThan(0); +}If some providers don’t persist toolResults, keep the guard as shown.
33-85: Add temperature to reduce variabilityLower temperature stabilizes post‑tool completions.
model: model, messages: [ // ... ], tools: [ // ... ], tool_choice: "auto", + temperature: 0,apps/gateway/src/chat-rs.e2e.ts (3)
59-61: Guard header accesscontent-type can be null; guard to avoid rare NPEs.
-expect(res.headers.get("content-type")).toContain("text/event-stream"); +expect((res.headers.get("content-type") ?? "")).toContain("text/event-stream");
62-76: Assert proper SSE terminationEnsure the stream ends with [DONE] to catch truncation.
const streamResult = await readAll(res.body); // logging... expect(streamResult.hasValidSSE).toBe(true); expect(streamResult.eventCount).toBeGreaterThan(0); expect(streamResult.hasContent).toBe(true); +expect(streamResult.fullContent?.includes("data: [DONE]")).toBe(true);
49-51: Stabilize with temperatureLower temperature reduces variability in streaming reasoning outputs.
reasoning_effort: "medium", stream: true, + temperature: 0,apps/gateway/src/chat-reasoning.e2e.ts (6)
18-22: Concurrent suite is fine; just be mindful of provider rate limits.If CI flakes due to upstream throttling, consider
describe.sequentialor per‑provider gating for those cases.
23-27: Verify Vitesttest.eachoptions usage; prefer explicitretrychaining.Passing an options object as the second arg to
test.eachmay be ignored depending on Vitest version. Useretrychaining to ensure it applies.Apply this diff to make retries explicit and keep the API stable:
-test.each(reasoningModels)( - "reasoning $model", - getTestOptions(), - async ({ model, providers }) => { +const { retry = 0 } = getTestOptions(); +const t = retry > 0 ? test.retry(retry) : test; +t.each(reasoningModels)( + "reasoning $model", + async ({ model, providers }) => {
62-72: Remove duplicate existence checks for usage fields.
validateResponse(json)already asserts presence; keep type/value assertions only.- expect(json).toHaveProperty("usage"); - expect(json.usage).toHaveProperty("prompt_tokens"); - expect(json.usage).toHaveProperty("completion_tokens"); - expect(json.usage).toHaveProperty("total_tokens"); expect(typeof json.usage.prompt_tokens).toBe("number"); expect(typeof json.usage.completion_tokens).toBe("number"); expect(typeof json.usage.total_tokens).toBe("number");
73-77: Use anincheck and stricter number validation for optionalreasoning_tokens.- if (json.usage.reasoning_tokens !== undefined) { - expect(typeof json.usage.reasoning_tokens).toBe("number"); - expect(json.usage.reasoning_tokens).toBeGreaterThanOrEqual(0); - } + if ("reasoning_tokens" in json.usage) { + expect(Number.isFinite(json.usage.reasoning_tokens)).toBe(true); + expect(json.usage.reasoning_tokens).toBeGreaterThanOrEqual(0); + }
79-91: Also assert log.reasoningContent consistently with response gating.This strengthens the end‑to‑end guarantee that reasoning output is persisted when expected.
const useResponsesApi = process.env.USE_RESPONSES_API === "true"; const isOpenAI = reasoningProvider?.providerId === "openai"; - // only enforce reasoning_content checks for where reasoningOutput is not "omit" and for openai, only if the responses api is used - if ( - reasoningProvider?.reasoningOutput !== "omit" && - (!isOpenAI || useResponsesApi) - ) { - expect(json.choices[0].message).toHaveProperty("reasoning_content"); - } + // enforce reasoning checks when provider returns reasoning output + const expectsReasoning = + reasoningProvider?.reasoningOutput !== "omit" && (!isOpenAI || useResponsesApi); + if (expectsReasoning) { + expect(json.choices[0].message).toHaveProperty("reasoning_content"); + expect(Boolean(log.reasoningContent)).toBe(true); + } else { + // Some providers intentionally omit reasoning + expect(json.choices[0].message.reasoning_content ?? null).toBeNull(); + }
80-85: Avoid the cast by using a type predicate infind.Removes the
as ProviderModelMappingand tightens types.- const reasoningProvider = providers?.find( - (p: ProviderModelMapping) => p.reasoning === true, - ) as ProviderModelMapping; + const reasoningProvider = providers?.find( + (p: ProviderModelMapping): p is ProviderModelMapping => p.reasoning === true, + );
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (9)
apps/gateway/src/api-individual.e2e.ts(1 hunks)apps/gateway/src/api.e2e.ts(0 hunks)apps/gateway/src/chat-api.e2e.ts(1 hunks)apps/gateway/src/chat-full.e2e.ts(1 hunks)apps/gateway/src/chat-reasoning.e2e.ts(1 hunks)apps/gateway/src/chat-rs.e2e.ts(1 hunks)apps/gateway/src/chat-streaming.e2e.ts(1 hunks)apps/gateway/src/chat-toolcalls-result.e2e.ts(1 hunks)apps/gateway/src/chat-toolcalls.e2e.ts(1 hunks)
💤 Files with no reviewable changes (1)
- apps/gateway/src/api.e2e.ts
🧰 Additional context used
📓 Path-based instructions (7)
**/*.{ts,tsx}
📄 CodeRabbit inference engine (CLAUDE.md)
Always use top-level import; never use require() or dynamic imports
Files:
apps/gateway/src/chat-streaming.e2e.tsapps/gateway/src/chat-reasoning.e2e.tsapps/gateway/src/chat-toolcalls.e2e.tsapps/gateway/src/chat-full.e2e.tsapps/gateway/src/chat-rs.e2e.tsapps/gateway/src/chat-toolcalls-result.e2e.tsapps/gateway/src/chat-api.e2e.tsapps/gateway/src/api-individual.e2e.ts
{apps/{api,gateway}/src,packages/db}/**/*.ts?(x)
📄 CodeRabbit inference engine (CLAUDE.md)
Use Drizzle ORM with the latest object syntax
Files:
apps/gateway/src/chat-streaming.e2e.tsapps/gateway/src/chat-reasoning.e2e.tsapps/gateway/src/chat-toolcalls.e2e.tsapps/gateway/src/chat-full.e2e.tsapps/gateway/src/chat-rs.e2e.tsapps/gateway/src/chat-toolcalls-result.e2e.tsapps/gateway/src/chat-api.e2e.tsapps/gateway/src/api-individual.e2e.ts
apps/{api,gateway}/src/**/*.{ts,tsx}
📄 CodeRabbit inference engine (CLAUDE.md)
For read queries, use db().query.
.findMany() or db().query..findFirst() Files:
apps/gateway/src/chat-streaming.e2e.tsapps/gateway/src/chat-reasoning.e2e.tsapps/gateway/src/chat-toolcalls.e2e.tsapps/gateway/src/chat-full.e2e.tsapps/gateway/src/chat-rs.e2e.tsapps/gateway/src/chat-toolcalls-result.e2e.tsapps/gateway/src/chat-api.e2e.tsapps/gateway/src/api-individual.e2e.ts**/*.e2e.ts
📄 CodeRabbit inference engine (CLAUDE.md)
Place end-to-end tests in files matching *.e2e.ts
Place end-to-end tests in files named *.e2e.ts
Files:
apps/gateway/src/chat-streaming.e2e.tsapps/gateway/src/chat-reasoning.e2e.tsapps/gateway/src/chat-toolcalls.e2e.tsapps/gateway/src/chat-full.e2e.tsapps/gateway/src/chat-rs.e2e.tsapps/gateway/src/chat-toolcalls-result.e2e.tsapps/gateway/src/chat-api.e2e.tsapps/gateway/src/api-individual.e2e.ts{apps/api,apps/gateway,apps/ui,apps/docs,packages}/**/*.{ts,tsx}
📄 CodeRabbit inference engine (AGENTS.md)
Always use top-level import; never use require() or dynamic imports (e.g., import(), next/dynamic)
Files:
apps/gateway/src/chat-streaming.e2e.tsapps/gateway/src/chat-reasoning.e2e.tsapps/gateway/src/chat-toolcalls.e2e.tsapps/gateway/src/chat-full.e2e.tsapps/gateway/src/chat-rs.e2e.tsapps/gateway/src/chat-toolcalls-result.e2e.tsapps/gateway/src/chat-api.e2e.tsapps/gateway/src/api-individual.e2e.ts{apps/api,apps/gateway,packages/db}/**/*.ts
📄 CodeRabbit inference engine (AGENTS.md)
{apps/api,apps/gateway,packages/db}/**/*.ts: Use Drizzle ORM with the latest object syntax for database access
For reads, use db().query..findMany() or db().query.
.findFirst() Files:
apps/gateway/src/chat-streaming.e2e.tsapps/gateway/src/chat-reasoning.e2e.tsapps/gateway/src/chat-toolcalls.e2e.tsapps/gateway/src/chat-full.e2e.tsapps/gateway/src/chat-rs.e2e.tsapps/gateway/src/chat-toolcalls-result.e2e.tsapps/gateway/src/chat-api.e2e.tsapps/gateway/src/api-individual.e2e.tsapps/gateway/src/api-individual.e2e.ts
📄 CodeRabbit inference engine (CLAUDE.md)
Put isolated E2E tests in apps/gateway/src/api-individual.e2e.ts
Put isolated E2E test cases in apps/gateway/src/api-individual.e2e.ts
Files:
apps/gateway/src/api-individual.e2e.ts🧠 Learnings (7)
📓 Common learnings
Learnt from: CR PR: theopenco/llmgateway#0 File: CLAUDE.md:0-0 Timestamp: 2025-09-15T13:15:00.724Z Learning: Applies to apps/gateway/src/api-individual.e2e.ts : Put isolated E2E tests in apps/gateway/src/api-individual.e2e.tsLearnt from: CR PR: theopenco/llmgateway#0 File: AGENTS.md:0-0 Timestamp: 2025-09-15T13:16:05.355Z Learning: Applies to apps/gateway/src/api-individual.e2e.ts : Put isolated E2E test cases in apps/gateway/src/api-individual.e2e.tsLearnt from: CR PR: theopenco/llmgateway#0 File: AGENTS.md:0-0 Timestamp: 2025-09-15T13:16:05.355Z Learning: Applies to apps/gateway/src/api.e2e.ts : Put parallelizable .each() E2E tests in apps/gateway/src/api.e2e.ts (uses concurrent mode)Learnt from: CR PR: theopenco/llmgateway#0 File: CLAUDE.md:0-0 Timestamp: 2025-09-15T13:15:00.724Z Learning: Applies to apps/gateway/src/api.e2e.ts : Put parallelized .each() E2E tests in apps/gateway/src/api.e2e.ts (uses concurrent mode)Learnt from: CR PR: theopenco/llmgateway#0 File: AGENTS.md:0-0 Timestamp: 2025-09-15T13:16:05.355Z Learning: Applies to **/*.e2e.ts : Place end-to-end tests in files named *.e2e.tsLearnt from: CR PR: theopenco/llmgateway#0 File: CLAUDE.md:0-0 Timestamp: 2025-09-15T13:15:00.724Z Learning: Applies to **/*.e2e.ts : Place end-to-end tests in files matching *.e2e.ts📚 Learning: 2025-09-15T13:15:00.724Z
Learnt from: CR PR: theopenco/llmgateway#0 File: CLAUDE.md:0-0 Timestamp: 2025-09-15T13:15:00.724Z Learning: Applies to apps/gateway/src/api-individual.e2e.ts : Put isolated E2E tests in apps/gateway/src/api-individual.e2e.tsApplied to files:
apps/gateway/src/chat-streaming.e2e.tsapps/gateway/src/chat-reasoning.e2e.tsapps/gateway/src/chat-toolcalls.e2e.tsapps/gateway/src/chat-full.e2e.tsapps/gateway/src/chat-rs.e2e.tsapps/gateway/src/chat-toolcalls-result.e2e.tsapps/gateway/src/chat-api.e2e.tsapps/gateway/src/api-individual.e2e.ts📚 Learning: 2025-09-15T13:16:05.355Z
Learnt from: CR PR: theopenco/llmgateway#0 File: AGENTS.md:0-0 Timestamp: 2025-09-15T13:16:05.355Z Learning: Applies to apps/gateway/src/api.e2e.ts : Put parallelizable .each() E2E tests in apps/gateway/src/api.e2e.ts (uses concurrent mode)Applied to files:
apps/gateway/src/chat-streaming.e2e.tsapps/gateway/src/chat-reasoning.e2e.tsapps/gateway/src/chat-toolcalls.e2e.tsapps/gateway/src/chat-full.e2e.tsapps/gateway/src/chat-rs.e2e.tsapps/gateway/src/chat-toolcalls-result.e2e.tsapps/gateway/src/chat-api.e2e.tsapps/gateway/src/api-individual.e2e.ts📚 Learning: 2025-09-15T13:15:00.724Z
Learnt from: CR PR: theopenco/llmgateway#0 File: CLAUDE.md:0-0 Timestamp: 2025-09-15T13:15:00.724Z Learning: Applies to apps/gateway/src/api.e2e.ts : Put parallelized .each() E2E tests in apps/gateway/src/api.e2e.ts (uses concurrent mode)Applied to files:
apps/gateway/src/chat-streaming.e2e.tsapps/gateway/src/chat-reasoning.e2e.tsapps/gateway/src/chat-toolcalls.e2e.tsapps/gateway/src/chat-full.e2e.tsapps/gateway/src/chat-rs.e2e.tsapps/gateway/src/chat-toolcalls-result.e2e.tsapps/gateway/src/chat-api.e2e.tsapps/gateway/src/api-individual.e2e.ts📚 Learning: 2025-09-15T13:16:05.355Z
Learnt from: CR PR: theopenco/llmgateway#0 File: AGENTS.md:0-0 Timestamp: 2025-09-15T13:16:05.355Z Learning: Applies to apps/gateway/src/api-individual.e2e.ts : Put isolated E2E test cases in apps/gateway/src/api-individual.e2e.tsApplied to files:
apps/gateway/src/chat-streaming.e2e.tsapps/gateway/src/chat-reasoning.e2e.tsapps/gateway/src/chat-toolcalls.e2e.tsapps/gateway/src/chat-full.e2e.tsapps/gateway/src/chat-rs.e2e.tsapps/gateway/src/chat-toolcalls-result.e2e.tsapps/gateway/src/chat-api.e2e.tsapps/gateway/src/api-individual.e2e.ts📚 Learning: 2025-09-15T13:16:05.355Z
Learnt from: CR PR: theopenco/llmgateway#0 File: AGENTS.md:0-0 Timestamp: 2025-09-15T13:16:05.355Z Learning: Applies to **/*.e2e.ts : Place end-to-end tests in files named *.e2e.tsApplied to files:
apps/gateway/src/chat-streaming.e2e.tsapps/gateway/src/chat-reasoning.e2e.tsapps/gateway/src/chat-toolcalls.e2e.tsapps/gateway/src/chat-full.e2e.tsapps/gateway/src/chat-rs.e2e.tsapps/gateway/src/chat-api.e2e.ts📚 Learning: 2025-09-15T13:15:00.724Z
Learnt from: CR PR: theopenco/llmgateway#0 File: CLAUDE.md:0-0 Timestamp: 2025-09-15T13:15:00.724Z Learning: Applies to **/*.e2e.ts : Place end-to-end tests in files matching *.e2e.tsApplied to files:
apps/gateway/src/chat-streaming.e2e.tsapps/gateway/src/chat-reasoning.e2e.tsapps/gateway/src/chat-toolcalls.e2e.tsapps/gateway/src/chat-full.e2e.ts🧬 Code graph analysis (7)
apps/gateway/src/chat-streaming.e2e.ts (3)
apps/gateway/src/chat-api.e2e.ts (7)
beforeAllHook(364-429)beforeEachHook(431-433)streamingModels(277-286)getTestOptions(33-35)generateTestRequestId(28-30)logMode(40-40)validateLogByRequestId(346-362)apps/gateway/src/test-utils/test-helpers.ts (1)
readAll(106-194)packages/db/src/schema.ts (1)
log(324-386)apps/gateway/src/chat-reasoning.e2e.ts (3)
apps/gateway/src/chat-api.e2e.ts (8)
beforeAllHook(364-429)beforeEachHook(431-433)reasoningModels(288-290)getTestOptions(33-35)generateTestRequestId(28-30)logMode(40-40)validateResponse(338-344)validateLogByRequestId(346-362)packages/db/src/schema.ts (1)
log(324-386)packages/models/src/models.ts (1)
ProviderModelMapping(23-100)apps/gateway/src/chat-toolcalls.e2e.ts (3)
apps/gateway/src/chat-api.e2e.ts (7)
beforeAllHook(364-429)beforeEachHook(431-433)toolCallModels(303-305)getTestOptions(33-35)generateTestRequestId(28-30)logMode(40-40)validateLogByRequestId(346-362)packages/db/src/schema.ts (2)
message(483-503)log(324-386)packages/db/src/types.ts (1)
toolCall(35-42)apps/gateway/src/chat-full.e2e.ts (4)
apps/gateway/src/chat-api.e2e.ts (8)
beforeAllHook(364-429)beforeEachHook(431-433)fullMode(39-39)testModels(164-222)getTestOptions(33-35)generateTestRequestId(28-30)logMode(40-40)validateLogByRequestId(346-362)packages/models/src/models.ts (1)
ProviderModelMapping(23-100)packages/db/src/schema.ts (2)
message(483-503)log(324-386)packages/db/src/types.ts (1)
toolCall(35-42)apps/gateway/src/chat-rs.e2e.ts (4)
apps/gateway/src/chat-api.e2e.ts (7)
beforeAllHook(364-429)beforeEachHook(431-433)streamingReasoningModels(292-301)getTestOptions(33-35)generateTestRequestId(28-30)logMode(40-40)validateLogByRequestId(346-362)apps/gateway/src/test-utils/test-helpers.ts (1)
readAll(106-194)packages/models/src/models.ts (1)
ProviderModelMapping(23-100)packages/db/src/schema.ts (1)
log(324-386)apps/gateway/src/chat-toolcalls-result.e2e.ts (2)
apps/gateway/src/chat-api.e2e.ts (7)
beforeAllHook(364-429)beforeEachHook(431-433)toolCallModels(303-305)getTestOptions(33-35)generateTestRequestId(28-30)logMode(40-40)validateLogByRequestId(346-362)packages/db/src/schema.ts (2)
message(483-503)log(324-386)apps/gateway/src/chat-api.e2e.ts (4)
packages/models/src/models.ts (2)
ProviderModelMapping(23-100)ModelDefinition(104-157)packages/models/src/providers.ts (1)
providers(19-236)packages/db/src/schema.ts (2)
log(324-386)message(483-503)apps/gateway/src/test-utils/test-helpers.ts (3)
waitForLogByRequestId(59-99)clearCache(9-11)readAll(106-194)🪛 ESLint
apps/gateway/src/chat-streaming.e2e.ts
[error] 1-1: Resolve error: EACCES: permission denied, open '/dnMnRzZziC'
at Object.writeFileSync (node:fs:2409:20)
at l (/home/jailuser/git/node_modules/.pnpm/get-tsconfig@4.10.1/node_modules/get-tsconfig/dist/index.cjs:7:13685)
at createFilesMatcher (/home/jailuser/git/node_modules/.pnpm/get-tsconfig@4.10.1/node_modules/get-tsconfig/dist/index.cjs:7:14437)
at Object.resolve (/home/jailuser/git/node_modules/.pnpm/eslint-import-resolver-typescript@4.4.4_eslint-plugin-import@2.32.0_eslint@9.34.0_jiti@2.5.1_/node_modules/eslint-import-resolver-typescript/lib/index.cjs:298:107)
at withResolver (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:180:23)
at fullResolve (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:201:22)
at relative (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:217:10)
at resolve (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:233:12)
at resolveImportType (/home/jailuser/git/node_modules/.pnpm/eslint-plugin-import@2.32.0_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1___504bceda7a403f00735dcb51d6a813c7/node_modules/eslint-plugin-import/lib/core/importType.js:126:2822)
at computeRank (/home/jailuser/git/node_modules/.pnpm/eslint-plugin-import@2.32.0_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1___504bceda7a403f00735dcb51d6a813c7/node_modules/eslint-plugin-import/lib/rules/order.js:529:43)(import/order)
apps/gateway/src/chat-reasoning.e2e.ts
[error] 1-1: Resolve error: EACCES: permission denied, open '/VzGwYQMwMu'
at Object.writeFileSync (node:fs:2409:20)
at l (/home/jailuser/git/node_modules/.pnpm/get-tsconfig@4.10.1/node_modules/get-tsconfig/dist/index.cjs:7:13685)
at createFilesMatcher (/home/jailuser/git/node_modules/.pnpm/get-tsconfig@4.10.1/node_modules/get-tsconfig/dist/index.cjs:7:14437)
at Object.resolve (/home/jailuser/git/node_modules/.pnpm/eslint-import-resolver-typescript@4.4.4_eslint-plugin-import@2.32.0_eslint@9.34.0_jiti@2.5.1_/node_modules/eslint-import-resolver-typescript/lib/index.cjs:298:107)
at withResolver (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:180:23)
at fullResolve (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:201:22)
at relative (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:217:10)
at resolve (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:233:12)
at resolveImportType (/home/jailuser/git/node_modules/.pnpm/eslint-plugin-import@2.32.0_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1___504bceda7a403f00735dcb51d6a813c7/node_modules/eslint-plugin-import/lib/core/importType.js:126:2822)
at computeRank (/home/jailuser/git/node_modules/.pnpm/eslint-plugin-import@2.32.0_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1___504bceda7a403f00735dcb51d6a813c7/node_modules/eslint-plugin-import/lib/rules/order.js:529:43)(import/order)
apps/gateway/src/chat-toolcalls.e2e.ts
[error] 1-1: Resolve error: EACCES: permission denied, open '/MADuDEEzsq'
at Object.writeFileSync (node:fs:2409:20)
at l (/home/jailuser/git/node_modules/.pnpm/get-tsconfig@4.10.1/node_modules/get-tsconfig/dist/index.cjs:7:13685)
at createFilesMatcher (/home/jailuser/git/node_modules/.pnpm/get-tsconfig@4.10.1/node_modules/get-tsconfig/dist/index.cjs:7:14437)
at Object.resolve (/home/jailuser/git/node_modules/.pnpm/eslint-import-resolver-typescript@4.4.4_eslint-plugin-import@2.32.0_eslint@9.34.0_jiti@2.5.1_/node_modules/eslint-import-resolver-typescript/lib/index.cjs:298:107)
at withResolver (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:180:23)
at fullResolve (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:201:22)
at relative (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:217:10)
at resolve (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:233:12)
at resolveImportType (/home/jailuser/git/node_modules/.pnpm/eslint-plugin-import@2.32.0_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1___504bceda7a403f00735dcb51d6a813c7/node_modules/eslint-plugin-import/lib/core/importType.js:126:2822)
at computeRank (/home/jailuser/git/node_modules/.pnpm/eslint-plugin-import@2.32.0_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1___504bceda7a403f00735dcb51d6a813c7/node_modules/eslint-plugin-import/lib/rules/order.js:529:43)(import/order)
apps/gateway/src/chat-full.e2e.ts
[error] 1-1: Resolve error: EACCES: permission denied, open '/EMFaZpObMx'
at Object.writeFileSync (node:fs:2409:20)
at l (/home/jailuser/git/node_modules/.pnpm/get-tsconfig@4.10.1/node_modules/get-tsconfig/dist/index.cjs:7:13685)
at createFilesMatcher (/home/jailuser/git/node_modules/.pnpm/get-tsconfig@4.10.1/node_modules/get-tsconfig/dist/index.cjs:7:14437)
at Object.resolve (/home/jailuser/git/node_modules/.pnpm/eslint-import-resolver-typescript@4.4.4_eslint-plugin-import@2.32.0_eslint@9.34.0_jiti@2.5.1_/node_modules/eslint-import-resolver-typescript/lib/index.cjs:298:107)
at withResolver (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:180:23)
at fullResolve (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:201:22)
at relative (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:217:10)
at resolve (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:233:12)
at resolveImportType (/home/jailuser/git/node_modules/.pnpm/eslint-plugin-import@2.32.0_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1___504bceda7a403f00735dcb51d6a813c7/node_modules/eslint-plugin-import/lib/core/importType.js:126:2822)
at computeRank (/home/jailuser/git/node_modules/.pnpm/eslint-plugin-import@2.32.0_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1___504bceda7a403f00735dcb51d6a813c7/node_modules/eslint-plugin-import/lib/rules/order.js:529:43)(import/order)
apps/gateway/src/chat-rs.e2e.ts
[error] 1-1: Resolve error: EACCES: permission denied, open '/TRglNdYsKP'
at Object.writeFileSync (node:fs:2409:20)
at l (/home/jailuser/git/node_modules/.pnpm/get-tsconfig@4.10.1/node_modules/get-tsconfig/dist/index.cjs:7:13685)
at createFilesMatcher (/home/jailuser/git/node_modules/.pnpm/get-tsconfig@4.10.1/node_modules/get-tsconfig/dist/index.cjs:7:14437)
at Object.resolve (/home/jailuser/git/node_modules/.pnpm/eslint-import-resolver-typescript@4.4.4_eslint-plugin-import@2.32.0_eslint@9.34.0_jiti@2.5.1_/node_modules/eslint-import-resolver-typescript/lib/index.cjs:298:107)
at withResolver (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:180:23)
at fullResolve (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:201:22)
at relative (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:217:10)
at resolve (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:233:12)
at resolveImportType (/home/jailuser/git/node_modules/.pnpm/eslint-plugin-import@2.32.0_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1___504bceda7a403f00735dcb51d6a813c7/node_modules/eslint-plugin-import/lib/core/importType.js:126:2822)
at computeRank (/home/jailuser/git/node_modules/.pnpm/eslint-plugin-import@2.32.0_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1___504bceda7a403f00735dcb51d6a813c7/node_modules/eslint-plugin-import/lib/rules/order.js:529:43)(import/order)
apps/gateway/src/chat-toolcalls-result.e2e.ts
[error] 1-1: Resolve error: EACCES: permission denied, open '/STLzzbIYfL'
at Object.writeFileSync (node:fs:2409:20)
at l (/home/jailuser/git/node_modules/.pnpm/get-tsconfig@4.10.1/node_modules/get-tsconfig/dist/index.cjs:7:13685)
at createFilesMatcher (/home/jailuser/git/node_modules/.pnpm/get-tsconfig@4.10.1/node_modules/get-tsconfig/dist/index.cjs:7:14437)
at Object.resolve (/home/jailuser/git/node_modules/.pnpm/eslint-import-resolver-typescript@4.4.4_eslint-plugin-import@2.32.0_eslint@9.34.0_jiti@2.5.1_/node_modules/eslint-import-resolver-typescript/lib/index.cjs:298:107)
at withResolver (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:180:23)
at fullResolve (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:201:22)
at relative (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:217:10)
at resolve (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:233:12)
at resolveImportType (/home/jailuser/git/node_modules/.pnpm/eslint-plugin-import@2.32.0_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1___504bceda7a403f00735dcb51d6a813c7/node_modules/eslint-plugin-import/lib/core/importType.js:126:2822)
at computeRank (/home/jailuser/git/node_modules/.pnpm/eslint-plugin-import@2.32.0_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1___504bceda7a403f00735dcb51d6a813c7/node_modules/eslint-plugin-import/lib/rules/order.js:529:43)(import/order)
apps/gateway/src/chat-api.e2e.ts
[error] 1-1: Resolve error: EACCES: permission denied, open '/BIAjYtnSrS'
at Object.writeFileSync (node:fs:2409:20)
at l (/home/jailuser/git/node_modules/.pnpm/get-tsconfig@4.10.1/node_modules/get-tsconfig/dist/index.cjs:7:13685)
at createFilesMatcher (/home/jailuser/git/node_modules/.pnpm/get-tsconfig@4.10.1/node_modules/get-tsconfig/dist/index.cjs:7:14437)
at Object.resolve (/home/jailuser/git/node_modules/.pnpm/eslint-import-resolver-typescript@4.4.4_eslint-plugin-import@2.32.0_eslint@9.34.0_jiti@2.5.1_/node_modules/eslint-import-resolver-typescript/lib/index.cjs:298:107)
at withResolver (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:180:23)
at fullResolve (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:201:22)
at relative (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:217:10)
at resolve (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:233:12)
at resolveImportType (/home/jailuser/git/node_modules/.pnpm/eslint-plugin-import@2.32.0_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1___504bceda7a403f00735dcb51d6a813c7/node_modules/eslint-plugin-import/lib/core/importType.js:126:2822)
at computeRank (/home/jailuser/git/node_modules/.pnpm/eslint-plugin-import@2.32.0_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1___504bceda7a403f00735dcb51d6a813c7/node_modules/eslint-plugin-import/lib/rules/order.js:529:43)(import/order)
[error] 1-1: Resolve error: EACCES: permission denied, open '/xfRXmfKBuH'
at Object.writeFileSync (node:fs:2409:20)
at l (/home/jailuser/git/node_modules/.pnpm/get-tsconfig@4.10.1/node_modules/get-tsconfig/dist/index.cjs:7:13685)
at createFilesMatcher (/home/jailuser/git/node_modules/.pnpm/get-tsconfig@4.10.1/node_modules/get-tsconfig/dist/index.cjs:7:14437)
at Object.resolve (/home/jailuser/git/node_modules/.pnpm/eslint-import-resolver-typescript@4.4.4_eslint-plugin-import@2.32.0_eslint@9.34.0_jiti@2.5.1_/node_modules/eslint-import-resolver-typescript/lib/index.cjs:298:107)
at withResolver (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:180:23)
at fullResolve (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:201:22)
at relative (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:217:10)
at resolve (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:233:12)
at checkSourceValue (/home/jailuser/git/node_modules/.pnpm/eslint-plugin-import@2.32.0_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1___504bceda7a403f00735dcb51d6a813c7/node_modules/eslint-plugin-import/lib/rules/no-useless-path-segments.js:85:53)
at checkSourceValue (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/moduleVisitor.js:32:5)(import/no-useless-path-segments)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
- GitHub Check: build / run
- GitHub Check: e2e / run
- GitHub Check: test / run
🔇 Additional comments (9)
apps/gateway/src/chat-toolcalls.e2e.ts (1)
20-125: Solid tool-calls coverage and assertionsGood end-to-end checks for structure, finish_reason, usage, and logging.
apps/gateway/src/api-individual.e2e.ts (1)
21-539: Per-test data isolation looks goodUsing per-test IDs avoids collisions and enables parallelism. Provider-key IDs are also namespaced per test.
apps/gateway/src/chat-streaming.e2e.ts (1)
21-120: Streaming validations are thoroughSSE shape, OpenAI chunk format, usage in-stream, and log streamed=true are well covered.
apps/gateway/src/chat-api.e2e.ts (1)
364-429: Avoid cross-file DB wipes & static IDs — high flake risk with parallel E2E runnersRepo search returned no files; verification couldn't complete. Confirm and apply:
- Gate cross-file deletes behind CLEAN_E2E_DB === "true" in beforeAllHook (apps/gateway/src/chat-api.e2e.ts ~lines 364–429).
- Make setup inserts idempotent: use .onConflictDoNothing() (or equivalent upsert) for user, organization, userOrganization, project, apiKey.
- Avoid static IDs — generate per-run IDs or append worker/random suffix to prevent duplicate-key races.
- Make provider-key creation idempotent (upsert or find-or-create) to avoid duplicate inserts/races.
- Verify test-runner parallel settings (maxWorkers/maxConcurrency/workers/sequence) and search repo for CLEAN_E2E_DB and onConflictDoNothing usage.
apps/gateway/src/chat-full.e2e.ts (1)
30-34: Nice provider/model filteringSelecting only models that support both reasoning and tools keeps this suite focused.
apps/gateway/src/chat-toolcalls-result.e2e.ts (1)
20-24: Good focused coverageThis suite neatly validates the “tool call followed by tool result” path across providers.
apps/gateway/src/chat-rs.e2e.ts (1)
83-93: Thorough OpenAI-format chunk validationGood shape checks on id/object/created/model and delta content.
apps/gateway/src/chat-reasoning.e2e.ts (2)
1-16: Imports and setup look correct.Top‑level ES imports only; aligns with repo guidelines. dotenv preload is fine for e2e.
51-58: LGTM on status and base validation.Happy path assertions are clear; logging is gated by env.
| export const imageModels = testModels.filter((m) => { | ||
| const model = models.find((mo) => m.originalModel === mo.id); | ||
| return (model as ModelDefinition).output?.includes("image"); | ||
| }); | ||
|
|
There was a problem hiding this comment.
imageModels can throw when TEST_ALL_VARIATIONS adds root entries
For root test cases, m.originalModel is undefined and models.find(...) returns undefined; accessing .output then throws.
Apply:
-export const imageModels = testModels.filter((m) => {
- const model = models.find((mo) => m.originalModel === mo.id);
- return (model as ModelDefinition).output?.includes("image");
-});
+export const imageModels = testModels.filter((m) => {
+ const defId =
+ m.originalModel ?? (typeof m.model === "string" ? m.model.split("/").pop()! : m.model);
+ const def = models.find((mo) => mo.id === defId);
+ return def?.output?.includes("image") ?? false;
+});📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| export const imageModels = testModels.filter((m) => { | |
| const model = models.find((mo) => m.originalModel === mo.id); | |
| return (model as ModelDefinition).output?.includes("image"); | |
| }); | |
| export const imageModels = testModels.filter((m) => { | |
| const defId = | |
| m.originalModel ?? (typeof m.model === "string" ? m.model.split("/").pop()! : m.model); | |
| const def = models.find((mo) => mo.id === defId); | |
| return def?.output?.includes("image") ?? false; | |
| }); |
🤖 Prompt for AI Agents
In apps/gateway/src/chat-api.e2e.ts around lines 307 to 311, the filter callback
can throw when m.originalModel is undefined or when models.find(...) returns
undefined because the code immediately accesses .output; change the predicate to
first check that m.originalModel is defined, then only call models.find, and
guard on the found model and its output (e.g., ensure model is truthy and
model.output is an array or use optional chaining) before calling
includes("image") so root test cases are skipped instead of throwing.
| export function validateResponse(json: any) { | ||
| expect(json).toHaveProperty("choices.[0].message.content"); | ||
|
|
||
| expect(json).toHaveProperty("usage.prompt_tokens"); | ||
| expect(json).toHaveProperty("usage.completion_tokens"); | ||
| expect(json).toHaveProperty("usage.total_tokens"); | ||
| } |
There was a problem hiding this comment.
💡 Verification agent
🧩 Analysis chain
toHaveProperty path is brittle
Jest/Vitest paths use "choices.0.message.content". Current "choices.[0]..." may not behave as intended.
-export function validateResponse(json: any) {
- expect(json).toHaveProperty("choices.[0].message.content");
+export function validateResponse(json: any) {
+ expect(json).toHaveProperty("choices.0.message.content");
expect(json).toHaveProperty("usage.prompt_tokens");
expect(json).toHaveProperty("usage.completion_tokens");
expect(json).toHaveProperty("usage.total_tokens");
}🏁 Script executed:
#!/bin/bash
# Find all brittle property-path usages to fix consistently
rg -nP "choices\.\[0\]\.message\.content|choices\.\[0\]\.message" --type ts --type tsxLength of output: 27
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "rg version:"
rg --version || true
echo
echo "Searching for bracketed 'choices.[0]' occurrences (showing file:line:match)..."
rg -nS "choices\.\[0\]" --hidden -g '!*node_modules/*' || true
echo
echo "Searching for 'toHaveProperty' lines that mention 'choices'..."
rg -nS "toHaveProperty.*choices" --hidden -g '!*node_modules/*' || true
echo
echo "Searching for dot-notation 'choices.0' occurrences..."
rg -nS "choices\.0" --hidden -g '!*node_modules/*' || true
echo
echo "Printing snippet of apps/gateway/src/chat-api.e2e.ts around reported lines (320-360) if present:"
if [ -f "apps/gateway/src/chat-api.e2e.ts" ]; then
sed -n '320,360p' apps/gateway/src/chat-api.e2e.ts || true
else
echo "File not found: apps/gateway/src/chat-api.e2e.ts"
fiLength of output: 1990
Fix brittle toHaveProperty path in validateResponse
Use a valid deep-path — toHaveProperty accepts dot-index (choices.0.message.content) or bracket-index (choices[0].message.content); "choices.[0].message.content" is malformed/brittle and may fail. (jestjs.io)
File: apps/gateway/src/chat-api.e2e.ts (lines ~338-344)
-export function validateResponse(json: any) {
- expect(json).toHaveProperty("choices.[0].message.content");
+export function validateResponse(json: any) {
+ expect(json).toHaveProperty("choices.0.message.content");
expect(json).toHaveProperty("usage.prompt_tokens");
expect(json).toHaveProperty("usage.completion_tokens");
expect(json).toHaveProperty("usage.total_tokens");
}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| export function validateResponse(json: any) { | |
| expect(json).toHaveProperty("choices.[0].message.content"); | |
| expect(json).toHaveProperty("usage.prompt_tokens"); | |
| expect(json).toHaveProperty("usage.completion_tokens"); | |
| expect(json).toHaveProperty("usage.total_tokens"); | |
| } | |
| export function validateResponse(json: any) { | |
| expect(json).toHaveProperty("choices.0.message.content"); | |
| expect(json).toHaveProperty("usage.prompt_tokens"); | |
| expect(json).toHaveProperty("usage.completion_tokens"); | |
| expect(json).toHaveProperty("usage.total_tokens"); | |
| } |
🤖 Prompt for AI Agents
In apps/gateway/src/chat-api.e2e.ts around lines 338 to 344, the toHaveProperty
path "choices.[0].message.content" is malformed and brittle; replace it with a
valid deep-path such as "choices.0.message.content" (or
"choices[0].message.content") so the Jest expectation correctly checks the first
choice's message content; keep the remaining usage property assertions as-is.
| test.each( | ||
| testModels.filter((m) => { | ||
| const modelDef = models.find((def) => def.id === m.model); | ||
| return (modelDef as ModelDefinition)?.jsonOutput === true; | ||
| }), |
There was a problem hiding this comment.
JSON-output filter compares against provider-qualified id; yields empty suite
m.model is "provider/model". ModelDefinition.id is just "model". Use originalModel or strip prefix.
-test.each(
- testModels.filter((m) => {
- const modelDef = models.find((def) => def.id === m.model);
- return (modelDef as ModelDefinition)?.jsonOutput === true;
- }),
+test.each(
+ testModels.filter((m) => {
+ const defId = m.originalModel ?? (typeof m.model === "string" ? m.model.split("/").pop()! : m.model);
+ const modelDef = models.find((def) => def.id === defId);
+ return modelDef?.jsonOutput === true;
+ }),📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| test.each( | |
| testModels.filter((m) => { | |
| const modelDef = models.find((def) => def.id === m.model); | |
| return (modelDef as ModelDefinition)?.jsonOutput === true; | |
| }), | |
| test.each( | |
| testModels.filter((m) => { | |
| const defId = m.originalModel ?? (typeof m.model === "string" ? m.model.split("/").pop()! : m.model); | |
| const modelDef = models.find((def) => def.id === defId); | |
| return modelDef?.jsonOutput === true; | |
| }), |
🤖 Prompt for AI Agents
In apps/gateway/src/chat-api.e2e.ts around lines 495 to 499, the filter uses
m.model (which can be provider-qualified like "provider/model") to compare
against ModelDefinition.id (which is just "model"), causing the suite to be
empty; change the comparison to use the unqualified model id by either using
m.originalModel if present or strip the provider prefix (e.g., split on '/' and
take the last segment) before comparing to modelDef.id, and keep
types/null-checks so the filter still works when originalModel is undefined.
There was a problem hiding this comment.
Actionable comments posted: 3
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
apps/gateway/src/api-individual.e2e.ts (1)
74-76: Avoid global FLUSHDB per testSame parallelism hazard as in helpers; either gate it or remove.
-beforeEach(async () => { - await clearCache(); -}); +beforeEach(async () => { + // Avoid global FLUSHDB in parallel runs; enable only when explicitly needed. + if (process.env.CLEAR_CACHE_EACH === "true") { + await clearCache(); + } +});
♻️ Duplicate comments (5)
apps/gateway/src/chat-toolcalls.e2e.ts (1)
16-18: Per‑test cache flush duplicates the global issueThis suite calls beforeEachHook which FLUSHDBs; see helper comment.
Confirm after applying the helper change that no tests rely on global cache state between steps.
apps/gateway/src/chat-reasoning.e2e.ts (1)
21-22: Per‑test cache flush duplicates the global issueThis suite depends on beforeEachHook which FLUSHDBs; see helper fix.
apps/gateway/src/chat-api.e2e.ts (3)
341-347: Fix brittle toHaveProperty path-export function validateResponse(json: any) { - expect(json).toHaveProperty("choices.[0].message.content"); +export function validateResponse(json: any) { + expect(json).toHaveProperty("choices.0.message.content");
307-310: imageModels filter can throw and misses root entries; derive model id safely-export const imageModels = testModels.filter((m) => { - const model = models.find((mo) => m.originalModel === mo.id); - return (model as ModelDefinition).output?.includes("image"); -}); +export const imageModels = testModels.filter((m) => { + const defId = + m.originalModel ?? + (typeof m.model === "string" ? m.model.split("/").pop()! : (m.model as string)); + const def = models.find((mo) => mo.id === defId); + return def?.output?.includes("image") ?? false; +});
493-497: jsonOutput filter compares provider-qualified id; suite becomes empty-test.each( - testModels.filter((m) => { - const modelDef = models.find((def) => def.id === m.model); - return (modelDef as ModelDefinition)?.jsonOutput === true; - }), +test.each( + testModels.filter((m) => { + const defId = + m.originalModel ?? + (typeof m.model === "string" ? m.model.split("/").pop()! : (m.model as string)); + const modelDef = models.find((def) => def.id === defId); + return modelDef?.jsonOutput === true; + }),
🧹 Nitpick comments (19)
apps/gateway/src/chat-helpers.e2e.ts (4)
19-21: Use a collision‑proof requestIdPrefer crypto.randomUUID() to avoid rare collisions and remove legacy substr.
Apply:
-export function generateTestRequestId(): string { - return `test-${Date.now()}-${Math.random().toString(36).substr(2, 9)}`; -} +export function generateTestRequestId(): string { + return `test-${crypto.randomUUID()}`; +}Add at top of file:
import crypto from "node:crypto";
28-28: Reduce default test log noiseGate the options log behind a DEBUG flag.
-console.log("running with test options:", getTestOptions()); +if (process.env.DEBUG_E2E) { + console.log("running with test options:", getTestOptions()); +}
333-338: Fragile toHaveProperty path"choices.[0].message.content" is nonstandard; use bracket or index form.
-export function validateResponse(json: any) { - expect(json).toHaveProperty("choices.[0].message.content"); +export function validateResponse(json: any) { + expect(json).toHaveProperty("choices[0].message.content");
424-428: Drop the empty testThis file is a helper; the empty test adds noise and time.
-describe("e2e", { concurrent: true }, () => { - it("empty", () => { - expect(true).toBe(true); - }); -});apps/gateway/src/chat-toolcalls.e2e.ts (1)
106-108: Finish reason may vary across providersSome providers might not return "tool_calls" as finish_reason even with tools. Consider asserting presence of tool_calls and allowing finish_reason in {"tool_calls","stop"}.
-expect(json.choices[0]).toHaveProperty("finish_reason", "tool_calls"); +expect(json.choices[0]).toHaveProperty("finish_reason"); +expect(["tool_calls","stop"]).toContain(json.choices[0].finish_reason);apps/gateway/src/api-individual.e2e.ts (7)
13-13: Use the same index import path as other suitesKeeps imports consistent and avoids resolver ambiguities.
-import { app } from "."; +import { app } from "@/index";
93-99: Fix toHaveProperty pathUse bracket/index form.
-function validateResponse(json: any) { - expect(json).toHaveProperty("choices.[0].message.content"); +function validateResponse(json: any) { + expect(json).toHaveProperty("choices[0].message.content");
245-246: Fix toHaveProperty path-expect(json).toHaveProperty("choices.[0].message.content"); +expect(json).toHaveProperty("choices[0].message.content");
286-288: Fix toHaveProperty path-const json = await res.json(); -expect(json).toHaveProperty("choices.[0].message.content"); +const json = await res.json(); +expect(json).toHaveProperty("choices[0].message.content");
353-355: Fix toHaveProperty path-const json = await res.json(); -expect(json).toHaveProperty("choices.[0].message.content"); +const json = await res.json(); +expect(json).toHaveProperty("choices[0].message.content");
362-407: Gate ZERO_TOKENS (non‑stream) test on OpenAI keyPrevents false failures when OPENAI key is absent.
test("Prompt tokens are never zero even when provider returns 0", async () => { - const { token } = await createTestData("zero-tokens"); + const envVarName = getProviderEnvVar("openai"); + const envVarValue = envVarName ? process.env[envVarName] : undefined; + if (!envVarValue) { + console.log("Skipping ZERO_TOKENS non-stream test - no OpenAI API key provided"); + return; + } + const { token } = await createTestData("zero-tokens");
409-443: Gate ZERO_TOKENS streaming test on OpenAI keySame rationale as above.
test("Prompt tokens are calculated for streaming when provider returns 0", async () => { - const { token } = await createTestData("zero-tokens-streaming"); + const envVarName = getProviderEnvVar("openai"); + const envVarValue = envVarName ? process.env[envVarName] : undefined; + if (!envVarValue) { + console.log("Skipping ZERO_TOKENS streaming test - no OpenAI API key provided"); + return; + } + const { token } = await createTestData("zero-tokens-streaming");apps/gateway/src/chat-streaming.e2e.ts (1)
90-111: Usage-in-stream may be provider-dependent; add fallback to DB log tokensSome providers don’t emit usage in SSE. If usageChunks is empty, assert via log tokens instead of failing the test.
- expect(usageChunks.length).toBeGreaterThan(0); + if (usageChunks.length === 0) { + const log = await validateLogByRequestId(requestId); + expect(Number(log.totalTokens ?? 0)).toBeGreaterThan(0); + } else { + expect(usageChunks.length).toBeGreaterThan(0); + }apps/gateway/src/chat-full.e2e.ts (1)
119-124: Guard tool_call.arguments parsing for providers that may return structured argsIf a provider returns an object instead of a JSON string, JSON.parse will throw. Defensive parse avoids spurious failures.
- const args = JSON.parse(toolCall.function.arguments); + const rawArgs = toolCall.function.arguments; + const args = + typeof rawArgs === "string" ? JSON.parse(rawArgs) : rawArgs ?? {};apps/gateway/src/chat-api.e2e.ts (3)
19-19: Align app import with other tests-import { app } from "."; +import { app } from "@/index";
27-71: Deduplicate helpers by using shared chat-helpers.e2e exports
Current file re-implements helpers and test-matrix derivations that already live in chat-helpers.e2e.ts, increasing drift risk. Prefer importing the shared exports and deleting the local copies.Do you want a follow-up patch to replace the local helper/matrix code with imports from "@/chat-helpers.e2e"?
Also applies to: 164-223, 224-276, 323-339, 367-431
770-774: Token-sum equality may not hold across providers
Some providers return non-integer or rounded totals; strict equality can be flaky. Consider ≥ instead of exact sum, or compare within tolerance.-expect(json.usage.total_tokens).toEqual( - json.usage.prompt_tokens + - json.usage.completion_tokens + - (json.usage.reasoning_tokens || 0), -); +expect(json.usage.total_tokens).toBeGreaterThanOrEqual( + json.usage.prompt_tokens + + json.usage.completion_tokens + + (json.usage.reasoning_tokens || 0), +);apps/gateway/src/chat-toolcalls-result.e2e.ts (1)
112-116: Content-or-tool_calls check: assert presence without relying on truthiness of empty stringEmpty string content is falsy. Make the intent explicit.
-// verify either content is string or tool_calls is present -expect(message.content || message.tool_calls).toBeTruthy(); +// verify either content is a non-empty string or tool_calls exists +expect( + (typeof message.content === "string" && message.content.length > 0) || + Array.isArray(message.tool_calls), +).toBe(true);apps/gateway/src/chat-rs.e2e.ts (1)
124-141: Reasoning content gating looks sensible; consider also asserting at-least-once assistant role chunkOptional hardening similar to streaming test to ensure role is emitted once.
+const roleChunks = streamResult.chunks.filter( + (c: any) => c.choices?.[0]?.delta?.role === "assistant", +); +expect(roleChunks.length).toBeGreaterThan(0);
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (10)
apps/gateway/src/api-individual.e2e.ts(1 hunks)apps/gateway/src/chat-api.e2e.ts(1 hunks)apps/gateway/src/chat-full.e2e.ts(1 hunks)apps/gateway/src/chat-helpers.e2e.ts(1 hunks)apps/gateway/src/chat-reasoning.e2e.ts(1 hunks)apps/gateway/src/chat-rs.e2e.ts(1 hunks)apps/gateway/src/chat-streaming.e2e.ts(1 hunks)apps/gateway/src/chat-toolcalls-result.e2e.ts(1 hunks)apps/gateway/src/chat-toolcalls.e2e.ts(1 hunks)package.json(1 hunks)
🧰 Additional context used
📓 Path-based instructions (7)
**/*.{ts,tsx}
📄 CodeRabbit inference engine (CLAUDE.md)
Always use top-level import; never use require() or dynamic imports
Files:
apps/gateway/src/chat-helpers.e2e.tsapps/gateway/src/chat-toolcalls-result.e2e.tsapps/gateway/src/api-individual.e2e.tsapps/gateway/src/chat-streaming.e2e.tsapps/gateway/src/chat-api.e2e.tsapps/gateway/src/chat-full.e2e.tsapps/gateway/src/chat-reasoning.e2e.tsapps/gateway/src/chat-rs.e2e.tsapps/gateway/src/chat-toolcalls.e2e.ts
{apps/{api,gateway}/src,packages/db}/**/*.ts?(x)
📄 CodeRabbit inference engine (CLAUDE.md)
Use Drizzle ORM with the latest object syntax
Files:
apps/gateway/src/chat-helpers.e2e.tsapps/gateway/src/chat-toolcalls-result.e2e.tsapps/gateway/src/api-individual.e2e.tsapps/gateway/src/chat-streaming.e2e.tsapps/gateway/src/chat-api.e2e.tsapps/gateway/src/chat-full.e2e.tsapps/gateway/src/chat-reasoning.e2e.tsapps/gateway/src/chat-rs.e2e.tsapps/gateway/src/chat-toolcalls.e2e.ts
apps/{api,gateway}/src/**/*.{ts,tsx}
📄 CodeRabbit inference engine (CLAUDE.md)
For read queries, use db().query.
.findMany() or db().query..findFirst() Files:
apps/gateway/src/chat-helpers.e2e.tsapps/gateway/src/chat-toolcalls-result.e2e.tsapps/gateway/src/api-individual.e2e.tsapps/gateway/src/chat-streaming.e2e.tsapps/gateway/src/chat-api.e2e.tsapps/gateway/src/chat-full.e2e.tsapps/gateway/src/chat-reasoning.e2e.tsapps/gateway/src/chat-rs.e2e.tsapps/gateway/src/chat-toolcalls.e2e.ts**/*.e2e.ts
📄 CodeRabbit inference engine (CLAUDE.md)
Place end-to-end tests in files matching *.e2e.ts
Place end-to-end tests in files named *.e2e.ts
Files:
apps/gateway/src/chat-helpers.e2e.tsapps/gateway/src/chat-toolcalls-result.e2e.tsapps/gateway/src/api-individual.e2e.tsapps/gateway/src/chat-streaming.e2e.tsapps/gateway/src/chat-api.e2e.tsapps/gateway/src/chat-full.e2e.tsapps/gateway/src/chat-reasoning.e2e.tsapps/gateway/src/chat-rs.e2e.tsapps/gateway/src/chat-toolcalls.e2e.ts{apps/api,apps/gateway,apps/ui,apps/docs,packages}/**/*.{ts,tsx}
📄 CodeRabbit inference engine (AGENTS.md)
Always use top-level import; never use require() or dynamic imports (e.g., import(), next/dynamic)
Files:
apps/gateway/src/chat-helpers.e2e.tsapps/gateway/src/chat-toolcalls-result.e2e.tsapps/gateway/src/api-individual.e2e.tsapps/gateway/src/chat-streaming.e2e.tsapps/gateway/src/chat-api.e2e.tsapps/gateway/src/chat-full.e2e.tsapps/gateway/src/chat-reasoning.e2e.tsapps/gateway/src/chat-rs.e2e.tsapps/gateway/src/chat-toolcalls.e2e.ts{apps/api,apps/gateway,packages/db}/**/*.ts
📄 CodeRabbit inference engine (AGENTS.md)
{apps/api,apps/gateway,packages/db}/**/*.ts: Use Drizzle ORM with the latest object syntax for database access
For reads, use db().query..findMany() or db().query.
.findFirst() Files:
apps/gateway/src/chat-helpers.e2e.tsapps/gateway/src/chat-toolcalls-result.e2e.tsapps/gateway/src/api-individual.e2e.tsapps/gateway/src/chat-streaming.e2e.tsapps/gateway/src/chat-api.e2e.tsapps/gateway/src/chat-full.e2e.tsapps/gateway/src/chat-reasoning.e2e.tsapps/gateway/src/chat-rs.e2e.tsapps/gateway/src/chat-toolcalls.e2e.tsapps/gateway/src/api-individual.e2e.ts
📄 CodeRabbit inference engine (CLAUDE.md)
Put isolated E2E tests in apps/gateway/src/api-individual.e2e.ts
Put isolated E2E test cases in apps/gateway/src/api-individual.e2e.ts
Files:
apps/gateway/src/api-individual.e2e.ts🧠 Learnings (7)
📓 Common learnings
Learnt from: CR PR: theopenco/llmgateway#0 File: CLAUDE.md:0-0 Timestamp: 2025-09-15T13:15:00.724Z Learning: Applies to apps/gateway/src/api-individual.e2e.ts : Put isolated E2E tests in apps/gateway/src/api-individual.e2e.tsLearnt from: CR PR: theopenco/llmgateway#0 File: AGENTS.md:0-0 Timestamp: 2025-09-15T13:16:05.355Z Learning: Applies to apps/gateway/src/api-individual.e2e.ts : Put isolated E2E test cases in apps/gateway/src/api-individual.e2e.tsLearnt from: CR PR: theopenco/llmgateway#0 File: AGENTS.md:0-0 Timestamp: 2025-09-15T13:16:05.355Z Learning: Applies to apps/gateway/src/api.e2e.ts : Put parallelizable .each() E2E tests in apps/gateway/src/api.e2e.ts (uses concurrent mode)Learnt from: CR PR: theopenco/llmgateway#0 File: CLAUDE.md:0-0 Timestamp: 2025-09-15T13:15:00.724Z Learning: Applies to apps/gateway/src/api.e2e.ts : Put parallelized .each() E2E tests in apps/gateway/src/api.e2e.ts (uses concurrent mode)📚 Learning: 2025-09-15T13:15:00.724Z
Learnt from: CR PR: theopenco/llmgateway#0 File: CLAUDE.md:0-0 Timestamp: 2025-09-15T13:15:00.724Z Learning: Applies to apps/gateway/src/api-individual.e2e.ts : Put isolated E2E tests in apps/gateway/src/api-individual.e2e.tsApplied to files:
apps/gateway/src/chat-helpers.e2e.tsapps/gateway/src/chat-toolcalls-result.e2e.tsapps/gateway/src/api-individual.e2e.tsapps/gateway/src/chat-streaming.e2e.tsapps/gateway/src/chat-api.e2e.tsapps/gateway/src/chat-full.e2e.tsapps/gateway/src/chat-reasoning.e2e.tsapps/gateway/src/chat-rs.e2e.tsapps/gateway/src/chat-toolcalls.e2e.ts📚 Learning: 2025-09-15T13:16:05.355Z
Learnt from: CR PR: theopenco/llmgateway#0 File: AGENTS.md:0-0 Timestamp: 2025-09-15T13:16:05.355Z Learning: Applies to apps/gateway/src/api-individual.e2e.ts : Put isolated E2E test cases in apps/gateway/src/api-individual.e2e.tsApplied to files:
apps/gateway/src/chat-helpers.e2e.tsapps/gateway/src/chat-toolcalls-result.e2e.tsapps/gateway/src/api-individual.e2e.tsapps/gateway/src/chat-streaming.e2e.tsapps/gateway/src/chat-api.e2e.tsapps/gateway/src/chat-full.e2e.tsapps/gateway/src/chat-reasoning.e2e.tsapps/gateway/src/chat-rs.e2e.tsapps/gateway/src/chat-toolcalls.e2e.ts📚 Learning: 2025-09-15T13:15:00.724Z
Learnt from: CR PR: theopenco/llmgateway#0 File: CLAUDE.md:0-0 Timestamp: 2025-09-15T13:15:00.724Z Learning: Applies to apps/gateway/src/api.e2e.ts : Put parallelized .each() E2E tests in apps/gateway/src/api.e2e.ts (uses concurrent mode)Applied to files:
apps/gateway/src/chat-helpers.e2e.tsapps/gateway/src/chat-toolcalls-result.e2e.tsapps/gateway/src/api-individual.e2e.tsapps/gateway/src/chat-streaming.e2e.tsapps/gateway/src/chat-api.e2e.tsapps/gateway/src/chat-full.e2e.tsapps/gateway/src/chat-reasoning.e2e.tsapps/gateway/src/chat-rs.e2e.tsapps/gateway/src/chat-toolcalls.e2e.ts📚 Learning: 2025-09-15T13:16:05.355Z
Learnt from: CR PR: theopenco/llmgateway#0 File: AGENTS.md:0-0 Timestamp: 2025-09-15T13:16:05.355Z Learning: Applies to apps/gateway/src/api.e2e.ts : Put parallelizable .each() E2E tests in apps/gateway/src/api.e2e.ts (uses concurrent mode)Applied to files:
apps/gateway/src/chat-helpers.e2e.tsapps/gateway/src/chat-toolcalls-result.e2e.tsapps/gateway/src/api-individual.e2e.tsapps/gateway/src/chat-streaming.e2e.tsapps/gateway/src/chat-api.e2e.tsapps/gateway/src/chat-full.e2e.tsapps/gateway/src/chat-reasoning.e2e.tsapps/gateway/src/chat-rs.e2e.tsapps/gateway/src/chat-toolcalls.e2e.ts📚 Learning: 2025-09-15T13:16:05.355Z
Learnt from: CR PR: theopenco/llmgateway#0 File: AGENTS.md:0-0 Timestamp: 2025-09-15T13:16:05.355Z Learning: Applies to **/*.e2e.ts : Place end-to-end tests in files named *.e2e.tsApplied to files:
apps/gateway/src/chat-streaming.e2e.tsapps/gateway/src/chat-full.e2e.tsapps/gateway/src/chat-reasoning.e2e.tsapps/gateway/src/chat-toolcalls.e2e.ts📚 Learning: 2025-09-15T13:15:00.724Z
Learnt from: CR PR: theopenco/llmgateway#0 File: CLAUDE.md:0-0 Timestamp: 2025-09-15T13:15:00.724Z Learning: Applies to **/*.e2e.ts : Place end-to-end tests in files matching *.e2e.tsApplied to files:
apps/gateway/src/chat-streaming.e2e.tsapps/gateway/src/chat-full.e2e.tsapps/gateway/src/chat-reasoning.e2e.tsapps/gateway/src/chat-toolcalls.e2e.ts🧬 Code graph analysis (8)
apps/gateway/src/chat-helpers.e2e.ts (6)
packages/models/src/models.ts (2)
ProviderModelMapping(23-100)ModelDefinition(104-157)packages/models/src/providers.ts (1)
providers(19-236)packages/db/src/db.ts (1)
db(15-19)packages/db/src/index.ts (1)
tables(10-12)packages/db/src/schema.ts (1)
log(324-386)apps/gateway/src/test-utils/test-helpers.ts (2)
waitForLogByRequestId(59-99)clearCache(9-11)apps/gateway/src/chat-toolcalls-result.e2e.ts (1)
apps/gateway/src/chat-helpers.e2e.ts (7)
beforeAllHook(358-418)beforeEachHook(420-422)toolCallModels(294-296)getTestOptions(24-26)generateTestRequestId(19-21)logMode(31-31)validateLogByRequestId(340-356)apps/gateway/src/chat-streaming.e2e.ts (2)
apps/gateway/src/chat-helpers.e2e.ts (7)
beforeAllHook(358-418)beforeEachHook(420-422)streamingModels(268-277)getTestOptions(24-26)generateTestRequestId(19-21)logMode(31-31)validateLogByRequestId(340-356)apps/gateway/src/test-utils/test-helpers.ts (1)
readAll(106-194)apps/gateway/src/chat-api.e2e.ts (4)
packages/models/src/models.ts (2)
ProviderModelMapping(23-100)ModelDefinition(104-157)packages/models/src/providers.ts (1)
providers(19-236)packages/db/src/schema.ts (2)
log(324-386)message(483-503)apps/gateway/src/test-utils/test-helpers.ts (3)
waitForLogByRequestId(59-99)clearCache(9-11)readAll(106-194)apps/gateway/src/chat-full.e2e.ts (4)
apps/gateway/src/chat-helpers.e2e.ts (8)
beforeAllHook(358-418)beforeEachHook(420-422)fullMode(30-30)testModels(155-213)getTestOptions(24-26)generateTestRequestId(19-21)logMode(31-31)validateLogByRequestId(340-356)packages/models/src/models.ts (1)
ProviderModelMapping(23-100)packages/db/src/schema.ts (2)
message(483-503)log(324-386)packages/db/src/types.ts (1)
toolCall(35-42)apps/gateway/src/chat-reasoning.e2e.ts (2)
apps/gateway/src/chat-helpers.e2e.ts (8)
beforeAllHook(358-418)beforeEachHook(420-422)reasoningModels(279-281)getTestOptions(24-26)generateTestRequestId(19-21)logMode(31-31)validateResponse(332-338)validateLogByRequestId(340-356)packages/models/src/models.ts (1)
ProviderModelMapping(23-100)apps/gateway/src/chat-rs.e2e.ts (4)
apps/gateway/src/chat-helpers.e2e.ts (7)
beforeAllHook(358-418)beforeEachHook(420-422)streamingReasoningModels(283-292)getTestOptions(24-26)generateTestRequestId(19-21)logMode(31-31)validateLogByRequestId(340-356)apps/gateway/src/test-utils/test-helpers.ts (1)
readAll(106-194)packages/models/src/models.ts (1)
ProviderModelMapping(23-100)packages/db/src/schema.ts (1)
log(324-386)apps/gateway/src/chat-toolcalls.e2e.ts (3)
apps/gateway/src/chat-helpers.e2e.ts (7)
beforeAllHook(358-418)beforeEachHook(420-422)toolCallModels(294-296)getTestOptions(24-26)generateTestRequestId(19-21)logMode(31-31)validateLogByRequestId(340-356)packages/db/src/schema.ts (2)
message(483-503)log(324-386)packages/db/src/types.ts (1)
toolCall(35-42)🪛 ESLint
apps/gateway/src/chat-helpers.e2e.ts
[error] 1-1: Resolve error: EACCES: permission denied, open '/uHnchJAtVC'
at Object.writeFileSync (node:fs:2409:20)
at l (/home/jailuser/git/node_modules/.pnpm/get-tsconfig@4.10.1/node_modules/get-tsconfig/dist/index.cjs:7:13685)
at createFilesMatcher (/home/jailuser/git/node_modules/.pnpm/get-tsconfig@4.10.1/node_modules/get-tsconfig/dist/index.cjs:7:14437)
at Object.resolve (/home/jailuser/git/node_modules/.pnpm/eslint-import-resolver-typescript@4.4.4_eslint-plugin-import@2.32.0_eslint@9.34.0_jiti@2.5.1_/node_modules/eslint-import-resolver-typescript/lib/index.cjs:298:107)
at withResolver (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:180:23)
at fullResolve (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:201:22)
at relative (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:217:10)
at resolve (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:233:12)
at resolveImportType (/home/jailuser/git/node_modules/.pnpm/eslint-plugin-import@2.32.0_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1___504bceda7a403f00735dcb51d6a813c7/node_modules/eslint-plugin-import/lib/core/importType.js:126:2822)
at computeRank (/home/jailuser/git/node_modules/.pnpm/eslint-plugin-import@2.32.0_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1___504bceda7a403f00735dcb51d6a813c7/node_modules/eslint-plugin-import/lib/rules/order.js:529:43)(import/order)
[error] 1-1: Resolve error: EACCES: permission denied, open '/bVhNgoEuvh'
at Object.writeFileSync (node:fs:2409:20)
at l (/home/jailuser/git/node_modules/.pnpm/get-tsconfig@4.10.1/node_modules/get-tsconfig/dist/index.cjs:7:13685)
at createFilesMatcher (/home/jailuser/git/node_modules/.pnpm/get-tsconfig@4.10.1/node_modules/get-tsconfig/dist/index.cjs:7:14437)
at Object.resolve (/home/jailuser/git/node_modules/.pnpm/eslint-import-resolver-typescript@4.4.4_eslint-plugin-import@2.32.0_eslint@9.34.0_jiti@2.5.1_/node_modules/eslint-import-resolver-typescript/lib/index.cjs:298:107)
at withResolver (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:180:23)
at fullResolve (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:201:22)
at relative (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:217:10)
at resolve (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:233:12)
at checkSourceValue (/home/jailuser/git/node_modules/.pnpm/eslint-plugin-import@2.32.0_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1___504bceda7a403f00735dcb51d6a813c7/node_modules/eslint-plugin-import/lib/rules/no-useless-path-segments.js:85:53)
at checkSourceValue (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/moduleVisitor.js:32:5)(import/no-useless-path-segments)
apps/gateway/src/chat-toolcalls-result.e2e.ts
[error] 1-1: Resolve error: EACCES: permission denied, open '/EPvweckkWR'
at Object.writeFileSync (node:fs:2409:20)
at l (/home/jailuser/git/node_modules/.pnpm/get-tsconfig@4.10.1/node_modules/get-tsconfig/dist/index.cjs:7:13685)
at createFilesMatcher (/home/jailuser/git/node_modules/.pnpm/get-tsconfig@4.10.1/node_modules/get-tsconfig/dist/index.cjs:7:14437)
at Object.resolve (/home/jailuser/git/node_modules/.pnpm/eslint-import-resolver-typescript@4.4.4_eslint-plugin-import@2.32.0_eslint@9.34.0_jiti@2.5.1_/node_modules/eslint-import-resolver-typescript/lib/index.cjs:298:107)
at withResolver (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:180:23)
at fullResolve (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:201:22)
at relative (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:217:10)
at resolve (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:233:12)
at resolveImportType (/home/jailuser/git/node_modules/.pnpm/eslint-plugin-import@2.32.0_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1___504bceda7a403f00735dcb51d6a813c7/node_modules/eslint-plugin-import/lib/core/importType.js:126:2822)
at computeRank (/home/jailuser/git/node_modules/.pnpm/eslint-plugin-import@2.32.0_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1___504bceda7a403f00735dcb51d6a813c7/node_modules/eslint-plugin-import/lib/rules/order.js:529:43)(import/order)
apps/gateway/src/chat-streaming.e2e.ts
[error] 1-1: Resolve error: EACCES: permission denied, open '/ExUSNapeUa'
at Object.writeFileSync (node:fs:2409:20)
at l (/home/jailuser/git/node_modules/.pnpm/get-tsconfig@4.10.1/node_modules/get-tsconfig/dist/index.cjs:7:13685)
at createFilesMatcher (/home/jailuser/git/node_modules/.pnpm/get-tsconfig@4.10.1/node_modules/get-tsconfig/dist/index.cjs:7:14437)
at Object.resolve (/home/jailuser/git/node_modules/.pnpm/eslint-import-resolver-typescript@4.4.4_eslint-plugin-import@2.32.0_eslint@9.34.0_jiti@2.5.1_/node_modules/eslint-import-resolver-typescript/lib/index.cjs:298:107)
at withResolver (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:180:23)
at fullResolve (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:201:22)
at relative (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:217:10)
at resolve (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:233:12)
at resolveImportType (/home/jailuser/git/node_modules/.pnpm/eslint-plugin-import@2.32.0_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1___504bceda7a403f00735dcb51d6a813c7/node_modules/eslint-plugin-import/lib/core/importType.js:126:2822)
at computeRank (/home/jailuser/git/node_modules/.pnpm/eslint-plugin-import@2.32.0_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1___504bceda7a403f00735dcb51d6a813c7/node_modules/eslint-plugin-import/lib/rules/order.js:529:43)(import/order)
apps/gateway/src/chat-api.e2e.ts
[error] 1-1: Resolve error: EACCES: permission denied, open '/FaDIpSaOtK'
at Object.writeFileSync (node:fs:2409:20)
at l (/home/jailuser/git/node_modules/.pnpm/get-tsconfig@4.10.1/node_modules/get-tsconfig/dist/index.cjs:7:13685)
at createFilesMatcher (/home/jailuser/git/node_modules/.pnpm/get-tsconfig@4.10.1/node_modules/get-tsconfig/dist/index.cjs:7:14437)
at Object.resolve (/home/jailuser/git/node_modules/.pnpm/eslint-import-resolver-typescript@4.4.4_eslint-plugin-import@2.32.0_eslint@9.34.0_jiti@2.5.1_/node_modules/eslint-import-resolver-typescript/lib/index.cjs:298:107)
at withResolver (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:180:23)
at fullResolve (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:201:22)
at relative (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:217:10)
at resolve (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:233:12)
at resolveImportType (/home/jailuser/git/node_modules/.pnpm/eslint-plugin-import@2.32.0_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1___504bceda7a403f00735dcb51d6a813c7/node_modules/eslint-plugin-import/lib/core/importType.js:126:2822)
at computeRank (/home/jailuser/git/node_modules/.pnpm/eslint-plugin-import@2.32.0_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1___504bceda7a403f00735dcb51d6a813c7/node_modules/eslint-plugin-import/lib/rules/order.js:529:43)(import/order)
[error] 1-1: Resolve error: EACCES: permission denied, open '/UtbgWCDTJI'
at Object.writeFileSync (node:fs:2409:20)
at l (/home/jailuser/git/node_modules/.pnpm/get-tsconfig@4.10.1/node_modules/get-tsconfig/dist/index.cjs:7:13685)
at createFilesMatcher (/home/jailuser/git/node_modules/.pnpm/get-tsconfig@4.10.1/node_modules/get-tsconfig/dist/index.cjs:7:14437)
at Object.resolve (/home/jailuser/git/node_modules/.pnpm/eslint-import-resolver-typescript@4.4.4_eslint-plugin-import@2.32.0_eslint@9.34.0_jiti@2.5.1_/node_modules/eslint-import-resolver-typescript/lib/index.cjs:298:107)
at withResolver (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:180:23)
at fullResolve (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:201:22)
at relative (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:217:10)
at resolve (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:233:12)
at checkSourceValue (/home/jailuser/git/node_modules/.pnpm/eslint-plugin-import@2.32.0_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1___504bceda7a403f00735dcb51d6a813c7/node_modules/eslint-plugin-import/lib/rules/no-useless-path-segments.js:85:53)
at checkSourceValue (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/moduleVisitor.js:32:5)(import/no-useless-path-segments)
apps/gateway/src/chat-full.e2e.ts
[error] 1-1: Resolve error: EACCES: permission denied, open '/LlrudiIQlA'
at Object.writeFileSync (node:fs:2409:20)
at l (/home/jailuser/git/node_modules/.pnpm/get-tsconfig@4.10.1/node_modules/get-tsconfig/dist/index.cjs:7:13685)
at createFilesMatcher (/home/jailuser/git/node_modules/.pnpm/get-tsconfig@4.10.1/node_modules/get-tsconfig/dist/index.cjs:7:14437)
at Object.resolve (/home/jailuser/git/node_modules/.pnpm/eslint-import-resolver-typescript@4.4.4_eslint-plugin-import@2.32.0_eslint@9.34.0_jiti@2.5.1_/node_modules/eslint-import-resolver-typescript/lib/index.cjs:298:107)
at withResolver (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:180:23)
at fullResolve (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:201:22)
at relative (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:217:10)
at resolve (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:233:12)
at resolveImportType (/home/jailuser/git/node_modules/.pnpm/eslint-plugin-import@2.32.0_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1___504bceda7a403f00735dcb51d6a813c7/node_modules/eslint-plugin-import/lib/core/importType.js:126:2822)
at computeRank (/home/jailuser/git/node_modules/.pnpm/eslint-plugin-import@2.32.0_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1___504bceda7a403f00735dcb51d6a813c7/node_modules/eslint-plugin-import/lib/rules/order.js:529:43)(import/order)
apps/gateway/src/chat-reasoning.e2e.ts
[error] 1-1: Resolve error: EACCES: permission denied, open '/IPUkdTQvmn'
at Object.writeFileSync (node:fs:2409:20)
at l (/home/jailuser/git/node_modules/.pnpm/get-tsconfig@4.10.1/node_modules/get-tsconfig/dist/index.cjs:7:13685)
at createFilesMatcher (/home/jailuser/git/node_modules/.pnpm/get-tsconfig@4.10.1/node_modules/get-tsconfig/dist/index.cjs:7:14437)
at Object.resolve (/home/jailuser/git/node_modules/.pnpm/eslint-import-resolver-typescript@4.4.4_eslint-plugin-import@2.32.0_eslint@9.34.0_jiti@2.5.1_/node_modules/eslint-import-resolver-typescript/lib/index.cjs:298:107)
at withResolver (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:180:23)
at fullResolve (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:201:22)
at relative (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:217:10)
at resolve (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:233:12)
at resolveImportType (/home/jailuser/git/node_modules/.pnpm/eslint-plugin-import@2.32.0_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1___504bceda7a403f00735dcb51d6a813c7/node_modules/eslint-plugin-import/lib/core/importType.js:126:2822)
at computeRank (/home/jailuser/git/node_modules/.pnpm/eslint-plugin-import@2.32.0_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1___504bceda7a403f00735dcb51d6a813c7/node_modules/eslint-plugin-import/lib/rules/order.js:529:43)(import/order)
apps/gateway/src/chat-rs.e2e.ts
[error] 1-1: Resolve error: EACCES: permission denied, open '/FUWCATAUwe'
at Object.writeFileSync (node:fs:2409:20)
at l (/home/jailuser/git/node_modules/.pnpm/get-tsconfig@4.10.1/node_modules/get-tsconfig/dist/index.cjs:7:13685)
at createFilesMatcher (/home/jailuser/git/node_modules/.pnpm/get-tsconfig@4.10.1/node_modules/get-tsconfig/dist/index.cjs:7:14437)
at Object.resolve (/home/jailuser/git/node_modules/.pnpm/eslint-import-resolver-typescript@4.4.4_eslint-plugin-import@2.32.0_eslint@9.34.0_jiti@2.5.1_/node_modules/eslint-import-resolver-typescript/lib/index.cjs:298:107)
at withResolver (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:180:23)
at fullResolve (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:201:22)
at relative (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:217:10)
at resolve (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:233:12)
at resolveImportType (/home/jailuser/git/node_modules/.pnpm/eslint-plugin-import@2.32.0_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1___504bceda7a403f00735dcb51d6a813c7/node_modules/eslint-plugin-import/lib/core/importType.js:126:2822)
at computeRank (/home/jailuser/git/node_modules/.pnpm/eslint-plugin-import@2.32.0_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1___504bceda7a403f00735dcb51d6a813c7/node_modules/eslint-plugin-import/lib/rules/order.js:529:43)(import/order)
apps/gateway/src/chat-toolcalls.e2e.ts
[error] 1-1: Resolve error: EACCES: permission denied, open '/SJoUkKKRlG'
at Object.writeFileSync (node:fs:2409:20)
at l (/home/jailuser/git/node_modules/.pnpm/get-tsconfig@4.10.1/node_modules/get-tsconfig/dist/index.cjs:7:13685)
at createFilesMatcher (/home/jailuser/git/node_modules/.pnpm/get-tsconfig@4.10.1/node_modules/get-tsconfig/dist/index.cjs:7:14437)
at Object.resolve (/home/jailuser/git/node_modules/.pnpm/eslint-import-resolver-typescript@4.4.4_eslint-plugin-import@2.32.0_eslint@9.34.0_jiti@2.5.1_/node_modules/eslint-import-resolver-typescript/lib/index.cjs:298:107)
at withResolver (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:180:23)
at fullResolve (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:201:22)
at relative (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:217:10)
at resolve (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:233:12)
at resolveImportType (/home/jailuser/git/node_modules/.pnpm/eslint-plugin-import@2.32.0_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1___504bceda7a403f00735dcb51d6a813c7/node_modules/eslint-plugin-import/lib/core/importType.js:126:2822)
at computeRank (/home/jailuser/git/node_modules/.pnpm/eslint-plugin-import@2.32.0_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1___504bceda7a403f00735dcb51d6a813c7/node_modules/eslint-plugin-import/lib/rules/order.js:529:43)(import/order)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (7)
- GitHub Check: build / run
- GitHub Check: test-build-split (gateway)
- GitHub Check: test-build-split (docs)
- GitHub Check: test-build-split (worker)
- GitHub Check: test-build-split (api)
- GitHub Check: test-build-split (ui)
- GitHub Check: test-build-unified
🔇 Additional comments (5)
apps/gateway/src/chat-toolcalls.e2e.ts (1)
20-24: No change required — Vitest v3 supports passing an options object as the second parameter to test.each. Keep the current usage.apps/gateway/src/chat-reasoning.e2e.ts (1)
23-27: No change required — test.each supports TestOptionsVitest 3.2.x accepts test.each(cases)('name', { retry: 3 }, fn); keep the current usage.
package.json (1)
26-26: Re-enable file-parallelism risks cross-test cache/DB interference — verification blocked (concurrently missing)Global FLUSHDB in per-test hooks will cause nondeterministic failures when test files run in parallel. Automated verification failed in the execution environment: "concurrently: command not found".
Fix options:
- Remove global FLUSHDB in per-test hooks (apps/gateway/src/chat-helpers.e2e.ts, apps/gateway/src/api-individual.e2e.ts).
- Isolate cache by per-test namespace/key prefix (include requestId).
- Keep file parallelism but run tests inside each file serially.
Reproduce locally (alternative to concurrently):
#!/bin/bash set -euo pipefail for i in {1..5}; do pnpm exec vitest run -c vitest/vitest.e2e.config.mts apps/gateway/src/chat-toolcalls.e2e.ts & pid1=$! pnpm exec vitest run -c vitest/vitest.e2e.config.mts apps/gateway/src/chat-reasoning.e2e.ts & pid2=$! wait $pid1 $pid2 doneapps/gateway/src/chat-toolcalls-result.e2e.ts (1)
20-86: LGTM overall – good end-to-end mapping of tool_call_id and resultConfirm that every provider in toolCallModels supports assistant-primed tool_calls; otherwise gate those providers behind FULL_MODE or skip.
apps/gateway/src/chat-rs.e2e.ts (1)
23-67: LGTM – robust SSE validation and usage chunk handling
| export const imageModels = testModels.filter((m) => { | ||
| const model = models.find((mo) => m.originalModel === mo.id); | ||
| return (model as ModelDefinition).output?.includes("image"); | ||
| }); |
There was a problem hiding this comment.
Crash if TEST_ALL_VARIATIONS produces root‑model cases
imageModels dereferences m.originalModel which is undefined for root cases, causing a runtime TypeError.
-export const imageModels = testModels.filter((m) => {
- const model = models.find((mo) => m.originalModel === mo.id);
- return (model as ModelDefinition).output?.includes("image");
-});
+export const imageModels = testModels.filter((m: any) => {
+ const modelId = m.originalModel ?? m.model; // handle root cases
+ const def = models.find((mo) => mo.id === modelId);
+ return !!def?.output?.includes("image");
+});📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| export const imageModels = testModels.filter((m) => { | |
| const model = models.find((mo) => m.originalModel === mo.id); | |
| return (model as ModelDefinition).output?.includes("image"); | |
| }); | |
| export const imageModels = testModels.filter((m: any) => { | |
| const modelId = m.originalModel ?? m.model; // handle root cases | |
| const def = models.find((mo) => mo.id === modelId); | |
| return !!def?.output?.includes("image"); | |
| }); |
🤖 Prompt for AI Agents
In apps/gateway/src/chat-helpers.e2e.ts around lines 298 to 301, the filter
dereferences m.originalModel which is undefined for root-model test cases and
causes a runtime TypeError; fix by guarding that m.originalModel is defined
before using it (e.g., return false for root cases), only call models.find when
m.originalModel exists, and use optional chaining when checking (model as
ModelDefinition)?.output?.includes("image") so root cases are safely excluded.
| export async function beforeEachHook() { | ||
| await clearCache(); | ||
| } |
There was a problem hiding this comment.
Global FLUSHDB in per‑test hook causes cross‑suite races
With file parallelism enabled, clearing the whole cache in every test will interfere with other suites.
-export async function beforeEachHook() {
- await clearCache();
-}
+export async function beforeEachHook() {
+ // Intentionally no-op to avoid cross-test cache interference in parallel runs.
+ // If needed for a specific test, clear only namespaced keys for that test.
+}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| export async function beforeEachHook() { | |
| await clearCache(); | |
| } | |
| export async function beforeEachHook() { | |
| // Intentionally no-op to avoid cross-test cache interference in parallel runs. | |
| // If needed for a specific test, clear only namespaced keys for that test. | |
| } |
🤖 Prompt for AI Agents
In apps/gateway/src/chat-helpers.e2e.ts around lines 420–422, the per-test hook
currently performs a global cache flush which races with other parallel test
suites; change the hook to avoid FLUSHDB on every test by either (a) scoping
test data with a unique per-worker/test key prefix and deleting only keys
matching that prefix, (b) using a dedicated Redis DB index per test worker and
flushing that DB only in beforeAll/afterAll for that worker, or (c) move a
single global FLUSHDB to a beforeAll/afterAll for the entire suite and ensure
each test uses isolated keys; implement one of these approaches and replace the
global flush with targeted key deletion or worker-level DB isolation.
Introduce a `setupComplete` flag to optimize `beforeAllHook` by skipping redundant setup steps after the initial run.
Removed the `setupComplete` flag from `beforeAllHook`, utilizing `onConflictDoNothing` to prevent duplicate data issues during setup.
Introduced an empty test case to ensure the `chat-full` file runs without errors.
Updated the `test:e2e` script in `package.json` to remove the `--no-file-parallelism` flag for streamlined test execution.
Moved shared helper functions and constants to a new `chat-helpers.e2e.ts` file for better reusability and organization. Updated imports across relevant test files. Removed redundant cleanup logic from `beforeAllHook` in `chat-api.e2e.ts`.
Replaced repetitive setup logic in `beforeAll` and `beforeEach` hooks with shared utilities. Updated log validation tests to use request-specific IDs for precise querying. Modified `test:e2e` script to enable `--no-file-parallelism`.
4a43111 to
2f8d52e
Compare
1 participant
Summary by CodeRabbit