feat: add ratelimit and validation to llm search#3311
Conversation
ogzhanolguncu
requested review from
chronark,
mcstepp and
perkinsjr
as code owners
|
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
1 Skipped Deployment
|
|
Warning Rate limit exceeded@ogzhanolguncu has exceeded the limit for the number of commits or files that can be reviewed per hour. Please wait 12 minutes and 44 seconds before requesting another review. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughA new middleware, Changes
Sequence Diagram(s)sequenceDiagram
participant User
participant SearchInput
participant TRPC_Server
participant withLlmAccess
participant LLM_Procedure
participant OpenAI
User->>SearchInput: Enter search query (max 120 chars)
SearchInput->>TRPC_Server: Submit query
TRPC_Server->>withLlmAccess: Apply rate limiting & validate query
withLlmAccess-->>TRPC_Server: Pass validated query in context
TRPC_Server->>LLM_Procedure: Call procedure with validated query
LLM_Procedure->>OpenAI: Send request (temperature 0.1)
OpenAI-->>LLM_Procedure: Return results
LLM_Procedure-->>TRPC_Server: Return results
TRPC_Server-->>SearchInput: Return results
SearchInput-->>User: Display search results
✨ Finishing Touches
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
|
Thank you for following the naming conventions for pull request titles! 🙏 |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (4)
apps/dashboard/components/logs/llm-search/components/search-input.tsx(2 hunks)apps/dashboard/lib/trpc/routers/logs/llm-search/index.ts(2 hunks)apps/dashboard/lib/trpc/routers/logs/llm-search/utils.ts(1 hunks)apps/dashboard/lib/trpc/trpc.ts(2 hunks)
🧰 Additional context used
🧬 Code Graph Analysis (2)
apps/dashboard/lib/trpc/routers/logs/llm-search/index.ts (2)
apps/dashboard/lib/trpc/trpc.ts (1)
withLlmAccess(155-189)apps/dashboard/lib/trpc/routers/logs/llm-search/utils.ts (1)
getStructuredSearchFromLLM(7-75)
apps/dashboard/lib/trpc/trpc.ts (1)
apps/dashboard/lib/env.ts (1)
env(3-49)
⏰ Context from checks skipped due to timeout of 90000ms (8)
- GitHub Check: Test Packages / Test ./packages/rbac
- GitHub Check: Test Packages / Test ./packages/hono
- GitHub Check: Test Packages / Test ./packages/cache
- GitHub Check: Test Packages / Test ./packages/nextjs
- GitHub Check: Test Packages / Test ./internal/clickhouse
- GitHub Check: Test Packages / Test ./internal/billing
- GitHub Check: autofix
- GitHub Check: Analyze (javascript-typescript)
🔇 Additional comments (11)
apps/dashboard/lib/trpc/trpc.ts (5)
5-5: LGTM: Zod import added for validation schema.The import is necessary for the new input validation functionality.
130-136: Well-structured LLM limits configuration.The constants are clearly defined with reasonable values. Using
as constensures type safety and immutability.
138-145: Proper rate limiter instantiation with environment-based configuration.The conditional creation and dedicated namespace "trpc_llm" provides good separation from other rate limiters.
147-153: Robust input validation schema with helpful error messages.The schema correctly trims input and enforces length constraints with user-friendly error messages.
155-189: 🛠️ Refactor suggestionConsider adding type safety for rawInput validation.
The middleware implementation is solid, but there's a potential runtime issue if
rawInputis not an object or lacks the expected structure.Consider adding a type guard or safer parsing approach:
export const withLlmAccess = () => t.middleware(async ({ next, ctx, rawInput }) => { if (llmRatelimit) { const response = await llmRatelimit.limit(ctx.user!.id); if (!response.success) { throw new TRPCError({ code: "TOO_MANY_REQUESTS", message: `LLM rate limit exceeded. You can make ${LLM_LIMITS.RATE_LIMIT} requests per minute.`, }); } } let validatedInput: z.infer<typeof llmQuerySchema>; try { + // Ensure rawInput is an object before parsing + if (!rawInput || typeof rawInput !== 'object') { + throw new TRPCError({ + code: "BAD_REQUEST", + message: "Invalid input format", + }); + } validatedInput = llmQuerySchema.parse(rawInput); } catch (error) { if (error instanceof z.ZodError) { const firstError = error.errors[0]; throw new TRPCError({ code: "BAD_REQUEST", message: firstError?.message || "Invalid query format", }); } throw new TRPCError({ code: "BAD_REQUEST", message: "Invalid input format", }); } return next({ ctx: { validatedQuery: validatedInput.query, }, }); });Likely an incorrect or invalid review comment.
apps/dashboard/lib/trpc/routers/logs/llm-search/utils.ts (1)
22-22: Good adjustment for more deterministic search results.Reducing the temperature to 0.1 will make the LLM responses more focused and consistent, which is appropriate for structured search query generation.
apps/dashboard/components/logs/llm-search/components/search-input.tsx (1)
43-43: Proper enforcement of input length constraint.The maxLength attribute correctly enforces the character limit on the frontend, providing immediate feedback to users.
apps/dashboard/lib/trpc/routers/logs/llm-search/index.ts (4)
2-2: Clean import update for new middleware approach.Properly replaces the generic rate limiting with the specialized LLM access middleware.
16-16: Correct middleware integration.The withLlmAccess middleware properly handles both rate limiting and input validation for LLM queries.
17-17: Simplified input schema reflects middleware-handled validation.Removing the query field from input makes sense since the middleware now validates and provides it via context.
19-19: Proper usage of validated query from context.Correctly accesses the validated query from the middleware context instead of the raw input.
2 participants
What does this PR do?
This PR adds stricter ratelimit for LLM and also adds input length checks to prevent abuse.
Fixes # (issue)
If there is not an issue for this, please create one first. This is used to tracking purposes and also helps use understand why this PR exists
Type of change
How should this be tested?
Checklist
Required
pnpm buildpnpm fmtconsole.logsgit pull origin mainAppreciated
Summary by CodeRabbit
New Features
Improvements