perf: thread pre-fetched token to checkRateLimit avoiding re-query - #6930
Conversation
|
Warning You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again! |
|
Thanks for the perf fix — threading the already-fetched A few things need to happen before this can merge as-is:
Once split down to just the relay fix + a regression test + changelog fragment, this looks mergeable. |
getRelayTokenByHash already fetches the full RelayToken row. A few lines later checkRateLimit(token.id) does a second SELECT * FROM relay_tokens on a different predicate (id instead of token_hash). Change: - checkRateLimit accepts an optional existingToken parameter; when provided, skips the re-query entirely. - Both relay routes (chat completions + bifrost) pass the already- fetched token. - The function now uses RelayToken (camelCase) instead of RelayTokenRow (snake_case) when the token is passed in. PR-URL: fix-relay-thread-token
…st-path Adds node:test coverage for src/lib/db/relayProxies.ts::checkRateLimit proving the existingToken fast-path (pre-fetched RelayToken threaded in, no re-query) agrees with the legacy re-query path (no token passed), and that the per-minute cap is still enforced through the fast-path. Also adds a changelog.d fragment for the perf fix in 9d4cd90. Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
55449c3 to
aed01e9
Compare
…iegosouzapw#6930) * perf: thread pre-fetched token to checkRateLimit avoiding re-query getRelayTokenByHash already fetches the full RelayToken row. A few lines later checkRateLimit(token.id) does a second SELECT * FROM relay_tokens on a different predicate (id instead of token_hash). Change: - checkRateLimit accepts an optional existingToken parameter; when provided, skips the re-query entirely. - Both relay routes (chat completions + bifrost) pass the already- fetched token. - The function now uses RelayToken (camelCase) instead of RelayTokenRow (snake_case) when the token is passed in. PR-URL: fix-relay-thread-token * test(db): add regression coverage for checkRateLimit existingToken fast-path Adds node:test coverage for src/lib/db/relayProxies.ts::checkRateLimit proving the existingToken fast-path (pre-fetched RelayToken threaded in, no re-query) agrees with the legacy re-query path (no token passed), and that the per-minute cap is still enforced through the fast-path. Also adds a changelog.d fragment for the perf fix in 9d4cd90. Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com> --------- Co-authored-by: oyi77 <oyi77@users.noreply.github.com> Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
…iegosouzapw#6930) * perf: thread pre-fetched token to checkRateLimit avoiding re-query getRelayTokenByHash already fetches the full RelayToken row. A few lines later checkRateLimit(token.id) does a second SELECT * FROM relay_tokens on a different predicate (id instead of token_hash). Change: - checkRateLimit accepts an optional existingToken parameter; when provided, skips the re-query entirely. - Both relay routes (chat completions + bifrost) pass the already- fetched token. - The function now uses RelayToken (camelCase) instead of RelayTokenRow (snake_case) when the token is passed in. PR-URL: fix-relay-thread-token * test(db): add regression coverage for checkRateLimit existingToken fast-path Adds node:test coverage for src/lib/db/relayProxies.ts::checkRateLimit proving the existingToken fast-path (pre-fetched RelayToken threaded in, no re-query) agrees with the legacy re-query path (no token passed), and that the per-minute cap is still enforced through the fast-path. Also adds a changelog.d fragment for the perf fix in 9d4cd90. Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com> --------- Co-authored-by: oyi77 <oyi77@users.noreply.github.com> Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
Problem: getRelayTokenByHash fetches full relay token row. A few lines later checkRateLimit(token.id) does a second SELECT * FROM relay_tokens on a different predicate.
Change:
Verification: all relay unit tests pass (only pre-existing Cloudflare deploy failures).