5.1 event summarizer - #18
Merged
Merged
Conversation
Contributor
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
WalkthroughAdds end-to-end AI pipelines and supporting infrastructure: event summarization, article atomic-fact extraction, bias scoring, and claim-divergence analysis, plus schema/table additions, seeded config defaults, cron jobs, budget-aware OpenAI wrapper, prompt builders, NLP extraction, and node actions for claiming, processing, and persisting results. (50 words) Changes
Sequence Diagram(s)sequenceDiagram
participant Cron as Cron Scheduler
participant SummNode as Summarization Node
participant Budget as AI Budget Service
participant DB as Convex DB
participant OpenAI as OpenAI
Cron->>SummNode: trigger summarizeQueuedEvents()
SummNode->>DB: enqueueEligibleEventSummaries()
SummNode->>DB: listDueSummaryJobs()
loop per job
SummNode->>Budget: checkDailyBudget()
alt within budget
SummNode->>DB: startSummaryJob(jobId, runId, leaseExpiresAt)
SummNode->>DB: getEventSummaryInput(eventId)
SummNode->>OpenAI: callOpenAI(prompt, json_schema)
OpenAI-->>SummNode: structured summary
SummNode->>DB: applyEventSummaryResult(...)
SummNode->>Budget: recordUsage(...)
else budget exhausted
SummNode->>DB: markSummaryJobFailed(..., retryAfterMs)
end
end
sequenceDiagram
participant Cron as Cron Scheduler
participant EnrichNode as Enrichment Node
participant Budget as AI Budget Service
participant NLP as wink-nlp
participant OpenAI as OpenAI
participant DB as Convex DB
Cron->>EnrichNode: trigger reenrichEventArticles(eventId, limit)
EnrichNode->>DB: claimEventArticlesForReenrichment(eventId, limit, runId, lease)
loop per claimed article
EnrichNode->>Budget: checkDailyBudget()
alt atomic-fact & bias enabled & within budget
EnrichNode->>NLP: extract candidate entities(article)
EnrichNode->>OpenAI: callOpenAI(buildArticleFactExtractionPrompt(...))
OpenAI-->>EnrichNode: atomic facts JSON
EnrichNode->>OpenAI: callOpenAI(buildArticleBiasScoringPrompt(...))
OpenAI-->>EnrichNode: bias scoring JSON
EnrichNode->>DB: markArticleEnriched(atomicFacts, aiBiasScore, biasComponents...)
EnrichNode->>Budget: recordUsage(...)
else skip or preserve existing fields
EnrichNode->>DB: markArticleEnriched(preserve existing values)
end
end
EnrichNode->>DB: refreshEventClaimCoverage(eventId)
Estimated code review effort🎯 5 (Critical) | ⏱️ ~120 minutes Possibly related PRs
🚥 Pre-merge checks | ✅ 3 | ❌ 2❌ Failed checks (1 warning, 1 inconclusive)
✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 4
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@packages/backend/convex/enrichmentNode.ts`:
- Around line 236-240: The ctx parameter in extractAtomicFactsForArticles is
typed as any; replace it with the proper Convex action context type by importing
the Convex server context type (e.g. import type { ActionContext } from
"convex/server") and change the function signature to use that type (e.g. ctx:
ActionContext) so extractAtomicFactsForArticles and any callers get type-safe
access to Convex methods and stores.
In `@packages/backend/convex/lib/articleExtraction.ts`:
- Around line 197-199: The acronym detection in isUsefulEntityCandidate is
broken because normalizeEntityCandidate lowercases the entity, so the check
entity.toUpperCase() === entity never succeeds; update normalizeEntityCandidate
to also return a flag indicating whether the original string was all-caps (e.g.,
wasAllUppercase) or preserve the original form, then change
isUsefulEntityCandidate to accept that flag (or the original) and use it in the
single-word branch: replace the current uppercase check with the new
wasAllUppercase flag and add parentheses around the && clause for clarity;
reference normalizeEntityCandidate and isUsefulEntityCandidate and update their
signatures/usages accordingly.
- Around line 223-230: The token callback currently types its parameter as any;
replace this with a proper type by either importing wink-nlp's Token type (e.g.,
import { Token } from 'wink-nlp') and changing the callback to
doc.tokens().each((token: Token) => ...) or, if that type is unavailable,
declare a minimal local interface (e.g., interface TokenLike { out(method?:
any): string }) and use doc.tokens().each((token: TokenLike) => ...), ensuring
the typed token is used where token.out(its.normal), token.out(its.pos) and
token.out(its.type) are called to remove the any usage.
In `@packages/backend/convex/summarization.ts`:
- Around line 53-63: getLatestSummaryJob currently fetches all eventSummaryJobs
and sorts in memory; add a composite index named by_event_updatedAt on
["eventId","updatedAt"] and change getLatestSummaryJob to use
ctx.db.query("eventSummaryJobs").withIndex("by_event_updatedAt", q =>
q.eq("eventId", eventId) /* apply descending updatedAt / limit 1 via index query
*/) to fetch only the most recent job instead of collecting and sorting; update
the DB index definition (eventSummaryJobs) and replace the in-memory sort in
getLatestSummaryJob accordingly.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 28d42cf9-ced3-4d11-87d6-6100a8d8d64c
⛔ Files ignored due to path filters (2)
packages/backend/convex/_generated/api.d.tsis excluded by!**/_generated/**,!**/_generated/**pnpm-lock.yamlis excluded by!**/pnpm-lock.yaml,!**/pnpm-lock.yaml
📒 Files selected for processing (10)
packages/backend/convex/config.tspackages/backend/convex/crons.tspackages/backend/convex/enrichment.tspackages/backend/convex/enrichmentNode.tspackages/backend/convex/lib/articleExtraction.tspackages/backend/convex/prompts.tspackages/backend/convex/schema.tspackages/backend/convex/summarization.tspackages/backend/convex/summarizationNode.tspackages/backend/package.json
Contributor
There was a problem hiding this comment.
Actionable comments posted: 6
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
packages/backend/convex/enrichmentNode.ts (1)
363-376:⚠️ Potential issue | 🟡 MinorAvoid
anytype forctxparameter.The
runEnrichmentBatchfunction still usesanyfor thectxparameter, which loses type safety. This should use the proper Convex action context type.🛡️ Proposed fix
+import type { ActionCtx } from "./_generated/server"; + async function runEnrichmentBatch( - ctx: any, + ctx: ActionCtx, articles: Array<{Note: The
ActionCtximport already exists at line 19, so only the function signature needs updating.As per coding guidelines: "Check for proper type safety (avoid 'any' types)".
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/backend/convex/enrichmentNode.ts` around lines 363 - 376, The ctx parameter of runEnrichmentBatch is typed as any—replace it with the Convex action context type already imported as ActionCtx: update the function signature (runEnrichmentBatch) to use ctx: ActionCtx instead of ctx: any and ensure any internal usages remain compatible with ActionCtx; no other behavior changes required.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@packages/backend/convex/claimDivergence.ts`:
- Around line 264-289: getEventClaims currently only calls requireBetaAccess and
then returns claims, allowing any beta user with an eventId to read claims;
fetch the event record first (e.g., via
ctx.db.query("events").get(args.eventId)) and enforce the event's
visibility/published/readable rules before querying "eventClaims" — either call
the existing visibility helper (e.g., requireEventReadable or similar) or
explicitly check event.published/visibility and throw/return an authorization
error if the caller shouldn't see the event, then continue to query and return
claims; update getEventClaims to perform this check using ctx and args.eventId
before the claims query.
- Around line 86-123: The loop in claimDivergence.ts currently issues an
articles.by_event.collect() per event (N+1 reads) inside the candidate selection
(see events query and the for loop using eventNeedsAnalysis and articles query),
so change candidate selection to use an index-driven check instead of scanning
articles: add/consume denormalized fields on the events record (e.g.,
factualArticleCount, factualSourceCount, lastFactualUpdateAt) or create a
lightweight coverage table keyed by eventId that is maintained on article
create/update/delete, then replace the per-event articles.collect() and derived
sourceCount logic with checks against those denormalized fields (use
event.factualArticleCount and event.factualSourceCount or a single coverage
lookup via an indexed query) and remove the inner articles.collect(); ensure any
writer code that adds/removes articles updates the denormalized counts/coverage
table atomically so the events query remains index-driven and avoids the N+1
reads.
In `@packages/backend/convex/claimDivergenceNode.ts`:
- Around line 543-548: The branch in claimDivergenceNode that handles ineligible
inputs calls internal.claimDivergence.markEventClaimAnalysisSkipped
unconditionally, which patches lastClaimAnalysisAt and fails when input.reason
=== "event_missing"; change the logic in the block (where input.eligible,
input.reason and internal.claimDivergence.markEventClaimAnalysisSkipped are
used) to skip calling markEventClaimAnalysisSkipped when input.reason ===
"event_missing" (i.e., only run
ctx.runMutation(internal.claimDivergence.markEventClaimAnalysisSkipped, {
eventId }) for other reasons) and then return { skipped: true, reason:
input.reason } as before.
In `@packages/backend/convex/crons.ts`:
- Around line 79-96: Remove the unnecessary empty fourth argument object from
the two cron registrations to match the established pattern; update the calls to
crons.interval for "summarize-published-events" and "detect-event-claims" by
invoking crons.interval("summarize-published-events", { minutes: 30 },
internal.summarizationNode.summarizeQueuedEvents) and
crons.interval("detect-event-claims", { minutes: 30 },
internal.claimDivergenceNode.processStaleEventClaims) respectively, so the empty
{} is not passed as the function-arguments parameter.
In `@packages/backend/convex/summarization.ts`:
- Around line 27-35: The function sourceBiasLabel currently falls through to
return "center" for baseBias === 0 implicitly; update sourceBiasLabel to check
baseBias === 0 explicitly and return "center" before the other numeric
comparisons (retain the existing mbfcCategory check), so the logic reads: null
check, mbfcCategory, explicit baseBias === 0 => "center", then the
left/left-center/right-center/right branches; this makes the center case
explicit and clearer.
---
Outside diff comments:
In `@packages/backend/convex/enrichmentNode.ts`:
- Around line 363-376: The ctx parameter of runEnrichmentBatch is typed as
any—replace it with the Convex action context type already imported as
ActionCtx: update the function signature (runEnrichmentBatch) to use ctx:
ActionCtx instead of ctx: any and ensure any internal usages remain compatible
with ActionCtx; no other behavior changes required.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 37dcc84f-d289-4af5-ad3d-1cb928cff59a
⛔ Files ignored due to path filters (1)
packages/backend/convex/_generated/api.d.tsis excluded by!**/_generated/**,!**/_generated/**
📒 Files selected for processing (10)
packages/backend/convex/claimDivergence.tspackages/backend/convex/claimDivergenceNode.tspackages/backend/convex/config.tspackages/backend/convex/crons.tspackages/backend/convex/enrichment.tspackages/backend/convex/enrichmentNode.tspackages/backend/convex/lib/articleExtraction.tspackages/backend/convex/prompts.tspackages/backend/convex/schema.tspackages/backend/convex/summarization.ts
Comment on lines
+522
to
+577
| const budget = await ctx.runQuery(internal.aiBudget.checkBudget, {}); | ||
| if (!budget.allowed) { | ||
| return { | ||
| skipped: true as const, | ||
| reason: "budget_exhausted", | ||
| spentUsd: budget.spentUsd, | ||
| dailyLimitUsd: budget.dailyLimitUsd, | ||
| }; | ||
| } | ||
|
|
||
| const input = await ctx.runQuery( | ||
| internal.claimDivergence.getClaimAnalysisInput, | ||
| { | ||
| eventId, | ||
| minArticles: settings.minArticles, | ||
| minSources: settings.minSources, | ||
| maxArticles: settings.maxInputArticles, | ||
| maxFactsPerArticle: settings.maxFactsPerArticle, | ||
| }, | ||
| ); | ||
|
|
||
| if (!input.eligible) { | ||
| await ctx.runMutation( | ||
| internal.claimDivergence.markEventClaimAnalysisSkipped, | ||
| { eventId }, | ||
| ); | ||
| return { skipped: true as const, reason: input.reason }; | ||
| } | ||
|
|
||
| try { | ||
| const { claims, inputTokens, outputTokens } = | ||
| await detectEventClaimsForInput(input, settings); | ||
|
|
||
| const costUsd = calculateCost(settings.model, inputTokens, outputTokens); | ||
| const usage = await ctx.runMutation(internal.aiBudget.logUsage, { | ||
| model: settings.model, | ||
| operation: "analyze_event_claims", | ||
| inputTokens, | ||
| outputTokens, | ||
| costUsd, | ||
| eventId: input.event._id, | ||
| }); | ||
|
|
||
| if (!usage.allowed) { | ||
| console.warn( | ||
| `[claimDivergence] Usage log rejected because budget would be exceeded ($${usage.spentUsd}/$${usage.dailyLimitUsd})`, | ||
| ); | ||
| } | ||
|
|
||
| const result = await ctx.runMutation( | ||
| internal.claimDivergence.replaceEventClaims, | ||
| { | ||
| eventId: input.event._id, | ||
| claims, | ||
| }, | ||
| ); |
Contributor
There was a problem hiding this comment.
Make budget enforcement atomic around the OpenAI call.
checkBudget and logUsage are separate round trips, so concurrent workers can all pass the preflight check and overspend the daily cap. Line 571 also persists claims even when logUsage comes back with allowed: false, which makes the budget rejection non-blocking in practice. Reserve/quota-check in a mutation before the model call, then reconcile the final usage afterward.
As per coding guidelines, packages/backend/**: Focus on Convex best practices; Check for proper error handling.
Owner
Author
|
@CodeRabbit review |
Contributor
✅ Actions performedReview triggered.
|
Contributor
There was a problem hiding this comment.
Actionable comments posted: 8
♻️ Duplicate comments (1)
packages/backend/convex/crons.ts (1)
79-108: 🧹 Nitpick | 🔵 TrivialRemove the empty
{}arguments for consistency with existing cron definitions.The 4th argument to
crons.interval()andcrons.daily()is for passing function arguments, not configuration options. While passing{}is valid since all three target functions have optional arguments, it's inconsistent with the other cron definitions in this file (lines 11-71) which don't pass a 4th argument at all.♻️ Proposed fix
crons.interval( "summarize-published-events", { minutes: 30 }, internal.summarizationNode.summarizeQueuedEvents, - {}, ); // --------------------------------------------------------------------------- // Claim Divergence Detection — Every 30 minutes // --------------------------------------------------------------------------- // Builds the eventClaims graph from atomic facts so the product can show // agreements, conflicts, framing differences, and lean-specific exclusives. crons.interval( "detect-event-claims", { minutes: 30 }, internal.claimDivergenceNode.processStaleEventClaims, - {}, ); // --------------------------------------------------------------------------- // Article Bias Outlier Detection — Daily // --------------------------------------------------------------------------- // Computes rolling per-source article bias stats and flags articles that are // unusually partisan for their outlet. crons.daily( "flag-bias-outliers", { hourUTC: 5, minuteUTC: 0 }, internal.bias.flagBiasOutliers, - {}, );🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/backend/convex/crons.ts` around lines 79 - 108, Remove the unnecessary empty fourth-argument object from the cron registrations: update the three calls that pass {} to instead call crons.interval("summarize-published-events", { minutes: 30 }, internal.summarizationNode.summarizeQueuedEvents), crons.interval("detect-event-claims", { minutes: 30 }, internal.claimDivergenceNode.processStaleEventClaims), and crons.daily("flag-bias-outliers", { hourUTC: 5, minuteUTC: 0 }, internal.bias.flagBiasOutliers) so they match the other cron definitions that omit the unused args.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@packages/backend/convex/aiBudget.ts`:
- Line 33: The entry using the deprecated model identifier "gpt-4.1-nano" in the
pricing map should be migrated to a supported model (e.g., "gpt-5-nano"); update
the key in the map and adjust the pricing values to the current GPT-5 nano
rates, and search for any other references to "gpt-4.1-nano" in the repo (usage
sites, tests, docs) and replace them with the new model identifier
(function/variable to inspect: the pricing map in aiBudget.ts that contains the
"gpt-4.1-nano" property).
In `@packages/backend/convex/bias.ts`:
- Around line 33-42: The mean and standardDeviation helpers can divide by zero
for empty arrays; add defensive input validation at the start of mean(values)
and standardDeviation(values, average) to detect values.length === 0 and throw a
clear Error (e.g., "mean requires non-empty array") or return a defined sentinel
(choose one consistently), and update any callers to handle that error/return;
this prevents silent NaNs or runtime division-by-zero when these functions are
invoked outside the current protected call site and keeps behavior explicit for
functions mean and standardDeviation.
In `@packages/backend/convex/claimDivergenceNode.ts`:
- Around line 253-274: The current statementSupportsClaim heuristic in
statementSupportsClaim (using meaningfulTokens, normalizeForComparison) is too
strict and can false-negative paraphrases; update it to be more tolerant by (1)
lowering the overlap threshold or reducing the minimum token requirement, (2)
adding a synonym/paraphrase check (e.g., consult a synonym map or a lightweight
stemming/lemmatization step when comparing tokens produced by meaningfulTokens),
and (3) during development emit debug logging of filtered variants (log the
canonicalStatement, statement, value, overlap, canonicalTokens.size and the
reason for rejection) so you can iteratively tune thresholds and the synonym
list. Ensure logging is gated behind a dev flag to avoid noise in production.
In `@packages/backend/convex/enrichmentNode.ts`:
- Around line 769-772: The two AI calls extractAtomicFactsForArticles and
scoreBiasForArticles are being launched in parallel which can bypass per-call
budget checks and exceed daily limits; change the invocation to run sequentially
(await extractAtomicFactsForArticles(...) first, then await
scoreBiasForArticles(...)) or implement a budget reservation API that atomically
checks and reserves budget before starting either call (modify the call site
around the Promise.all in enrichmentNode.ts to use the sequential approach or
integrate the reservation check so only one call can proceed when budget is
low).
In `@packages/backend/convex/lib/aiCall.ts`:
- Around line 100-122: The logUsage function currently calls
runtime.runMutation(internal.aiBudget.recordUsage, ...) without catching
exceptions, so any thrown error will bubble up and break the AI call flow; wrap
the runtime.runMutation(...) call in a try/catch inside logUsage, preserve the
existing check for result.allowed, and on catch log an error (including
context.callType, context.eventId, and the caught error) but do not rethrow so
logging failures are isolated from the AI call flow.
In `@packages/backend/convex/summarization.ts`:
- Around line 495-515: The admin mutation enqueueEventSummaryForAdmin currently
inserts a queued job unconditionally which can create jobs that will immediately
be skipped due to article/source minimums; update enqueueEventSummaryForAdmin to
either perform the same eligibility checks used by the normal enqueue flow
(reuse the eligibility function used elsewhere, e.g., the routine that computes
article/source minimums) before inserting into eventSummaryJobs, or, if you want
to preserve override behavior, add an explicit warning field to the response
(e.g., return { queued: true, warning: "ineligible_by_minimums" }) explaining
the job may be skipped and include which eligibility rule failed; reference the
mutation name enqueueEventSummaryForAdmin and the eventSummaryJobs insert to
locate where to add the check or change the response.
- Around line 241-260: In claimSummaryJobs where jobs are iterated and patched
(see claimSummaryJobs loop, variables job, ctx.db.patch, processingRunId,
leaseExpiresAt, runId), add a log entry when you're about to overwrite an
existing lease: detect if job.processingRunId is set and job.leaseExpiresAt
exists and is > now, then emit a warning/info (including job._id, job.eventId,
existing processingRunId, existing leaseExpiresAt, and new runId) before calling
ctx.db.patch so we have observability when a lease is being overridden.
In `@packages/backend/convex/summarizationNode.ts`:
- Around line 283-295: The code marks jobs as failed when
internal.aiBudget.checkBudget reports budget.allowed === false; change this to
mark the job as skipped instead of failed: call
internal.summarization.markSummaryJobSkipped (or create it if missing) with
jobId: job._id, runId, an explanatory error/message like "AI budget exhausted
($spent/$limit)", and keep budgetExhausted = true and the continue; do not
increment failed or touch job.attempts/maxAttempts or set a retryAfterMs that
would consume retries — skipped jobs should not count as an attempt. Ensure you
update the call-site that currently invokes
internal.summarization.markSummaryJobFailed to use the skipped mutation and
preserve relevant metadata (runId, jobId, message).
---
Duplicate comments:
In `@packages/backend/convex/crons.ts`:
- Around line 79-108: Remove the unnecessary empty fourth-argument object from
the cron registrations: update the three calls that pass {} to instead call
crons.interval("summarize-published-events", { minutes: 30 },
internal.summarizationNode.summarizeQueuedEvents),
crons.interval("detect-event-claims", { minutes: 30 },
internal.claimDivergenceNode.processStaleEventClaims), and
crons.daily("flag-bias-outliers", { hourUTC: 5, minuteUTC: 0 },
internal.bias.flagBiasOutliers) so they match the other cron definitions that
omit the unused args.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 852d91e8-7cda-4d37-9dd7-0d6a38c99ad6
⛔ Files ignored due to path filters (1)
packages/backend/convex/_generated/api.d.tsis excluded by!**/_generated/**,!**/_generated/**
📒 Files selected for processing (15)
packages/backend/convex/aiBudget.tspackages/backend/convex/bias.tspackages/backend/convex/claimDivergence.tspackages/backend/convex/claimDivergenceNode.tspackages/backend/convex/clustering.tspackages/backend/convex/config.tspackages/backend/convex/crons.tspackages/backend/convex/enrichment.tspackages/backend/convex/enrichmentNode.tspackages/backend/convex/lib/aiCall.tspackages/backend/convex/lib/eventClaimCoverage.tspackages/backend/convex/prompts.tspackages/backend/convex/schema.tspackages/backend/convex/summarization.tspackages/backend/convex/summarizationNode.ts
| const DEFAULT_MODEL_RATES: Record<string, { input: number; output: number }> = { | ||
| "gpt-4o-mini": { input: 0.00000015, output: 0.0000006 }, | ||
| "gpt-4o": { input: 0.0000025, output: 0.00001 }, | ||
| "gpt-4.1-nano": { input: 0.0000001, output: 0.0000004 }, |
Contributor
There was a problem hiding this comment.
🧩 Analysis chain
🌐 Web query:
OpenAI gpt-4.1-nano model API
💡 Result:
The OpenAI gpt-4.1-nano model is a fast, cost-efficient nano variant of the GPT-4.1 series, available via the OpenAI API. It was launched around April 2025 as part of the GPT-4.1 family, outperforming GPT-4o mini in benchmarks like coding and instruction following, with a 1M token context window, text and image input, text output, and a knowledge cutoff of June 2024. Key specs: - Context window: 1,047,576 tokens - Max output: 32,768 tokens - Pricing (per 1M tokens): Input $0.10, Cached input $0.025, Output $0.40 (Batch API 50% discount) - Features: Streaming, function calling, structured outputs, fine-tuning supported - Endpoints: Chat Completions (v1/chat/completions), Responses (v1/responses), Realtime (v1/realtime), Assistants, Batch, etc. - Model ID: gpt-4.1-nano (alias); snapshot gpt-4.1-nano-2025-04-14 (deprecated) As of April 2026, it is listed as deprecated in OpenAI's model catalog, with recommendations to use newer models like GPT-5 nano for complex tasks. Specific snapshots like gpt-4.1-nano-2025-04-14 are marked deprecated, and full deprecation is scheduled for October 23, 2026, after which it will be replaced by gpt-5-nano. It remains documented and usable in the API for now, but developers should plan to migrate to current frontier models like GPT-5.4 nano or GPT-5 nano. To use: Specify "gpt-4.1-nano" in API calls to compatible endpoints, e.g., curl https://api.openai.com/v1/chat/completions \ -H "Authorization: Bearer $OPENAI_API_KEY" \ -d '{ "model": "gpt-4.1-nano", "messages": [{"role": "user", "content": "Hello!"}] }' Check your usage tier for rate limits (e.g., Tier 1: 500 RPM, 200k TPM).
Citations:
- 1: https://developers.openai.com/api/docs/models/gpt-4.1-nano
- 2: https://openai.com/research/gpt-4-1
- 3: https://platform.openai.com/docs/models/gpt-4.1-nano
- 4: https://developers.openai.com/api/docs/models/all
- 5: https://platform.openai.com/docs/deprecations/
Consider migrating from the deprecated gpt-4.1-nano model.
The model identifier is valid and correctly priced ($0.0000001 input, $0.0000004 output per token), but gpt-4.1-nano is deprecated as of April 2026 with full deprecation scheduled for October 23, 2026. Plan migration to current models like GPT-5 nano before removal.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@packages/backend/convex/aiBudget.ts` at line 33, The entry using the
deprecated model identifier "gpt-4.1-nano" in the pricing map should be migrated
to a supported model (e.g., "gpt-5-nano"); update the key in the map and adjust
the pricing values to the current GPT-5 nano rates, and search for any other
references to "gpt-4.1-nano" in the repo (usage sites, tests, docs) and replace
them with the new model identifier (function/variable to inspect: the pricing
map in aiBudget.ts that contains the "gpt-4.1-nano" property).
Comment on lines
+769
to
+772
| const [factsByArticleId, biasByArticleId] = await Promise.all([ | ||
| extractAtomicFactsForArticles(ctx, preparedArticles, factSettings), | ||
| scoreBiasForArticles(ctx, preparedArticles, biasSettings), | ||
| ]); |
Contributor
There was a problem hiding this comment.
Parallel AI calls may cause budget race conditions.
extractAtomicFactsForArticles and scoreBiasForArticles are called in parallel, but each independently checks the budget. If the budget is nearly exhausted, both calls may pass their preflight checks and then both proceed, potentially exceeding the daily limit.
Consider running these sequentially or implementing a budget reservation mechanism.
♻️ Suggested sequential approach
- const [factsByArticleId, biasByArticleId] = await Promise.all([
- extractAtomicFactsForArticles(ctx, preparedArticles, factSettings),
- scoreBiasForArticles(ctx, preparedArticles, biasSettings),
- ]);
+ const factsByArticleId = await extractAtomicFactsForArticles(ctx, preparedArticles, factSettings);
+ const biasByArticleId = await scoreBiasForArticles(ctx, preparedArticles, biasSettings);🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@packages/backend/convex/enrichmentNode.ts` around lines 769 - 772, The two AI
calls extractAtomicFactsForArticles and scoreBiasForArticles are being launched
in parallel which can bypass per-call budget checks and exceed daily limits;
change the invocation to run sequentially (await
extractAtomicFactsForArticles(...) first, then await scoreBiasForArticles(...))
or implement a budget reservation API that atomically checks and reserves budget
before starting either call (modify the call site around the Promise.all in
enrichmentNode.ts to use the sequential approach or integrate the reservation
check so only one call can proceed when budget is low).
Comment on lines
+241
to
+260
| const claimed = []; | ||
| for (const job of jobs) { | ||
| if (claimed.length >= safeLimit) break; | ||
| if (job.attempts >= maxAttempts) continue; | ||
|
|
||
| await ctx.db.patch(job._id, { | ||
| status: "processing", | ||
| attempts: job.attempts + 1, | ||
| processingRunId: runId, | ||
| leaseExpiresAt, | ||
| updatedAt: now, | ||
| lastError: undefined, | ||
| }); | ||
|
|
||
| claimed.push({ | ||
| _id: job._id, | ||
| eventId: job.eventId, | ||
| attempts: job.attempts + 1, | ||
| }); | ||
| } |
Contributor
There was a problem hiding this comment.
🧹 Nitpick | 🔵 Trivial
Job claiming could steal jobs from active runs with expired leases.
The claimSummaryJobs mutation claims queued and failed jobs without checking leaseExpiresAt. While expired leases indicate the previous run likely died, there's a race window where a slow run completes just as another claims its job.
The downstream mutations (applyEventSummaryResult, markSummaryJobFailed) validate processingRunId, so double-processing is prevented, but the original run's result would be discarded.
This is acceptable given the lease TTL (10 minutes) provides ample buffer, but consider logging when a lease is being overwritten for observability.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@packages/backend/convex/summarization.ts` around lines 241 - 260, In
claimSummaryJobs where jobs are iterated and patched (see claimSummaryJobs loop,
variables job, ctx.db.patch, processingRunId, leaseExpiresAt, runId), add a log
entry when you're about to overwrite an existing lease: detect if
job.processingRunId is set and job.leaseExpiresAt exists and is > now, then emit
a warning/info (including job._id, job.eventId, existing processingRunId,
existing leaseExpiresAt, and new runId) before calling ctx.db.patch so we have
observability when a lease is being overridden.
Contributor
There was a problem hiding this comment.
Actionable comments posted: 8
♻️ Duplicate comments (1)
packages/backend/convex/summarization.ts (1)
173-175:⚠️ Potential issue | 🟠 MajorStop when the queue is full, not when inspection hits the limit.
This breaks after looking at
safeLimitevents, so a run with many ineligible recent events can enqueue far fewer than requested even though you already fetched additional candidates with.take(safeLimit * 3).♻️ Suggested fix
for (const event of events) { - if (inspected >= safeLimit) break; + if (queued >= safeLimit) break; inspected++;🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/backend/convex/summarization.ts` around lines 173 - 175, The loop currently breaks based on inspected >= safeLimit which stops after scanning safeLimit events rather than stopping when you've actually enqueued the desired number; change the loop to break when the output queue/enqueued count reaches the target (e.g., check summariesQueue.length or an enqueuedCount variable) and only use inspected as a secondary safety cap. Locate the for (const event of events) loop and replace the primary break condition from inspected >= safeLimit to a check like queue.length >= safeLimit (or enqueuedCount >= safeLimit), keeping the .take(safeLimit * 3) fetch and the inspected counter as an optional fallback to avoid infinite loops.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@packages/backend/convex/aiBudget.ts`:
- Around line 154-177: The daily window queries use a strict lower bound (q.gt)
which excludes events exactly at startOfUtcDay; update the by_timestamp index
queries in getDailyBudgetState (and likewise in reserveBudget,
recordUsageInternal, getTodaysUsage) to use an inclusive lower bound
(q.gte(startOfUtcDay(now))) so timestamps at 00:00:00.000Z are counted; keep the
same startOfUtcDay(now) value but change the query predicate from gt to gte for
those by_timestamp queries.
In `@packages/backend/convex/bias.ts`:
- Around line 107-113: When the code hits the fallback branches that update
source metadata (the branch that checks scored.length < minSamples and the other
failure branch around lines referenced), also clear any stale article-level
outlier state and rolling stats: in addition to updating rollingBiasSampleSize
and rollingBiasUpdatedAt via ctx.db.patch(source._id, ...), call the article
update routine to reset biasOutlierFlag on recent articles for that source and
remove/clear rollingBiasMean and rollingBiasStddev on the source record so no
stale badges remain; apply the same clearing logic in both the scored.length <
minSamples branch and the alternate failure branch referenced (around lines
124–135) so both branches reset biasOutlierFlag and rolling mean/stddev
consistently.
In `@packages/backend/convex/claimDivergenceNode.ts`:
- Around line 263-266: canonicalizeToken currently stems first then consults
TOKEN_SYNONYMS which prevents synonyms like "increased" -> "increase" from
matching; change the order so you consult TOKEN_SYNONYMS before calling
stemToken: look up the raw token (or its normalized lowercase) in TOKEN_SYNONYMS
and use that mapped value if present, otherwise fall back to stemming via
stemToken; update the canonicalizeToken function accordingly (use TOKEN_SYNONYMS
and stemToken).
In `@packages/backend/convex/lib/aiCall.ts`:
- Around line 286-291: The finally block that calls
args.runtime.runMutation(internal.aiBudget.releaseReservation, { reservationId
}) can throw and bubble up, converting expected model errors into unexpected
failures; wrap that call in its own try/catch so any error during cleanup is
caught and handled (log or swallow), ensuring releaseReservation failures do not
reject callOpenAI and that the function still returns the structured { result:
null, usage, error } response when appropriate; keep the existing reservationId
and usageLogged checks and only attempt cleanup when reservationId &&
!usageLogged.
- Around line 143-175: logUsage currently swallows errors and returns void,
causing callers to set usageLogged = true and clear reservationId even when
internal.aiBudget.recordUsage failed; change logUsage to return a boolean (or
throw) indicating success, ensure it returns true only after recordUsage
completes without error and false (or rethrow) on failure, and update callers
that reference usageLogged and reservationId to only set usageLogged = true and
clear reservationId when logUsage returns true (or after the awaited call
succeeds) so reservations aren't leaked on failed mutations.
In `@packages/backend/convex/summarization.ts`:
- Around line 188-193: The check that blocks enqueuing treats undefined
leaseExpiresAt as Number.POSITIVE_INFINITY, so exhausted failed jobs
(leaseExpiresAt undefined, nextAttemptAt MAX_SAFE_INTEGER) appear active
forever; update the condition in the getLatestSummaryJob handling (where
(latestJob.leaseExpiresAt ?? Number.POSITIVE_INFINITY) > now is used) to treat a
missing lease as already expired (e.g., use 0 or Number.NEGATIVE_INFINITY
instead of POSITIVE_INFINITY) so a failed/exhausted job no longer blocks
re-enqueueing; apply the same fix to the identical check in the other occurrence
(around the 508-516 block) and keep references to ACTIVE_JOB_STATUSES,
latestJob.leaseExpiresAt, nextAttemptAt, getLatestSummaryJob, and
markSummaryJobFailed in mind when making the change.
- Around line 224-287: The handler for listDueSummaryJobs has several
syntax/scope errors: remove the duplicate const now declaration, declare and
initialize jobs as an array before pushing (e.g., let jobs: EventSummaryJob[] =
[]), remove or relocate the erroneous event/eventId lookup (either derive
eventId from each job or accept it as an arg) so event and eventId are not
referenced out of scope, fix the premature return that interrupts the
failed-query chain (ensure the queued/failed queries complete, pushing results
into jobs, before returning), and restore correct variable use for queued and
failed query results using the
ctx.db.query("eventSummaryJobs").withIndex(...).take(...) results; after
collecting jobs return the mapped job summary objects. Ensure references to
eligibility are handled per-event or removed if inapplicable.
In `@packages/backend/convex/summarizationNode.ts`:
- Around line 549-550: The finally block currently awaits shutdownPostHog(),
which can reject and override the actual job outcome set by
applyEventSummaryResult or markSummaryJobFailed; change the finally to catch and
log any shutdownPostHog errors instead of letting them propagate (i.e., wrap the
await shutdownPostHog() call in a try/catch and log the error via the existing
logger/processLogger) so cleanup failures don't replace the committed job
result.
---
Duplicate comments:
In `@packages/backend/convex/summarization.ts`:
- Around line 173-175: The loop currently breaks based on inspected >= safeLimit
which stops after scanning safeLimit events rather than stopping when you've
actually enqueued the desired number; change the loop to break when the output
queue/enqueued count reaches the target (e.g., check summariesQueue.length or an
enqueuedCount variable) and only use inspected as a secondary safety cap. Locate
the for (const event of events) loop and replace the primary break condition
from inspected >= safeLimit to a check like queue.length >= safeLimit (or
enqueuedCount >= safeLimit), keeping the .take(safeLimit * 3) fetch and the
inspected counter as an optional fallback to avoid infinite loops.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 2d88ff0a-ee22-4fef-a38c-6dd34c36c461
📒 Files selected for processing (10)
packages/backend/convex/aiBudget.tspackages/backend/convex/bias.tspackages/backend/convex/claimDivergenceNode.tspackages/backend/convex/crons.tspackages/backend/convex/enrichmentNode.tspackages/backend/convex/lib/aiCall.tspackages/backend/convex/lib/openai.tspackages/backend/convex/schema.tspackages/backend/convex/summarization.tspackages/backend/convex/summarizationNode.ts
Contributor
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@packages/backend/convex/aiBudget.ts`:
- Around line 213-219: The reservation mutation currently runs an unbounded
ctx.db.query("aiBudgetReservations").withIndex("by_expiresAt", q =>
q.lte("expiresAt", now)).collect() and then deletes everything, which is a
hot-path O(N) cleanup that can exhaust Convex quotas; remove that delete loop
from the reservation hot path and rely on the existing expiresAt filter (e.g.,
use only rows with expiresAt > now when computing available budget). Implement a
separate bounded background cleanup job (e.g.,
cleanupExpiredAiBudgetReservations) that runs off the hot path and deletes
expired rows in small batches (use the same index "by_expiresAt" with a
limit/page loop of N items per run) via a scheduled worker or maintenance
endpoint, ensuring the reservation function (where
ctx.db.query("aiBudgetReservations") is used) only performs filtered reads
without collect()-driven global deletion.
In `@packages/backend/convex/claimDivergence.ts`:
- Around line 96-112: The loop currently breaks based on inspected but should
stop when refreshed reaches the scan limit; change the termination check from
using inspected to using refreshed so the backfill can skip already-covered
events and continue scanning until it actually refreshes `limit` items. Keep
incrementing `inspected` for visibility, but replace the leading/inner condition
`if (inspected >= limit) break;` with `if (refreshed >= limit) break;` (ensure
`refreshed++` still happens after the `await refreshEventClaimCoverage(ctx,
event._id)` call and that the `includeExisting`/skip logic remains unchanged).
In `@packages/backend/convex/summarizationNode.ts`:
- Around line 532-547: The current retry logic schedules a retry even when
markSummaryJobFailed did not actually update the job; change the check so we
only schedule ctx.scheduler.runAfter(...) when the mutation reported an update
and attempts are not exhausted by using failedResult.updated &&
!failedResult.attemptsExhausted; treat failedResult.updated === false as
terminal (do not retry). Ensure this uses the result returned by
internal.summarization.markSummaryJobFailed (the failedResult variable) and
applies to the branch that would call
internal.summarizationNode.processSummaryJob with jobId and runId.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 52d7b6a5-e153-412f-bb79-45a1f344d3dc
📒 Files selected for processing (11)
packages/backend/convex/aiBudget.tspackages/backend/convex/bias.tspackages/backend/convex/claimDivergence.tspackages/backend/convex/claimDivergenceNode.tspackages/backend/convex/crons.tspackages/backend/convex/enrichment.tspackages/backend/convex/enrichmentNode.tspackages/backend/convex/lib/aiCall.tspackages/backend/convex/schema.tspackages/backend/convex/summarization.tspackages/backend/convex/summarizationNode.ts
This was referenced May 9, 2026
Merged
Merged
This was referenced Jul 8, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary by CodeRabbit
New Features
Improvements