feat(dry-run): implement /internal/scheduler/dry-run-replay (was a v0.1 stub)#14
Conversation
….1 stub) Hard lock #3 — dry-run preview before Activate — was previously a no-op. Now, when a rule is compiled into the draft, the scheduled job replays the most recent posts through that draft rule (pure evaluation, ZERO actions), and writes a `${sub}:dryrun:${ruleId}` summary { status, sampledPosts, matched:[{thingId, authorName, would:[actions]}], note } that the Dashboard renders ("Dry-run preview (draft rules): r_x: would match N/M recent post(s) → modqueue"). - Samples up to 10 recent posts via reddit.getNewPosts({subredditName, limit}).all() → buildPostFactBag → selectMatchingRules([rule], 'onPostSubmit'|'onPostReport', facts). - A comment-only rule (no post trigger) gets status 'unavailable' with a "shadow mode it to see real comments" note (v0.1 has no getNewComments). - Any Reddit-API/rule failure → status 'unavailable' with the error in the note; the handler always returns { status: 'ok' } (never throws — scheduler jobs must). - Result TTL 7 days. Dashboard reads `${sub}:dryrun:${id}` for each draft rule id. Tests: routes-scheduler.test.ts gains 6 cases (no-draft, post-rule with matches, zero matches, comment-only, Reddit-API failure, missing ruleId); routes-dashboard.test.ts gains a dry-run-display case. test/devvit-testkit.ts gains a `getNewPosts` double. 157 tests pass; tsc/lint/format clean; acceptance 4/4. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
|
Warning Rate limit exceeded
You’ve run out of usage credits. Purchase more in the billing tab. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (5)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request implements a dry-run preview feature for draft rules, allowing moderators to see how new rules would have matched against recent posts before activation. The implementation includes a new scheduler endpoint to perform the replay, dashboard updates to display results, and comprehensive tests. Feedback focuses on correcting the use of the selftext property for post content (instead of body), adding error handling for JSON parsing of Redis data to prevent dashboard crashes, and ensuring the scheduler uses the provided subreddit name from the task payload.
| { | ||
| id: p.id, | ||
| title: p.title, | ||
| body: p.body ?? '', |
There was a problem hiding this comment.
In the Devvit SDK, the content of a post is stored in the selftext property, not body. Using p.body will result in an empty string for all sampled posts, causing the dry-run evaluation to be inaccurate for rules that inspect post content.
| body: p.body ?? '', | |
| body: p.selftext ?? '', |
| for (const r of draft?.rules ?? []) { | ||
| const raw = await redis.get(`${subredditName}:dryrun:${r.id}`); | ||
| if (!raw) continue; | ||
| const d = JSON.parse(raw) as DryRunResult; |
There was a problem hiding this comment.
Parsing JSON from Redis without a try-catch block can lead to a full application crash if the stored data is malformed or corrupted. Since this is in the dashboard route, it would prevent moderators from accessing the tool's management interface.
let d: DryRunResult;
try {
d = JSON.parse(raw) as DryRunResult;
} catch (e) {
continue;
}| // Full implementation reads rules:draft, builds factBags, evaluates, writes simulated audits. | ||
| const body = await c.req.json<TaskRequest<{ ruleId: string; subredditName: string }>>(); | ||
| const ruleId = body.data?.ruleId; | ||
| const sub = getCurrentSubredditName(); |
There was a problem hiding this comment.
The subredditName is explicitly passed in the task payload (body.data). It is safer to use the passed value as the primary source, falling back to the context helper if necessary, to ensure the job operates on the intended subreddit data.
| const sub = getCurrentSubredditName(); | |
| const sub = body.data?.subredditName ?? getCurrentSubredditName(); |
| const post = (id: string, over: Record<string, unknown> = {}) => ({ | ||
| id, | ||
| title: 'a title', | ||
| body: 'a body', |
There was a problem hiding this comment.
The test mock should use selftext instead of body to accurately reflect the structure of a Reddit Post object in the Devvit SDK. This ensures the tests are validating the actual runtime behavior.
| body: 'a body', | |
| selftext: 'a body', |
feat(dry-run): implement /internal/scheduler/dry-run-replay (was a v0.1 stub)
1 participant
Hard lock #3 (dry-run preview before Activate) was a no-op. Now the scheduled job — kicked off when a rule is compiled into the draft — replays the most recent posts through that draft rule (pure evaluation, zero actions) and writes a
${sub}:dryrun:${ruleId}summary the Dashboard renders.reddit.getNewPosts({subredditName, limit}).all()→buildPostFactBag→selectMatchingRules([rule], 'onPostSubmit'|'onPostReport', facts).status: 'unavailable'with a "shadow-mode it to see real comments" note (v0.1 has nogetNewComments).status: 'unavailable'with the error in the note; always returns{status:'ok'}(scheduler jobs must not throw). Result TTL 7d.${sub}:dryrun:${id}per draft rule.Tests: +6 in routes-scheduler.test.ts, +1 in routes-dashboard.test.ts,
getNewPostsdouble in test/devvit-testkit.ts. 157 tests pass; tsc/lint/format clean; acceptance 4/4;vite build→ loadable CJS.(Still pending of the original 'cover everything' list: @devvit/test adoption, fast-check PBT, marketingAssets.icon, hackathon-audit run — separate PRs.)
🤖 Generated with Claude Code