fix(pipeline): repair Agerpres feed, archive lock starvation, enrichment cadence

[flvvius]

Context

Production Convex audit found three live pipeline issues. Claim divergence and Apple auth were confirmed intentionally off and are left untouched.

Changes

1. Agerpres feed permanently quarantined → fixed

www.agerpres.ro/rss/stiri server-side 301-redirects to an allorigins.win proxy wrapping a bazqux scraper, and the allorigins layer returns 520/522 (dead) — keeping the feed quarantined and every ingest run degraded. Agerpres has no native RSS. Pointed the feed straight at the underlying bazqux createfeed extractor, which returns valid RSS (verified: 21 real articles dated today).

• Safe: sources row is keyed by domain (agerpres.ro, unchanged); quarantine state is keyed by feedUrl, so the new URL starts un-quarantined.

2. archiveStaleSingletonEvents skipped 100% of runs → fixed

Root cause is cadence arithmetic, not bad luck: Convex interval crons are epoch-phase-aligned, and the job's hourly cadence is an exact multiple of the 20-min merge (60=3×20) and 30-min recluster (60=2×30) jobs — so it fired simultaneously with them every time and always hit a blocking pipeline lock (verified in run logs).

• crons.ts: { hours: 1 } → { minutes: 53 } (coprime with 20/30/40/60) so archive drifts into quiet windows. The core clustering pipeline keeps priority — not lease priority, which would starve it and not remove the concurrent-mutation hazard the lock guards.
• pipeline.ts: skipped runs no longer count as failures in the job_error_rate SLO (a yield is a no-op). The archive_run_absent check still catches a genuinely stuck job.

3. Enrichment cadence 40m → 30m

Works down the unprocessed-article backlog faster.

Testing

• tsc --noEmit clean
• feeds.test.ts + feeds.smoke.test.ts pass (14 passed, 4 pre-existing skips)
• Agerpres extractor endpoint verified returning live RSS

🤖 Generated with Claude Code

Summary by CodeRabbit

• Bug Fixes
  • Improved pipeline alert accuracy by excluding intentionally skipped runs from job success-rate checks, reducing false error alerts.
  • Updated a news feed source to use a more reliable endpoint, helping prevent broken or dead feed responses.
  • Adjusted scheduled processing intervals so content enrichment and stale-event cleanup run more consistently and with better timing.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
news	Error		Jul 7, 2026 2:47pm
news-web	Ready	Preview, Comment	Jul 7, 2026 2:47pm

[coderabbitai]

Walkthrough

Cron schedules for enrich-articles and archive-stale-singleton-events were adjusted with updated comments. The Agerpres RSS feed URL was replaced with a Bazqux extractor proxy URL. checkPipelineAlerts now skips runs with status "skipped" when calculating per-job success ratios.

Changes

Backend maintenance tweaks

Layer / File(s)	Summary
Cron interval updates packages/backend/convex/crons.ts	The enrich-articles interval changes from 40 to 30 minutes, and archive-stale-singleton-events changes from hourly to a 53-minute interval, with updated comments.
Agerpres feed URL replacement packages/backend/convex/feeds.ts	The direct Agerpres RSS URL is replaced with a Bazqux extractor proxy URL, with a rewritten comment explaining the reason.
Pipeline alert skip handling packages/backend/convex/pipeline.ts	checkPipelineAlerts now skips runs with status "skipped" so they no longer contribute to per-job success-rate calculations used for job_error_rate alerts.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related PRs

• flvvius/news#31: Introduced the pipeline alerts/run-logs framework whose checkPipelineAlerts logic is directly modified here.
• flvvius/news#32: Also modifies checkPipelineAlerts alert evaluation logic in the same file.

Poem

A rabbit tweaks the clockwork gears,
Thirty minutes, fifty-three appears,
A feed URL swapped for one that's true,
Skipped runs no longer skew the queue,
Small fixes hopping, steady and clear. 🐇

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly summarizes the three main pipeline fixes: Agerpres feed repair, archive lock starvation, and enrichment cadence changes.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/convex-pipeline-health

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

packages/backend/convex/pipeline.ts (1)

953-966: 🩺 Stability & Availability | 🟠 Major | 🏗️ Heavy lift

Alerting gap for jobs that only emit skipped.

Excluding skipped logs from byJob means a job that keeps short-circuiting never enters job_error_rate:<jobName>, so it can fail silently. Only archiveStaleSingletonEvents has a separate absence alert here; other jobs don’t appear to have one.

Either keep skipped in the denominator for jobs without a dedicated absence check, or add a generic no-ok alert per job.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/backend/convex/pipeline.ts` around lines 953 - 966, The per-job
error-rate aggregation in pipeline.ts is dropping all skipped runs, which lets
jobs that only short-circuit disappear from job_error_rate alerts. Update the
byJob accumulation around the logs loop so skipped entries are either counted in
the denominator for jobs that don’t have a dedicated absence check, or add a
generic no-ok alert path for every job. Keep the existing special handling for
archiveStaleSingletonEvents and ensure the logic around byJob, log.status, and
the job_error_rate:<jobName> metric still reflects skipped-only behavior.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@packages/backend/convex/pipeline.ts`:
- Around line 953-966: The per-job error-rate aggregation in pipeline.ts is
dropping all skipped runs, which lets jobs that only short-circuit disappear
from job_error_rate alerts. Update the byJob accumulation around the logs loop
so skipped entries are either counted in the denominator for jobs that don’t
have a dedicated absence check, or add a generic no-ok alert path for every job.
Keep the existing special handling for archiveStaleSingletonEvents and ensure
the logic around byJob, log.status, and the job_error_rate:<jobName> metric
still reflects skipped-only behavior.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 711cc59d-e21f-4984-9971-4712ff077bbe

📥 Commits

Reviewing files that changed from the base of the PR and between 36b8972 and d72e262.

📒 Files selected for processing (3)

• packages/backend/convex/crons.ts
• packages/backend/convex/feeds.ts
• packages/backend/convex/pipeline.ts