t1251: Fix frequent stale-evaluating recovery — heartbeat, configurable timeout, PR fast-path

[marcusquinn]

WIP - incremental commits

Root causes of 73% stale-evaluating rate:

1. Hardcoded 60s AI eval timeout — too short under load
2. No heartbeat during eval — Phase 0.7 could not distinguish active eval from dead eval
3. Phase 1 called AI eval even when PR was already in DB (wasted 60-90s)

Fixes:

• Configurable eval timeout via SUPERVISOR_EVAL_TIMEOUT (default 90s)
• Heartbeat: evaluate_with_ai() updates updated_at at start of each eval
• New root cause: eval_in_progress_heartbeat_Ns — Phase 0.7/1c skip recovery
• Phase 1 fast-path: skip AI eval when PR already in DB (t1245 early-persist)

[coderabbitai]

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch feature/t1251

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[github-actions]

🔍 Code Quality Report

�[0;35m[MONITOR]�[0m Code Review Monitoring Report

�[0;34m[INFO]�[0m Latest Quality Status:
SonarCloud: 0 bugs, 0 vulnerabilities, 30 code smells

�[0;34m[INFO]�[0m Recent monitoring activity:
Thu Feb 19 09:10:37 UTC 2026: Code review monitoring started
Thu Feb 19 09:10:38 UTC 2026: SonarCloud - Bugs: 0, Vulnerabilities: 0, Code Smells: 30

📈 Current Quality Metrics

• BUGS: 0
• CODE SMELLS: 30
• VULNERABILITIES: 0

Generated on: Thu Feb 19 09:10:40 UTC 2026

---

Generated by AI DevOps Framework Code Review Monitoring

[gemini-code-assist]

Summary of Changes

Hello @marcusquinn, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly improves the robustness and efficiency of the supervisor's task evaluation process by addressing the root causes of frequent "stale-evaluating" recoveries. It introduces a configurable timeout for AI evaluations, implements a heartbeat mechanism to prevent premature recovery of active evaluations, and optimizes the evaluation flow by skipping AI processing when a pull request has already been persisted. These changes aim to reduce false positives in stale task detection and streamline the overall task lifecycle.

Highlights

• Configurable AI Evaluation Timeout: The hardcoded 60-second AI evaluation timeout was replaced with a configurable SUPERVISOR_EVAL_TIMEOUT (defaulting to 90 seconds) to prevent tasks from getting stuck due to insufficient time under load.
• AI Evaluation Heartbeat: A heartbeat mechanism was introduced in evaluate_with_ai() to update the task's updated_at timestamp at the start of each AI evaluation, preventing premature stale-evaluating recovery by indicating active processing.
• Enhanced Stale-Evaluating Recovery Logic: The _diagnose_stale_root_cause function now identifies tasks actively undergoing AI evaluation via the heartbeat, allowing Phase 0.7 and Phase 1c recovery processes to skip these tasks and avoid unnecessary interruptions.
• AI Evaluation Fast-Path: A fast-path was added to cmd_pulse (Phase 1) to skip AI evaluation for tasks where the PR URL is already present in the database, leveraging early persistence and reducing unnecessary evaluation time.

Changelog

• .agents/scripts/supervisor/evaluate.sh
  • Made the AI evaluation timeout configurable via SUPERVISOR_EVAL_TIMEOUT, defaulting to 90 seconds.
  • Added a log entry at the start of AI evaluation to aid in diagnosing stale tasks.
  • Implemented a database update (updated_at) at the beginning of AI evaluation to serve as a heartbeat.
• .agents/scripts/supervisor/pulse.sh
  • Enhanced _diagnose_stale_root_cause to check for active AI evaluations using the updated_at heartbeat, returning a specific eval_in_progress_heartbeat_* cause.
  • Modified Phase 0.7 of cmd_pulse to skip recovery for tasks identified as actively evaluating by the new heartbeat mechanism.
  • Introduced a fast-path in Phase 1 of cmd_pulse to bypass AI evaluation if a task's PR URL is already stored in the database.
  • Added a safety check in Phase 1c of cmd_pulse to skip recovery for tasks actively evaluating.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[gemini-code-assist]

Code Review

This pull request introduces several important fixes to address tasks getting stuck in an 'evaluating' state. The changes include making the evaluation timeout configurable, adding a heartbeat mechanism to signal that an evaluation is still active, and adding a fast-path to skip AI evaluation if a PR already exists. The changes are well-commented and align with the stated goals.