feat: add quality gate with model escalation to supervisor (t132.6) #788
Conversation
Post-completion quality checks that can re-dispatch to a higher-tier model if output quality is insufficient. Escalation chain: haiku->sonnet->opus, flash->pro. Configurable max_escalation per task and skip_quality_gate per batch. - get_next_tier(): maps current model to next tier in escalation chain - check_output_quality(): heuristic checks (log size, error patterns, file changes, syntax errors, token-to-substance ratio) - run_quality_gate(): orchestrates quality check + escalation decision - Schema: escalation_depth/max_escalation on tasks, skip_quality_gate on batches - Wired into pulse complete handler before accepting completion - --skip-quality-gate flag on batch command for trusted/trivial tasks Fixes: t132.6 ref:GH#738
Summary of ChangesHello @alex-solovyev, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request introduces a robust quality assurance mechanism for worker tasks. It establishes a post-completion quality gate that intelligently assesses the output of a task using various heuristics. If the output fails to meet quality standards, the system can automatically escalate the task by re-queuing it to be processed by a more advanced model, ensuring higher quality outcomes while providing configurable control over the escalation process. Highlights
Changelog
Activity
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
🔍 Code Quality Report�[0;35m[MONITOR]�[0m Code Review Monitoring Report �[0;34m[INFO]�[0m Latest Quality Status: �[0;34m[INFO]�[0m Recent monitoring activity: 📈 Current Quality Metrics
Generated on: Mon Feb 9 17:21:01 UTC 2026 Generated by AI DevOps Framework Code Review Monitoring |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a quality gate with model escalation for the supervisor, a significant feature enhancement. The changes include database schema updates, new command-line options, and the core logic for quality checking and task escalation. The implementation is solid and follows the project's shell scripting conventions. I have one suggestion to improve code clarity by aligning comments and variable names with the actual implementation in the check_output_quality function.
| # Check 5: ShellCheck for .sh files (quick heuristic) | ||
| local changed_sh_files | ||
| changed_sh_files=$(git -C "$tworktree" diff --name-only "main..HEAD" 2>/dev/null | grep '\.sh$' || true) | ||
| if [[ -n "$changed_sh_files" ]]; then | ||
| local shellcheck_errors=0 | ||
| while IFS= read -r sh_file; do | ||
| [[ -z "$sh_file" ]] && continue | ||
| local full_path="${tworktree}/${sh_file}" | ||
| [[ -f "$full_path" ]] || continue | ||
| local sc_count | ||
| sc_count=$(bash -n "$full_path" 2>&1 | wc -l | tr -d ' ') | ||
| shellcheck_errors=$((shellcheck_errors + sc_count)) | ||
| done <<< "$changed_sh_files" | ||
| if [[ "$shellcheck_errors" -gt 5 ]]; then | ||
| echo "fail:syntax_errors_${shellcheck_errors}" | ||
| return 0 | ||
| fi | ||
| fi |
There was a problem hiding this comment.
The comments and variable names in this block refer to "ShellCheck", but the implementation uses bash -n, which only performs a syntax check. This is misleading for future maintenance.
To improve clarity, I suggest updating the comments and variable names to accurately reflect that a syntax check is being performed. This also applies to the function's header comment on line 2953.
| # Check 5: ShellCheck for .sh files (quick heuristic) | |
| local changed_sh_files | |
| changed_sh_files=$(git -C "$tworktree" diff --name-only "main..HEAD" 2>/dev/null | grep '\.sh$' || true) | |
| if [[ -n "$changed_sh_files" ]]; then | |
| local shellcheck_errors=0 | |
| while IFS= read -r sh_file; do | |
| [[ -z "$sh_file" ]] && continue | |
| local full_path="${tworktree}/${sh_file}" | |
| [[ -f "$full_path" ]] || continue | |
| local sc_count | |
| sc_count=$(bash -n "$full_path" 2>&1 | wc -l | tr -d ' ') | |
| shellcheck_errors=$((shellcheck_errors + sc_count)) | |
| done <<< "$changed_sh_files" | |
| if [[ "$shellcheck_errors" -gt 5 ]]; then | |
| echo "fail:syntax_errors_${shellcheck_errors}" | |
| return 0 | |
| fi | |
| fi | |
| # Check 5: Syntax errors in .sh files (quick heuristic) | |
| local changed_sh_files | |
| changed_sh_files=$(git -C "$tworktree" diff --name-only "main..HEAD" 2>/dev/null | grep '\.sh$' || true) | |
| if [[ -n "$changed_sh_files" ]]; then | |
| local syntax_errors=0 | |
| while IFS= read -r sh_file; do | |
| [[ -z "$sh_file" ]] && continue | |
| local full_path="${tworktree}/${sh_file}" | |
| [[ -f "$full_path" ]] || continue | |
| local error_line_count | |
| error_line_count=$(bash -n "$full_path" 2>&1 | wc -l | tr -d ' ') | |
| syntax_errors=$((syntax_errors + error_line_count)) | |
| done <<< "$changed_sh_files" | |
| if [[ "$syntax_errors" -gt 5 ]]; then | |
| echo "fail:syntax_errors_${syntax_errors}" | |
| return 0 | |
| fi | |
| fi |
|
All 8 subtasks of t132 (Cross-Provider Model Routing) are now complete: - t132.1: Model-specific subagents (PR #758) - t132.2: Provider/model registry (PR #761) - t132.3: Model availability checker (PR #770) - t132.4: Fallback chain config (PR #781) - t132.5: Supervisor model resolution (PR #787) - t132.6: Quality gate with escalation (PR #788) - t132.7: Multi-provider runner/cron support (PR #789) - t132.8: Cross-model review workflow (PR #791) Also fixed stale git conflict markers in TODO.md.
1 participant
Summary
max_escalation, default 2) and per-batch (--skip-quality-gate)Quality Checks (
check_output_quality())bash -nfailures on changed .sh files (>5 errors)New Functions
get_next_tier()- Escalation chain mappingcheck_output_quality()- Heuristic quality checksrun_quality_gate()- Orchestrates check + escalation decisionSchema Changes
tasks.escalation_depth(INTEGER, default 0) - Current escalation counttasks.max_escalation(INTEGER, default 2) - Max allowed escalationsbatches.skip_quality_gate(INTEGER, default 0) - Skip gate for batchState Machine
evaluating:queuedtransition for quality gate escalationcmd_transitiontocompleteTesting
bash -nsyntax check: pass-S error: zero violationsFixes: t132.6 ref:GH#738