-
Notifications
You must be signed in to change notification settings - Fork 9
t1455: add adaptive pulse queue-governor and strict worker counting #4218
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
|
@@ -129,6 +129,7 @@ PULSE_MODEL="${PULSE_MODEL:-}" | |||||||||||||||||||||||||||||
| HEADLESS_RUNTIME_HELPER="${HEADLESS_RUNTIME_HELPER:-${SCRIPT_DIR}/headless-runtime-helper.sh}" | ||||||||||||||||||||||||||||||
| REPOS_JSON="${REPOS_JSON:-${HOME}/.config/aidevops/repos.json}" | ||||||||||||||||||||||||||||||
| STATE_FILE="${HOME}/.aidevops/logs/pulse-state.txt" | ||||||||||||||||||||||||||||||
| QUEUE_METRICS_FILE="${HOME}/.aidevops/logs/pulse-queue-metrics" | ||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
| if [[ ! -x "$HEADLESS_RUNTIME_HELPER" ]]; then | ||||||||||||||||||||||||||||||
| printf '[pulse-wrapper] ERROR: headless runtime helper is missing or not executable: %s (SCRIPT_DIR=%s)\n' "$HEADLESS_RUNTIME_HELPER" "$SCRIPT_DIR" >&2 | ||||||||||||||||||||||||||||||
|
|
@@ -347,6 +348,9 @@ prefetch_state() { | |||||||||||||||||||||||||||||
| # Append priority-class worker allocations (t1423) | ||||||||||||||||||||||||||||||
| _append_priority_allocations >>"$STATE_FILE" | ||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
| # Append adaptive queue-governor guidance (t1455) | ||||||||||||||||||||||||||||||
| append_adaptive_queue_governor | ||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
| # Export PULSE_SCOPE_REPOS — comma-separated list of repo slugs that | ||||||||||||||||||||||||||||||
| # workers are allowed to create PRs/branches on (t1405, GH#2928). | ||||||||||||||||||||||||||||||
| # Workers CAN file issues on any repo (cross-repo self-improvement), | ||||||||||||||||||||||||||||||
|
|
@@ -1678,11 +1682,111 @@ prefetch_contribution_watch() { | |||||||||||||||||||||||||||||
| ####################################### | ||||||||||||||||||||||||||||||
| count_active_workers() { | ||||||||||||||||||||||||||||||
| local count | ||||||||||||||||||||||||||||||
| count=$(ps axo command | grep '[/]full-loop' | grep -c '[.]opencode') || count=0 | ||||||||||||||||||||||||||||||
| count=$(ps axo command | grep '\.opencode run' | grep '/full-loop Implement issue #' | grep -v '/pulse' | grep -c -v grep) || count=0 | ||||||||||||||||||||||||||||||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The use of
Suggested change
|
||||||||||||||||||||||||||||||
| echo "$count" | ||||||||||||||||||||||||||||||
| return 0 | ||||||||||||||||||||||||||||||
| } | ||||||||||||||||||||||||||||||
|
Comment on lines
1683
to
1688
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🧩 Analysis chain🏁 Script executed: #!/bin/bash
# Check what worker dispatch patterns exist in the codebase
echo "=== Dispatch patterns in pulse.md ==="
rg --no-heading -n '/full-loop' .agents/scripts/commands/pulse.md | head -30
echo ""
echo "=== Mission worker dispatch patterns ==="
rg --no-heading -n 'headless-runtime-helper.sh run' .agents/scripts/commands/pulse.md | head -20
echo ""
echo "=== count_active_workers implementations ==="
rg --no-heading -A5 'count_active_workers\(\)' .agents/scripts/Repository: marcusquinn/aidevops Length of output: 3918 🏁 Script executed: #!/bin/bash
# Check how count_active_workers is used in pulse-wrapper.sh
echo "=== Usage of count_active_workers in pulse-wrapper.sh ==="
rg --no-heading -B5 -A5 'count_active_workers\(\)' .agents/scripts/pulse-wrapper.sh | head -50
echo ""
echo "=== MAX_WORKERS and dispatch logic in pulse-wrapper.sh ==="
rg --no-heading -n 'MAX_WORKERS\|WORKER_COUNT\|count_active_workers' .agents/scripts/pulse-wrapper.sh | head -30
echo ""
echo "=== Check if pulse-wrapper.sh dispatches mission workers ==="
rg --no-heading -n 'mission\|Mission' .agents/scripts/pulse-wrapper.sh | head -20Repository: marcusquinn/aidevops Length of output: 584 🏁 Script executed: #!/bin/bash
# Search for where count_active_workers is actually called
echo "=== Search for count_active_workers function calls ==="
rg --no-heading -n 'count_active_workers' .agents/scripts/pulse-wrapper.sh
echo ""
echo "=== Check what pulse-wrapper.sh dispatches ==="
rg --no-heading -n 'headless-runtime-helper\|\.opencode run' .agents/scripts/pulse-wrapper.sh | head -20
echo ""
echo "=== Get overall structure/purpose of pulse-wrapper.sh ==="
head -100 .agents/scripts/pulse-wrapper.sh | grep -E '^\#|^[A-Za-z_]+\(\)|script'
echo ""
echo "=== Check if count_active_workers is called anywhere ==="
rg --no-heading -n '\$count_active_workers\|count_active_workers' .agents/scripts/ --type shRepository: marcusquinn/aidevops Length of output: 3237 🏁 Script executed: #!/bin/bash
# Check context around line 1954 where count_active_workers is called
echo "=== Context around line 1954 (active_workers=$(count_active_workers)) ==="
sed -n '1945,1975p' .agents/scripts/pulse-wrapper.sh
echo ""
echo "=== Check for any logging or usage of active_workers variable ==="
rg --no-heading -B2 -A2 'active_workers=' .agents/scripts/pulse-wrapper.shRepository: marcusquinn/aidevops Length of output: 2135 🏁 Script executed: #!/bin/bash
# Check if mission/PR workers are dispatched from other contexts
echo "=== Check all worker dispatch points ==="
rg --no-heading -n 'headless-runtime-helper.sh run' .agents/scripts/*.sh | grep -v pulse.md
echo ""
echo "=== Check if there's queue enforcement for all worker types ==="
rg --no-heading -B5 -A5 'max_workers\|MAX_WORKERS' .agents/scripts/pulse-wrapper.sh | head -40
echo ""
echo "=== Check purpose/scope comments in pulse-wrapper.sh header ==="
head -50 .agents/scripts/pulse-wrapper.sh | tail -20Repository: marcusquinn/aidevops Length of output: 1370 🏁 Script executed: #!/bin/bash
# Check if pulse-wrapper.sh dispatches any workers or just manages the pulse
echo "=== Search for dispatch/worker dispatch in pulse-wrapper.sh ==="
rg --no-heading -n 'dispatch\|queue\|Implement issue' .agents/scripts/pulse-wrapper.sh | head -20
echo ""
echo "=== Check if count_runnable_candidates matches same pattern ==="
rg --no-heading -B2 -A8 'count_runnable_candidates\(\)' .agents/scripts/pulse-wrapper.sh
echo ""
echo "=== Check what queued issues look like ==="
rg --no-heading -B2 -A8 'count_queued_without_worker\(\)' .agents/scripts/pulse-wrapper.shRepository: marcusquinn/aidevops Length of output: 785 🏁 Script executed: #!/bin/bash
# Check where mission and PR fix workers are actually dispatched from
echo "=== Check what pulse-wrapper.sh actually calls/dispatches ==="
rg --no-heading -n 'function\|^[a-z_]*() {' .agents/scripts/pulse-wrapper.sh | head -30
echo ""
echo "=== Look for dispatch_* or similar functions in pulse-wrapper ==="
rg --no-heading -B3 -A10 'dispatch\|enqueue' .agents/scripts/pulse-wrapper.sh | head -50
echo ""
echo "=== Check if there's a separate handler for mission/PR workers ==="
ls -la .agents/scripts/*.sh | grep -E 'mission|pr|queue|dispatch'
echo ""
echo "=== Look at mission-dashboard-helper to see if it has its own enforcement ==="
rg --no-heading -n 'max_workers\|count_active_workers' .agents/scripts/mission-dashboard-helper.shRepository: marcusquinn/aidevops Length of output: 2376 🏁 Script executed: #!/bin/bash
# Check function definitions without problematic regex
echo "=== Main functions in pulse-wrapper.sh ==="
rg --no-heading -n '^[a-z_]+\(\)' .agents/scripts/pulse-wrapper.sh
echo ""
echo "=== Check if mission-dashboard-helper has worker enforcement ==="
rg --no-heading -B5 -A10 'count_active_workers\|max_workers' .agents/scripts/mission-dashboard-helper.sh | head -50
echo ""
echo "=== Check cron-dispatch.sh for mission/PR dispatch logic ==="
rg --no-heading -n 'mission\|PR\|Address unresolved' .agents/scripts/cron-dispatch.sh | head -20Repository: marcusquinn/aidevops Length of output: 1220 🏁 Script executed: #!/bin/bash
# Check what prefetch_active_workers does
echo "=== prefetch_active_workers function ==="
sed -n '880,940p' .agents/scripts/pulse-wrapper.sh
echo ""
echo "=== Check append_adaptive_queue_governor ==="
sed -n '1697,1750p' .agents/scripts/pulse-wrapper.sh
echo ""
echo "=== Check if pulse-wrapper.sh has any worker TYPE distinction ==="
rg --no-heading -n 'issue\|mission\|PR.*fix\|Address unresolved' .agents/scripts/pulse-wrapper.sh | head -20Repository: marcusquinn/aidevops Length of output: 4408 🏁 Script executed: #!/bin/bash
# Check if enforce_utilization_invariants actually dispatches workers
echo "=== Full enforce_utilization_invariants function ==="
sed -n '1942,2000p' .agents/scripts/pulse-wrapper.sh
echo ""
echo "=== Check what calls enforce_utilization_invariants ==="
rg --no-heading -n 'enforce_utilization_invariants' .agents/scripts/pulse-wrapper.sh
echo ""
echo "=== Check if pulse-wrapper dispatches PR fix or mission workers ==="
rg --no-heading -B3 -A3 'PR\|mission\|Address unresolved\|Implement.*task' .agents/scripts/pulse-wrapper.sh | head -40Repository: marcusquinn/aidevops Length of output: 2780 🏁 Script executed: #!/bin/bash
# Clarify: check if the narrow pattern is intentional by seeing how prefetch_active_workers is used
echo "=== Where is prefetch_active_workers output used? ==="
rg --no-heading -B5 -A5 'prefetch_active_workers' .agents/scripts/pulse-wrapper.sh | grep -v '^--$'
echo ""
echo "=== Check if mission workers or PR fix workers are explicitly SEPARATE from issue workers ==="
rg --no-heading -n 'run_pulse\|count_runnable_candidates\|count_queued_without_worker' .agents/scripts/pulse-wrapper.sh | head -5
echo ""
echo "=== Look at what run_pulse actually does ==="
sed -n '1289,1350p' .agents/scripts/pulse-wrapper.sh
echo ""
echo "=== Check if there's documentation about worker types in pulse.md ==="
head -100 .agents/scripts/commands/pulse.md | tail -50Repository: marcusquinn/aidevops Length of output: 8189 Backfill capacity check should account for all active worker types, not just issue workers. The pattern While 🤖 Prompt for AI Agents |
||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
| ####################################### | ||||||||||||||||||||||||||||||
| # Append adaptive queue-governor guidance to pre-fetched state | ||||||||||||||||||||||||||||||
| # | ||||||||||||||||||||||||||||||
| # Uses observed queue totals and trend vs previous cycle to derive an | ||||||||||||||||||||||||||||||
| # adaptive PR-vs-issue dispatch focus. This avoids static per-repo | ||||||||||||||||||||||||||||||
| # thresholds and shifts effort toward PR burn-down when PR backlog grows. | ||||||||||||||||||||||||||||||
| ####################################### | ||||||||||||||||||||||||||||||
| append_adaptive_queue_governor() { | ||||||||||||||||||||||||||||||
| if [[ ! -f "$STATE_FILE" ]]; then | ||||||||||||||||||||||||||||||
| return 0 | ||||||||||||||||||||||||||||||
| fi | ||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
| local total_prs total_issues ready_prs failing_prs | ||||||||||||||||||||||||||||||
| total_prs=$(awk '/^### Open PRs \([0-9]+\)/ { line=$0; gsub(/[^0-9]/, "", line); sum+=line } END { print sum+0 }' "$STATE_FILE") | ||||||||||||||||||||||||||||||
| total_issues=$(awk '/^### Open Issues \([0-9]+\)/ { line=$0; gsub(/[^0-9]/, "", line); sum+=line } END { print sum+0 }' "$STATE_FILE") | ||||||||||||||||||||||||||||||
| ready_prs=$(rg -c '\[checks: PASS\].*\[review: APPROVED\]' "$STATE_FILE" 2>/dev/null || echo "0") | ||||||||||||||||||||||||||||||
| failing_prs=$(rg -c '\[checks: FAIL\]|\[review: CHANGES_REQUESTED\]' "$STATE_FILE" 2>/dev/null || echo "0") | ||||||||||||||||||||||||||||||
|
Comment on lines
+1705
to
+1706
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The script relies on
Suggested change
References
Comment on lines
+1705
to
+1706
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🛠️ Refactor suggestion | 🟠 Major Use The script uses ♻️ Suggested fix using grep- ready_prs=$(rg -c '\[checks: PASS\].*\[review: APPROVED\]' "$STATE_FILE" 2>/dev/null || echo "0")
- failing_prs=$(rg -c '\[checks: FAIL\]|\[review: CHANGES_REQUESTED\]' "$STATE_FILE" 2>/dev/null || echo "0")
+ ready_prs=$(grep -cE '\[checks: PASS\].*\[review: APPROVED\]' "$STATE_FILE" 2>/dev/null || echo "0")
+ failing_prs=$(grep -cE '\[checks: FAIL\]|\[review: CHANGES_REQUESTED\]' "$STATE_FILE" 2>/dev/null || echo "0")📝 Committable suggestion
Suggested change
🤖 Prompt for AI Agents |
||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
| [[ "$total_prs" =~ ^[0-9]+$ ]] || total_prs=0 | ||||||||||||||||||||||||||||||
| [[ "$total_issues" =~ ^[0-9]+$ ]] || total_issues=0 | ||||||||||||||||||||||||||||||
| [[ "$ready_prs" =~ ^[0-9]+$ ]] || ready_prs=0 | ||||||||||||||||||||||||||||||
| [[ "$failing_prs" =~ ^[0-9]+$ ]] || failing_prs=0 | ||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
| local prev_total_prs=0 prev_total_issues=0 prev_ready_prs=0 prev_failing_prs=0 | ||||||||||||||||||||||||||||||
| if [[ -f "$QUEUE_METRICS_FILE" ]]; then | ||||||||||||||||||||||||||||||
| while IFS='=' read -r key value; do | ||||||||||||||||||||||||||||||
| case "$key" in | ||||||||||||||||||||||||||||||
| prev_total_prs) prev_total_prs="$value" ;; | ||||||||||||||||||||||||||||||
| prev_total_issues) prev_total_issues="$value" ;; | ||||||||||||||||||||||||||||||
| prev_ready_prs) prev_ready_prs="$value" ;; | ||||||||||||||||||||||||||||||
| prev_failing_prs) prev_failing_prs="$value" ;; | ||||||||||||||||||||||||||||||
| esac | ||||||||||||||||||||||||||||||
| done <"$QUEUE_METRICS_FILE" | ||||||||||||||||||||||||||||||
| fi | ||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
| [[ "$prev_total_prs" =~ ^-?[0-9]+$ ]] || prev_total_prs=0 | ||||||||||||||||||||||||||||||
| [[ "$prev_total_issues" =~ ^-?[0-9]+$ ]] || prev_total_issues=0 | ||||||||||||||||||||||||||||||
| [[ "$prev_ready_prs" =~ ^-?[0-9]+$ ]] || prev_ready_prs=0 | ||||||||||||||||||||||||||||||
| [[ "$prev_failing_prs" =~ ^-?[0-9]+$ ]] || prev_failing_prs=0 | ||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
| local pr_delta issue_delta ready_delta failing_delta | ||||||||||||||||||||||||||||||
| pr_delta=$((total_prs - prev_total_prs)) | ||||||||||||||||||||||||||||||
| issue_delta=$((total_issues - prev_total_issues)) | ||||||||||||||||||||||||||||||
| ready_delta=$((ready_prs - prev_ready_prs)) | ||||||||||||||||||||||||||||||
| failing_delta=$((failing_prs - prev_failing_prs)) | ||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
| local denominator pr_share_pct growth_bias pr_focus_pct new_issue_pct | ||||||||||||||||||||||||||||||
| denominator=$((total_prs + total_issues)) | ||||||||||||||||||||||||||||||
| if [[ "$denominator" -lt 1 ]]; then | ||||||||||||||||||||||||||||||
| denominator=1 | ||||||||||||||||||||||||||||||
| fi | ||||||||||||||||||||||||||||||
| pr_share_pct=$(((total_prs * 100) / denominator)) | ||||||||||||||||||||||||||||||
| growth_bias=0 | ||||||||||||||||||||||||||||||
| if [[ "$pr_delta" -gt 0 ]]; then | ||||||||||||||||||||||||||||||
| growth_bias=10 | ||||||||||||||||||||||||||||||
| elif [[ "$pr_delta" -lt 0 ]]; then | ||||||||||||||||||||||||||||||
| growth_bias=-5 | ||||||||||||||||||||||||||||||
| fi | ||||||||||||||||||||||||||||||
| pr_focus_pct=$((35 + (pr_share_pct / 2) + growth_bias)) | ||||||||||||||||||||||||||||||
| if [[ "$pr_focus_pct" -lt 35 ]]; then | ||||||||||||||||||||||||||||||
| pr_focus_pct=35 | ||||||||||||||||||||||||||||||
| elif [[ "$pr_focus_pct" -gt 85 ]]; then | ||||||||||||||||||||||||||||||
| pr_focus_pct=85 | ||||||||||||||||||||||||||||||
| fi | ||||||||||||||||||||||||||||||
| new_issue_pct=$((100 - pr_focus_pct)) | ||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
| local queue_mode | ||||||||||||||||||||||||||||||
| queue_mode="balanced" | ||||||||||||||||||||||||||||||
| if [[ "$ready_prs" -gt 0 && "$pr_delta" -ge 0 ]]; then | ||||||||||||||||||||||||||||||
| queue_mode="merge-heavy" | ||||||||||||||||||||||||||||||
| elif [[ "$pr_focus_pct" -ge 60 ]]; then | ||||||||||||||||||||||||||||||
| queue_mode="pr-heavy" | ||||||||||||||||||||||||||||||
| fi | ||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
| cat >"$QUEUE_METRICS_FILE" <<EOF | ||||||||||||||||||||||||||||||
| prev_total_prs=${total_prs} | ||||||||||||||||||||||||||||||
| prev_total_issues=${total_issues} | ||||||||||||||||||||||||||||||
| prev_ready_prs=${ready_prs} | ||||||||||||||||||||||||||||||
| prev_failing_prs=${failing_prs} | ||||||||||||||||||||||||||||||
| EOF | ||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
| { | ||||||||||||||||||||||||||||||
| echo "" | ||||||||||||||||||||||||||||||
| echo "## Adaptive Queue Governor" | ||||||||||||||||||||||||||||||
| echo "- Queue totals: PRs=${total_prs} (delta ${pr_delta}), issues=${total_issues} (delta ${issue_delta})" | ||||||||||||||||||||||||||||||
| echo "- PR execution pressure: ready=${ready_prs} (delta ${ready_delta}), failing_or_changes_requested=${failing_prs} (delta ${failing_delta})" | ||||||||||||||||||||||||||||||
| echo "- Adaptive mode this cycle: ${queue_mode}" | ||||||||||||||||||||||||||||||
| echo "- Recommended dispatch focus: PR remediation ${pr_focus_pct}% / new issue dispatch ${new_issue_pct}%" | ||||||||||||||||||||||||||||||
| echo "" | ||||||||||||||||||||||||||||||
| echo "PULSE_QUEUE_MODE=${queue_mode}" | ||||||||||||||||||||||||||||||
| echo "PR_REMEDIATION_FOCUS_PCT=${pr_focus_pct}" | ||||||||||||||||||||||||||||||
| echo "NEW_ISSUE_DISPATCH_PCT=${new_issue_pct}" | ||||||||||||||||||||||||||||||
| echo "" | ||||||||||||||||||||||||||||||
| echo "When PR backlog is rising, prioritize merge-ready and failing-check PR advancement before new issue starts." | ||||||||||||||||||||||||||||||
| } >>"$STATE_FILE" | ||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
| echo "[pulse-wrapper] Adaptive queue governor: mode=${queue_mode} prs=${total_prs} issues=${total_issues} pr_focus=${pr_focus_pct}%" >>"$LOGFILE" | ||||||||||||||||||||||||||||||
| return 0 | ||||||||||||||||||||||||||||||
| } | ||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
| ####################################### | ||||||||||||||||||||||||||||||
| # Get current max workers from pulse-max-workers file | ||||||||||||||||||||||||||||||
| # Returns: numeric value via stdout (defaults to 1) | ||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Multi-value grep returns could break variable comparisons.
If
pulse-state.txtcontains multiplePULSE_QUEUE_MODE=lines (e.g., from repeated governor calls or file corruption),grepreturns all matching lines. The variable then contains newlines, causing string comparisons like[[ "$PULSE_QUEUE_MODE" == "pr-heavy" ]]to fail unexpectedly.🛡️ Suggested fix using tail -1 to get the last (most recent) value
📝 Committable suggestion
🤖 Prompt for AI Agents