Skip to content
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
94 changes: 93 additions & 1 deletion .github/workflows/qwen-autofix.yml
Original file line number Diff line number Diff line change
Expand Up @@ -190,6 +190,16 @@ env:
# unbroken run of failures. Observed on #6723: 7 straight failed rounds (3
# timeouts, 4 gate rejections) over 8 hours, heading for 100.
CONSECUTIVE_FAILURE_CAP: '5'
# Cumulative agent-timeout sub-cap, the sibling of the consecutive cap for
# the failure shape it cannot see: timeouts INTERLEAVED with successful
# rounds. A success resets the consecutive streak, but it does not make the
# next timeout any cheaper — each one burns a full agent budget (~50m of
# runner time) and pushes nothing. Observed on #7929: three timeouts with
# pushed rounds in between, so the consecutive cap never fired and the PR
# kept walking into the same wall; #7846 the same, twice. Counted over the
# current counting window (window-scoped like every other census), so a
# re-arm clears it along with the round counter.
TIMEOUT_WINDOW_CAP: '3'
# Do not claim more issues when too many existing autofix PRs are still open.
MAX_OPEN_AUTOFIX_PRS: '5'

Expand Down Expand Up @@ -2998,6 +3008,42 @@ jobs:
echo
printf '%s\n' "${LAST_REJECTION}"
fi
# Time-budget exhaustions SINCE THE LAST SUCCESSFUL ROUND mean
# the standard address-everything prompt is not converging at
# this budget: re-running it unchanged just walks into the same
# wall (#7929 burned three 50-minute timeouts that way, #7846
# two — each a full agent run with nothing pushed). From the
# second attempt on, tell the agent to narrow. Counted since
# the last pushed/no-change round, NOT cumulatively: a push
# falsifies "not converging" and resets the count, so a recovered
# PR stops seeing the warning; until a round pushes or no-ops it
# fires on every failing round (gate rejections included) —
# correctly, since nothing has converged yet. (The
# BREAKER in the report step stays cumulative — a push does not
# make the next timeout cheaper in budget terms.) Window-scoped
# like every other census (LIVE_REARM_KEY is the live window),
# so a re-arm clears it. The needle matches the emitted
# headline verbatim: first lines can embed provider error text
# (API_ERROR_DETAIL), so a loose phrase could count a model
# error message as a timeout.
PRIOR_TIMEOUTS="$(jq -r --arg ab "${AUTOFIX_BOT}" --arg key "${LIVE_REARM_KEY}" '
[ .[] | select((.user.login // "") == $ab)
| select((.body // "") | contains("<!-- autofix-eval "))
| select(((.body // "") | contains("win=" + $key + " -->"))
or ($key == "none" and (((.body // "") | contains("win=")) | not)))
] | sort_by(.created_at)
| map((.body | gsub("\r"; "") | split("\n")[0]))
| (map(test("Addressed the latest review feedback|no changes needed")) | rindex(true) // -1) as $lastok
| [ .[($lastok + 1):][] | select(contains("AutoFix ran out of time before finishing")) ] | length' "${WORKDIR}/ic.json" 2> /dev/null || true)"
if [[ "${PRIOR_TIMEOUTS}" -ge 1 ]]; then
echo
echo '## Budget warning: previous round(s) ran out of time'
echo
echo "${PRIOR_TIMEOUTS} round(s) since the last successful round exhausted the agent time budget before finishing anything. Do NOT attempt everything at once this round:"
echo '- Address the smallest set of blocking (Critical) findings first, and commit as soon as that subset is done.'
echo '- Prefer minimal, focused diffs; decline refactors and nice-to-haves with a one-line reason rather than implementing them.'
echo '- If the remaining feedback cannot fit the budget, defer it the normal way: leave those findings out of resolved-comments.txt and record each deferral in comment-replies.json so every open thread gets its reason — never let a deferral live only in the summary.'
fi
} > "${WORKDIR}/feedback.md"
echo '--- feedback.md ---'
cat "${WORKDIR}/feedback.md"
Expand Down Expand Up @@ -3222,6 +3268,7 @@ jobs:
"${WORKDIR}/failure.md" \
"${WORKDIR}/handoff.md" \
"${WORKDIR}/gate-output.log" \
"${WORKDIR}/gate-rejection.md" \
"${WORKDIR}/agent-api-error" \
"${WORKDIR}/agent-api-error-kind" \
"${WORKDIR}/agent-timeout"
Expand Down Expand Up @@ -3783,7 +3830,22 @@ jobs:
MARK_TS='9999-12-31T23:59:59Z'
HEADLINE="🤖 AutoFix updated a stale base — the fix did not pass verification, but this PR was behind \`${DEFAULT_BRANCH:-main}\`, so it merged current main in via update-branch and will retry on the next scan. A stale base (a dependency or symbol main already changed) can fail the build without being the fix's fault; if it still fails once current, it hands off to a human."
else
HEADLINE="🤖 Could not address the latest feedback automatically (round ${MARK_ROUND}/${MAX_ROUNDS}). A human should take over this PR."
# Say what actually happens next. The old "A human should
# take over this PR" read as a full release, but the loop
# is NOT done with the PR: this feedback's watermark
# advances (no automatic retry of THIS item), while
# management continues for new feedback and base conflicts
# — #7929 posted the old wording and then kept pushing
# rounds, which read as a contradiction.
# Name the gate ONLY when it actually ran: this branch is
# reached for every outcome=failed verdict, but reject_fix
# is the sole writer of gate-rejection.md — the failure.md /
# dirty-tree / unchanged-branch / missing-summary paths made
# no gate decision, so a blanket clause would repeat the very
# wording-doesn't-match-behaviour bug this PR fixes.
GATE_CLAUSE=''
[[ -s "${WORKDIR}/gate-rejection.md" ]] && GATE_CLAUSE=' — the verification gate rejected the attempt'
HEADLINE="🤖 Could not produce a passing fix for this feedback (round ${MARK_ROUND}/${MAX_ROUNDS})${GATE_CLAUSE}. This item now needs a human; the loop stays engaged and still picks up new feedback and base conflicts, but will not retry this item on its own."
fi
fi
elif [[ "${PREPARE_OUTCOME}" != 'success' && "${PREPARE_OUTCOME}" != 'failure' ]]; then
Expand Down Expand Up @@ -3884,6 +3946,36 @@ jobs:
MARK_ROUND="${MAX_ROUNDS}"
HEADLINE="🤖 AutoFix stopped after ${CONSEC_FAIL} consecutive rounds that failed to push anything (timeouts and/or gate rejections). Retrying at the same per-round budget is not converging — this usually means the PR is too large or conflicts with a fast-moving \`main\`. A human should rebase, split, or reduce it, then comment \`${RETRY_COMMAND}\` to re-arm. Until then future scans will skip this PR."
fi
# CUMULATIVE timeout breaker — the sibling of the consecutive
# one above, for the failure shape it cannot see: timeouts
# interleaved with pushed rounds. A push resets CONSEC_FAIL,
# but it does not make the next timeout cheaper — each burns a
# full agent budget with nothing to show (observed on #7929:
# three timeouts with successes in between; #7846 twice). The
# census reuses PRIOR_HEADS, so it is window-scoped exactly
# like the consecutive one and a re-arm clears it. Only
# overrides a would-be RETRY: a round already terminal keeps
# its own headline (the consecutive breaker included).
if [[ "${MARK_ROUND}" != "${MAX_ROUNDS}" ]]; then
# Needle matches the emitted headline verbatim — first lines
# can embed provider error text (API_ERROR_DETAIL puts up to
# 200 bytes of it on the same line), so a loose phrase could
# count a model error message as a timeout.
TIMEOUT_N="$(grep -c 'AutoFix ran out of time before finishing' <<< "${PRIOR_HEADS}" || true)"
if [[ -n "${AGENT_TIMEOUT:-}" ]]; then
TIMEOUT_N=$(( TIMEOUT_N + 1 ))
fi
if [[ "${TIMEOUT_N}" -ge "${TIMEOUT_WINDOW_CAP}" ]]; then
MARK_ROUND="${MAX_ROUNDS}"
# The headline states what the census MEASURED — the
# window's cumulative count — not "stopped after N
# timeouts": the round that trips this can itself have
# failed differently (a gate rejection landing on a window
# that already carries the cap — the exact rollout state
# of #7929/#7846).
HEADLINE="🤖 AutoFix stopped: this counting window now contains ${TIMEOUT_N} time-budget exhaustions (pushed rounds in between included; this round itself may have failed differently). That is ${TIMEOUT_N} full agent runs that pushed nothing. A human should split or reduce the PR (or raise the agent time budget), then comment \`${RETRY_COMMAND}\` to re-arm. Until then future scans will skip this PR."
fi
fi
fi
{
echo "${HEADLINE}"
Expand Down
Loading
Loading