Skip to content
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 16 additions & 1 deletion .github/workflows/qwen-autofix.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2493,7 +2493,22 @@ jobs:
MARK_TS="${NEWEST:-${WATERMARK:-9999-12-31T23:59:59Z}}"
if [[ -n "${NEWEST:-}" ]]; then
MARK_ROUND="$(( ROUND + 1 ))"
HEADLINE="🤖 Could not address the latest feedback automatically (round ${MARK_ROUND}/${MAX_ROUNDS}). A human should take over this PR."
if [[ -z "${DETAIL_FILE}" ]]; then
# Prepare ran (NEWEST is set) but the agent produced NO output
# at all — it crashed before writing any verdict (e.g. a staged
# runner that fails to boot). It evaluated NOTHING, so the
# watermark must NOT advance past this feedback: an advance makes
# the next scan see "nothing new" and never retry, stranding the
# PR on a transient crash (a deploy, an infra blip, a base-image
# bug fixed minutes later). Stamp the sentinel ts (excluded from
# EVAL_WM) so the feedback stays live and the next scan retries;
# the incremented round still bounds retries to MAX_ROUNDS before
# a terminal handoff, so a PERSISTENT crash cannot loop forever.
MARK_TS='9999-12-31T23:59:59Z'
HEADLINE="🤖 AutoFix crashed before it could evaluate the feedback (attempt ${MARK_ROUND}/${MAX_ROUNDS}) — it will retry on the next scan. If it keeps failing to start through attempt ${MAX_ROUNDS}, a human should take over. Run log: ${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}/actions/runs/${GITHUB_RUN_ID}"
Comment thread
wenshao marked this conversation as resolved.
Resolved
Comment thread
wenshao marked this conversation as resolved.
Resolved
Comment thread
wenshao marked this conversation as resolved.
Resolved
else
HEADLINE="🤖 Could not address the latest feedback automatically (round ${MARK_ROUND}/${MAX_ROUNDS}). A human should take over this PR."
fi
else
# Crashed/timed out before reading the feedback: mark terminal so the
# scan skips (it can't advance the watermark without a read), but say
Expand Down
60 changes: 47 additions & 13 deletions scripts/tests/qwen-autofix-workflow.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -2572,20 +2572,54 @@ describe('qwen-autofix workflow', () => {
"STALE: '${{ steps.prepare.outputs.stale }}'",
);

// Terminal-round transition: feedback read (NEWEST set) → normal increment;
// feedback never read (empty) → MAX_ROUNDS so the scan skips instead of
// re-handing-off forever.
const markRound = reviewAddressReportStep.match(
/(if \[\[ -n "\$\{NEWEST:-\}" \]\]; then\n[\s\S]*?\n\s*fi)/,
// Handoff marker semantics across the three crash/handoff shapes. The block
// sets BOTH MARK_TS (watermark) and MARK_ROUND (retry budget); replay the
// real bash so a regression in either is caught, not string-matched. The
// `\n {12}fi` anchor matches the OUTER fi (12 spaces), skipping the nested
// DETAIL_FILE `fi` (14 spaces).
const markBlock = reviewAddressReportStep.match(
/(MARK_TS="\$\{NEWEST[\s\S]*?\n {12}fi)\n/,
)?.[1];
expect(markRound).toBeTruthy();
const runMarkRound = (env) =>
execFileSync('bash', ['-c', `${markRound}\nprintf '%s' "$MARK_ROUND"`], {
env: { ...process.env, MAX_ROUNDS: '5', ROUND: '2', ...env },
encoding: 'utf8',
});
expect(runMarkRound({ NEWEST: '2026-07-16T00:00:00Z' })).toBe('3');
expect(runMarkRound({ NEWEST: '' })).toBe('5');
expect(markBlock).toBeTruthy();
const runMark = (env) =>
execFileSync(
'bash',
['-c', `${markBlock}\nprintf '%s|%s' "$MARK_TS" "$MARK_ROUND"`],
{
env: {
...process.env,
MAX_ROUNDS: '5',
ROUND: '2',
WATERMARK: '',
DETAIL_FILE: '',
NEWEST: '',
...env,
},
encoding: 'utf8',
},
);
const SENTINEL = '9999-12-31T23:59:59Z';
// 1. Agent produced output but verify failed: advance the watermark to the
// evaluated feedback; round increments — a real evaluated handoff.
expect(
runMark({
NEWEST: '2026-07-16T00:00:00Z',
DETAIL_FILE: '/tmp/failure.md',
}),
).toBe('2026-07-16T00:00:00Z|3');
// 2. Crash BEFORE any verdict (no output) though prepare ran: the watermark
// must NOT advance (sentinel ts, excluded from EVAL_WM) so the next scan
// RETRIES the same feedback; round still increments to bound the retries.
// This is the #7219-class fix — a transient crash no longer strands a PR.
expect(runMark({ NEWEST: '2026-07-16T00:00:00Z', DETAIL_FILE: '' })).toBe(
`${SENTINEL}|3`,
);
// 3. Crash before prepare (NEWEST empty): terminal round so the scan skips
// instead of re-handing-off forever; ts falls back to WATERMARK/sentinel.
expect(runMark({ NEWEST: '', WATERMARK: '2026-07-10T00:00:00Z' })).toBe(
'2026-07-10T00:00:00Z|5',
);
expect(runMark({ NEWEST: '', WATERMARK: '' })).toBe(`${SENTINEL}|5`);

// Behaviorally replay the pending-staleness jq filter against sample checks so
// a flipped comparison (which would age out live checks → double-processing)
Expand Down
Loading