Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
a0e189d
feat(skills): add nemoclaw-maintainer-verify-stale
prekshivyas May 5, 2026
1ea1559
fix(verify-stale): tighten version detection and parsing
prekshivyas May 5, 2026
6996e29
fix(verify-stale): generalize version regex and validate against tags
prekshivyas May 5, 2026
4129edc
fix(verify-stale): anchor version regex to nemoclaw proximity
prekshivyas May 5, 2026
10268c4
fix(verify-stale): validate reproducer on baseline before trusting la…
prekshivyas May 5, 2026
ca865ad
fix(verify-stale): close gaps surfaced before E2E run
prekshivyas May 5, 2026
2c7e906
fix(verify-stale): close gaps 1, 4, 5, 6, 7 before E2E
prekshivyas May 6, 2026
ca50db5
feat(verify-stale): add preconditions, local-first short-circuit, rea…
prekshivyas May 6, 2026
6cf97dc
feat(verify-stale): detect behavior changed by design
prekshivyas May 6, 2026
9fdcb25
fix(verify-stale): polish html extraction, tighten gpu keywords, env-…
prekshivyas May 6, 2026
e2039a8
fix(verify-stale): correct path map, openclaw reset, and other v1 polish
prekshivyas May 6, 2026
e7be9f8
fix(verify-stale): tighten by-design detection with evidence and self…
prekshivyas May 6, 2026
32c61da
fix(verify-stale): broaden signal 2 candidate search to pickaxe
prekshivyas May 6, 2026
9552b59
fix(verify-stale): use wontfix label, add verification-mode + tag anc…
prekshivyas May 6, 2026
ea6be13
fix(verify-stale): require markdown-linked citations and link resolut…
prekshivyas May 6, 2026
1a20971
fix(verify-stale): align skill label vocabulary with actual repo labels
prekshivyas May 7, 2026
ef1d980
fix(verify-stale): default cap to 60 min and bootstrap reproducer dep…
prekshivyas May 7, 2026
d0fcf5e
fix(verify-stale): mandate reporter @-mention and 400 to 500-word tar…
prekshivyas May 7, 2026
296f7cd
fix(verify-stale): land six post-#2007 hardening fixes
prekshivyas May 7, 2026
24fae89
fix(verify-stale): make brev-login error directive, scrub stale insta…
prekshivyas May 7, 2026
8976176
fix(verify-stale): land five gaps from the post-#2007 audit
prekshivyas May 7, 2026
9e04701
fix(verify-stale): drop batch cap to 15 and actually enforce it
prekshivyas May 7, 2026
94d444b
fix(verify-stale): land six bug-shape gaps from the candidate-set audit
prekshivyas May 7, 2026
1d625dd
fix(verify-stale): land six gaps from the post-#2592 audit
prekshivyas May 7, 2026
c6e6508
fix(verify-stale): echo resolved nemoclaw version after install, fix …
prekshivyas May 7, 2026
13bc4c3
fix(verify-stale): skip issues with active maintainer discussion
prekshivyas May 7, 2026
22a3997
fix(verify-stale): file-based API key propagation, never via cmdline …
prekshivyas May 8, 2026
fe363cf
fix(verify-stale): elevate the comment-authoring principle into Step 10
prekshivyas May 8, 2026
b29958a
fix(verify-stale): tighten per-verdict length targets — 300 is a hard…
prekshivyas May 8, 2026
f0ec6b8
fix(verify-stale): two-tier maintainer-discussion handling, unanswere…
prekshivyas May 8, 2026
4ee3750
fix(verify-stale): close on-box subshell argv leak for API keys
prekshivyas May 8, 2026
dc5d714
Merge branch 'main' into feat/verify-stale-skill
prekshivyas May 8, 2026
3abffd7
fix(verify-stale): unbreak markdown link checker on placeholder examples
prekshivyas May 8, 2026
3bc1b8c
Merge branch 'main' into feat/verify-stale-skill
prekshivyas May 8, 2026
e48e573
fix(cut-release-tag): use xargs -r to skip empty input on label sweep
prekshivyas May 8, 2026
97407e1
Merge branch 'main' into feat/verify-stale-skill
prekshivyas May 8, 2026
b21f9a1
fix(verify-stale): consolidate post-review changes into one signed co…
prekshivyas May 11, 2026
a9def8c
Merge branch 'main' into feat/verify-stale-skill-v2
prekshivyas May 11, 2026
a78126b
fix(verify-stale): address three CodeRabbit findings on PR #3327
prekshivyas May 11, 2026
07bdcdb
feat(verify-stale): self-assign issue to maintainer on fixed-on-lates…
May 11, 2026
24b038b
Merge branch 'main' into feat/verify-stale-skill-v2
prekshivyas May 13, 2026
2ba881a
Merge branch 'main' into feat/verify-stale-skill-v2
cv May 17, 2026
c2ba511
Merge branch 'main' into feat/verify-stale-skill-v2
cv May 23, 2026
4962da5
refactor(skills): split verify-stale references
cv May 23, 2026
ef53a7e
Merge branch 'main' into feat/verify-stale-skill-v2
cv May 23, 2026
a6365be
Merge branch 'main' into feat/verify-stale-skill-v2
prekshivyas May 26, 2026
33a350c
fix(verify-stale): address 11 CodeRabbit findings on PR #3327
prekshivyas May 26, 2026
6ba04f4
fix(verify-stale): convert p50/p90 to ms to match SLA units
prekshivyas May 26, 2026
59aa03f
fix(verify-stale): v-prefix normalization + readable question-detecti…
prekshivyas May 26, 2026
64d7129
Merge branch 'main' into feat/verify-stale-skill-v2
prekshivyas May 26, 2026
222ca42
Merge branch 'main' into feat/verify-stale-skill-v2
prekshivyas May 26, 2026
bc2b1ab
Merge branch 'main' into feat/verify-stale-skill-v2
prekshivyas May 27, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
90 changes: 90 additions & 0 deletions .agents/skills/nemoclaw-maintainer-cut-release-tag/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -127,6 +127,96 @@ git ls-remote --tags origin | grep -E '(<new-version>|latest)'

Confirm both tags point to the same commit on the remote.

## Step 7: Conditionally Sweep Stale-Issue Verification Labels

Strip `fixed-on-latest` from open issues only when the verification has actually gone stale or a regression risk appeared since we verified — never blanket-sweep. A blanket sweep on every release re-verifies labels that were freshly applied yesterday, wasting Brev cost and creating noise. The skill's by-design path uses the existing repo `status: wont-fix` label, which is **not** swept (also applied for non-skill triage reasons, so clearing it would erase human work). `verify-inconclusive` is also kept on the same conditional cascade as `fixed-on-latest`.

**Decision cascade per labeled-and-open issue:**

| Order | Check | Action |
|---|---|---|
| 1 | Project [NVIDIA/199](https://github.com/orgs/NVIDIA/projects/199) status == **Done** | **Skip clear** — maintainer already accepted the verification; label can stay until the issue closes. |
| 2 | More than 14 days since the skill marker comment AND status != Done | **Clear** — verification is stale; reporter never confirmed in the review window. Re-verify on next skill run. |
| 3 | A PR merged since the marker date touches the paths the comment cited in `Relevant changes since v0.0.X` | **Clear** — regression risk; what was "fixed" may have been re-broken. |
| — | else | **Skip clear** — verification still holds; skill won't re-run on this issue (still excluded by Step 3 marker-TTL plus the live label). |

Closed issues are not iterated (the `--state open` filter on the listing excludes them implicitly).

Requires the `project` scope on the maintainer's gh CLI for the Project 199 status lookup. If missing, run `gh auth refresh -h github.com -s project` in a real terminal once (OAuth device-code flow). With the scope absent, the sweep falls back to the **time + regression** logic alone (skips check #1) and logs a warning.

```bash
PROJECT_NUMBER=199
TODAY_TS=$(date -u +%s)
HAVE_PROJECT_SCOPE=0
gh auth status 2>&1 | grep -q "'project'" && HAVE_PROJECT_SCOPE=1 || \
echo "[release-sweep] WARN gh missing 'project' scope — Done-state check disabled this run"

for label in fixed-on-latest verify-inconclusive; do
for n in $(gh issue list --repo NVIDIA/NemoClaw --state open --label "$label" --json number -q '.[].number'); do

# 1. Project Done-state check (only if we have project scope)
if [ "$HAVE_PROJECT_SCOPE" = "1" ]; then
STATUS=$(gh api graphql -F num="$n" -f query='
query($num: Int!) {
repository(owner: "NVIDIA", name: "NemoClaw") {
issue(number: $num) {
projectItems(first: 100) {
nodes {
project { number }
fieldValueByName(name: "Status") {
... on ProjectV2ItemFieldSingleSelectValue { name }
}
}
}
}
}
}' --jq '.data.repository.issue.projectItems.nodes[] | select(.project.number == 199) | .fieldValueByName.name' 2>/dev/null | head -1)
if [ "$STATUS" = "Done" ]; then
echo "[release-sweep] kept #$n ($label) — Project 199 status is Done"
continue
fi
fi

# 2. Find the most recent skill marker comment date
MARKER_DATE=$(gh issue view "$n" --repo NVIDIA/NemoClaw --json comments \
--jq '.comments | map(select(.body | test("nemoclaw-verify-stale v\\d+ \\d{4}-\\d{2}-\\d{2}"))) | last | .body | (capture("nemoclaw-verify-stale v\\d+ (?<d>\\d{4}-\\d{2}-\\d{2})") // {}) | .d // empty')
if [ -z "$MARKER_DATE" ]; then
# Label exists but no skill marker — applied manually; leave alone.
echo "[release-sweep] kept #$n ($label) — no skill marker, label applied manually"
continue
fi

AGE_DAYS=$(( (TODAY_TS - $(date -u -j -f "%Y-%m-%d" "$MARKER_DATE" +%s 2>/dev/null || date -u -d "$MARKER_DATE" +%s)) / 86400 ))
if [ "$AGE_DAYS" -ge 14 ]; then
gh issue edit "$n" --repo NVIDIA/NemoClaw --remove-label "$label"
echo "[release-sweep] cleared #$n ($label) — stale (verified ${AGE_DAYS}d ago, reporter not confirmed)"
continue
fi

# 3. Regression check — any PR-merge commit since MARKER_DATE touch the paths the
# comment's `Relevant changes since v0.0.X` block cited?
PATHS=$(gh issue view "$n" --repo NVIDIA/NemoClaw --json comments \
--jq '.comments | map(select(.body | test("nemoclaw-verify-stale v\\d+"))) | last | .body' \
| grep -oE '`[a-zA-Z0-9_/.-]+\.(ts|js|sh|py|yaml|yml|md)`' | tr -d '`' | sort -u)
if [ -n "$PATHS" ]; then
# Run from the current directory — Step 1's prerequisite already requires the maintainer
# to be inside the NemoClaw repo, and hardcoding ~/NemoClaw breaks anyone with a non-default
# checkout location.
REGRESSED=$(git log --since="$MARKER_DATE" origin/main --name-only --format=oneline -- $PATHS 2>/dev/null | head -1)
if [ -n "$REGRESSED" ]; then
gh issue edit "$n" --repo NVIDIA/NemoClaw --remove-label "$label"
echo "[release-sweep] cleared #$n ($label) — regression risk (commits since ${MARKER_DATE} touch implicated paths)"
continue
fi
fi

echo "[release-sweep] kept #$n ($label) — verified ${AGE_DAYS}d ago, no Done state, no regression touch"
done
done
```

The verification record itself stays in each issue's comment history — only the labels are reset, and only when the cascade above fires.

## Important Notes

- NEVER tag without explicit user confirmation of the version.
Expand Down
61 changes: 61 additions & 0 deletions .agents/skills/nemoclaw-maintainer-verify-stale/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
---
name: nemoclaw-maintainer-verify-stale
description: "Verifies whether stale NVIDIA/NemoClaw bug reports still reproduce on the latest tag. Use when maintainers ask to verify stale issues, reproduce old bugs on latest, drain the bug backlog, or apply fixed-on-latest, verify-inconclusive, or status: wont-fix. Runs candidate filtering, local/Brev reproduction, by-design detection, confidence scoring, redacted comments, and tag-only labeling; never auto-closes."
user_invocable: true
---

<!-- SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. -->
<!-- SPDX-License-Identifier: Apache-2.0 -->

# NemoClaw Maintainer — Verify Stale Issues

Automates the maintainer loop: choose an old bug, verify whether it still reproduces on the latest NemoClaw tag, then post an evidence-backed comment and label. It is tag-only: never close issues automatically.

## Progress checklist

Copy this checklist and update it as you work:

```text
Verify-stale progress:
- [ ] Select issue(s), latest tag, and reported version
- [ ] Apply skip/idempotency/active-discussion filters
- [ ] Classify environment, provider, and bug class
- [ ] Extract or synthesize a reproducer
- [ ] Verify preconditions and try local-first if eligible
- [ ] If Brev is needed, get plan approval before provisioning
- [ ] Validate the reproducer on baseline, then verify latest
- [ ] Check by-design/static-analysis branch when behavior was removed
- [ ] Score, redact, draft, and self-verify comment links
- [ ] Re-check issue state, post comment/label, update tracker when required
- [ ] Append activity log entry
```

## Workflow

1. **Select candidates and versions.** Read [reference/candidate-selection.md](reference/candidate-selection.md). Use it for single-issue mode, batch mode, latest-tag detection, filters, idempotency, active-discussion handling, and reported-version parsing.
2. **Classify and prepare.** Read [reference/environment-and-reproducer.md](reference/environment-and-reproducer.md). Use it for CPU/GPU/provider/bug-class classification, safe API-key handling, reproducer extraction, dependency checks, Brev auth, label checks, and local-first verification.
3. **Stop for approval before cost.** In batch mode, present one issue's plan and wait for maintainer approval before provisioning Brev.
4. **Provision and install.** If local-first does not settle the issue, read [reference/brev-provisioning.md](reference/brev-provisioning.md). Use it for Brev reuse/provisioning, reset, baseline/latest installs, dependency bootstrap, and `brev exec` footguns.
5. **Run the verification rubric.** Read [reference/reproduction-rubrics.md](reference/reproduction-rubrics.md). Use it to validate baseline behavior, retry with a synthesized reproducer if needed, run latest, handle architectural drift, and branch for performance or rebuild-cycle bugs.
6. **Check intentional changes.** If the symptom targets removed/deprecated behavior, read [reference/by-design.md](reference/by-design.md). Use static evidence to apply `status: wont-fix` only when the by-design branch self-verifies.
7. **Score, comment, label, and log.** Read [reference/scoring-comments-and-logging.md](reference/scoring-comments-and-logging.md). Use it for confidence scoring, redaction, concise templates, issue-state race checks, Project 199 movement, infra failures, and activity logging.

## Non-negotiables

- Never auto-close an issue. Apply labels and ask a maintainer/reporter to confirm.
- Never put API keys on a command line. Use the file-based pattern in `environment-and-reproducer.md`.
- Never post unredacted transcripts, issue excerpts, synthesized scripts, internal hostnames, email addresses, or tokens.
- Never post a comment with broken markdown links or tag-drifting `file:line` citations. Re-run cited commands and link-check at least one rendered link per comment section.
- Never use Brev for unsupported platforms or integration-token issues in v1.
- Keep comments concise: default to 200–300 words for fixed/by-design, 100–200 for inconclusive, and 30–80 for still-reproduces.

## Reference map

| Need | Read |
|---|---|
| Candidate query, filters, version parser | [reference/candidate-selection.md](reference/candidate-selection.md) |
| Environment classification, credentials, reproducer, preconditions, local-first | [reference/environment-and-reproducer.md](reference/environment-and-reproducer.md) |
| Brev box reuse/provision, reset, installs, dependency bootstrap | [reference/brev-provisioning.md](reference/brev-provisioning.md) |
| Baseline/latest matching, synth-repro, drift, performance, rebuild-cycle | [reference/reproduction-rubrics.md](reference/reproduction-rubrics.md) |
| Static by-design/wont-fix branch | [reference/by-design.md](reference/by-design.md) |
| Score, redact, comment, label, tracker move, infra handling, log | [reference/scoring-comments-and-logging.md](reference/scoring-comments-and-logging.md) |
Loading