Skip to content

Fix stats parse + add Summary of changes section #785

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
rdimitrov merged 2 commits into from
Apr 22, 2026
Merged

Fix stats parse + add Summary of changes section #785

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
235fe8a
Fix jq parse of claude-execution-output.json (array format)
rdimitrov Apr 22, 2026
ab7499c
Add Summary of changes section to PR body
rdimitrov Apr 22, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
112 changes: 103 additions & 9 deletions .github/workflows/upstream-release-docs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -652,6 +652,30 @@ jobs:
NO_CHANGES.md at repo root with a one-line explanation.
Still do not hand-edit any file.

BEFORE the final commit in Phase 6, write SUMMARY.md at
repo root with a concise bullet list of changes you made
on the docs side. This surfaces directly in the PR body
as a "Summary of changes" section so reviewers see what
actually shipped without clicking the diff.

Format each bullet as one of:
- `Added <what> at <path>` -- new pages/sections
- `Updated <what> in <path>` -- meaningful prose edits
- `Swept <what> across N files in <area>` -- repo-wide
renames, apiVersion bumps, etc.
- `Removed <what> from <path>` -- deletions

Keep the list to 3-8 bullets. Focus on what a reviewer
needs to know, not how you did it. If many files are
touched as one logical change (e.g. a v1alpha1 ->
v1beta1 apiVersion sweep), describe it as one bullet --
do not enumerate each file. Do NOT include bullets for
auto-generated reference files the refresh step
updated; only describe your hand-written edits.

Skip SUMMARY.md only when you wrote NO_CHANGES.md or
made zero hand-edits.

# Capture skill_gen's execution stats BEFORE skill_review runs
# and overwrites the shared execution-output JSON at the
# canonical claude-code-action location. Lets us report
Expand All @@ -663,10 +687,19 @@ jobs:
if: always() && steps.skill_gen.conclusion == 'success'
run: |
LOG="/home/runner/work/_temp/claude-execution-output.json"
# claude-code-action writes a top-level JSON array of
# messages (init, assistant turns, tool calls, result).
# The final entry is a "result" object with the session
# summary (num_turns, total_cost_usd, etc.). Use recursive
# descent with a last-match tail so we pick up the summary
# regardless of where in the structure it lives, and so
# the query still works if the upstream format shifts
# between array and object. `|| echo 0` guards against
# jq errors on missing/malformed files.
if [ -f "$LOG" ]; then
TURNS=$(jq -r '.num_turns // 0' "$LOG")
COST=$(jq -r '.total_cost_usd // 0' "$LOG")
DENIALS=$(jq -r '.permission_denials_count // 0' "$LOG")
TURNS=$(jq -r '[.. | .num_turns? // empty] | last // 0' "$LOG" 2>/dev/null || echo 0)
COST=$(jq -r '[.. | .total_cost_usd? // empty] | last // 0' "$LOG" 2>/dev/null || echo 0)
DENIALS=$(jq -r '[.. | .permission_denials_count? // empty] | last // 0' "$LOG" 2>/dev/null || echo 0)
else
TURNS=0
COST=0
Expand Down Expand Up @@ -739,9 +772,10 @@ jobs:
or docs/toolhive/reference/crds/ — those are
regenerated from release assets and aren't yours to edit.

Do NOT touch GAPS.md or NO_CHANGES.md at the repo root
if they exist -- they're signal files handed off to the
next workflow step, not part of the docs.
Do NOT touch GAPS.md, NO_CHANGES.md, or SUMMARY.md at
the repo root if they exist -- they're signal files
handed off to the next workflow step, not part of the
docs.

# claude-code-action works in a sibling scratch checkout (the
# .claude-pr/ dir we gitignore) and pushes commits to origin
Expand Down Expand Up @@ -777,10 +811,19 @@ jobs:
if: always() && steps.skill_review.conclusion == 'success'
run: |
LOG="/home/runner/work/_temp/claude-execution-output.json"
# claude-code-action writes a top-level JSON array of
# messages (init, assistant turns, tool calls, result).
# The final entry is a "result" object with the session
# summary (num_turns, total_cost_usd, etc.). Use recursive
# descent with a last-match tail so we pick up the summary
# regardless of where in the structure it lives, and so
# the query still works if the upstream format shifts
# between array and object. `|| echo 0` guards against
# jq errors on missing/malformed files.
if [ -f "$LOG" ]; then
TURNS=$(jq -r '.num_turns // 0' "$LOG")
COST=$(jq -r '.total_cost_usd // 0' "$LOG")
DENIALS=$(jq -r '.permission_denials_count // 0' "$LOG")
TURNS=$(jq -r '[.. | .num_turns? // empty] | last // 0' "$LOG" 2>/dev/null || echo 0)
COST=$(jq -r '[.. | .total_cost_usd? // empty] | last // 0' "$LOG" 2>/dev/null || echo 0)
DENIALS=$(jq -r '[.. | .permission_denials_count? // empty] | last // 0' "$LOG" 2>/dev/null || echo 0)
else
TURNS=0
COST=0
Expand Down Expand Up @@ -862,9 +905,12 @@ jobs:

- name: Capture skill signal files
id: signals
env:
BASELINE_SHA: ${{ steps.pre_skill.outputs.sha }}
run: |
GAPS_BODY=""
NO_CHANGES_BODY=""
SUMMARY_BODY=""
if [ -f GAPS.md ]; then
GAPS_BODY=$(cat GAPS.md)
rm GAPS.md
Expand All @@ -873,6 +919,26 @@ jobs:
NO_CHANGES_BODY=$(cat NO_CHANGES.md)
rm NO_CHANGES.md
fi
if [ -f SUMMARY.md ]; then
SUMMARY_BODY=$(cat SUMMARY.md)
rm SUMMARY.md
fi

# Fallback summary from skill-authored commit titles when
# the skill didn't write SUMMARY.md. Extracted from the
# range between the pre-skill baseline and HEAD so it
# captures the skill's commits only (refresh commit is
# before the baseline). Each line is pre-formatted as a
# markdown bullet so downstream just has to echo.
COMMIT_FALLBACK=""
if [ -z "$SUMMARY_BODY" ] && [ -n "$BASELINE_SHA" ]; then
# Raw "<sha> <subject>" lines; awk reformats each as a
# markdown bullet with the short SHA in backticks.
# Backticks via awk's \x60 to avoid shellcheck SC2016
# on a single-quoted git-log format string.
COMMIT_FALLBACK=$(git log --reverse --format='%h %s' "$BASELINE_SHA..HEAD" 2>/dev/null | \
awk '{ sha=$1; sub("^" sha " *", ""); printf "- %s (\x60%s\x60)\n", $0, sha }' || true)
fi

# Build full markdown fragments here so the PR body edit
# step can treat them as plain strings.
Expand All @@ -895,6 +961,21 @@ jobs:
echo "$GAPS_BODY"
fi
echo "GAPS_EOF"

# summary_block carries either the skill's hand-written
# SUMMARY.md body, or the commit-title fallback when the
# skill didn't write one. Empty when neither applies
# (e.g. silent run / NO_CHANGES case). The Augment step
# renders a "Summary of changes" section iff non-empty.
echo "summary_block<<SUMMARY_EOF"
if [ -n "$SUMMARY_BODY" ]; then
echo "$SUMMARY_BODY"
elif [ -n "$COMMIT_FALLBACK" ]; then
echo "_Auto-generated from commit titles -- skill did not write SUMMARY.md._"
echo ""
echo "$COMMIT_FALLBACK"
fi
echo "SUMMARY_EOF"
} >> "$GITHUB_OUTPUT"

- name: Apply pin_files substitutions
Expand Down Expand Up @@ -966,6 +1047,7 @@ jobs:
REPO: ${{ steps.detect.outputs.repo }}
NOTE_BLOCK: ${{ steps.signals.outputs.note_block }}
GAPS_BLOCK: ${{ steps.signals.outputs.gaps_block }}
SUMMARY_BLOCK: ${{ steps.signals.outputs.summary_block }}
AUTOGEN_TOUCHED: ${{ steps.autogen.outputs.touched }}
COMPARE_OK: ${{ steps.reviewers.outputs.compare_ok }}
MENTION_BLOCK: ${{ steps.reviewers.outputs.mention_block }}
Expand Down Expand Up @@ -1119,6 +1201,18 @@ jobs:
echo "| **Action required** | $ACTION_REQUIRED |"
echo ""

# ----- SUMMARY OF CHANGES -----
# Renders either the skill's hand-written SUMMARY.md body
# or the commit-title fallback from the signals step.
# Skipped entirely on silent / NO_CHANGES runs where the
# signals step leaves SUMMARY_BLOCK empty.
if [ -n "$SUMMARY_BLOCK" ]; then
echo "### Summary of changes"
echo ""
echo "$SUMMARY_BLOCK"
echo ""
fi

# ----- GAPS (when present) -----
if [ -n "$GAPS_BLOCK" ]; then
# Skill's GAPS.md starts at H2 ("## Gaps needing human
Expand Down