Skip to content

feat(ci): semantic package-lock.json diff as an upserted PR comment - #65206

Merged
ethernet8023 merged 1 commit into
mainfrom
ethie/package-lock-compare
Jul 16, 2026
Merged

feat(ci): semantic package-lock.json diff as an upserted PR comment#65206
ethernet8023 merged 1 commit into
mainfrom
ethie/package-lock-compare

Conversation

@ethernet8023

@ethernet8023 ethernet8023 commented Jul 15, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

What does this PR do?

Adds an advisory CI job that posts (and updates in place) a PR comment showing the semantic diff of package-lock.json changes — which packages were added, removed, or version-bumped.

The raw textual diff of a lockfile is noise: npm reorders entries, rewrites integrity hashes, and moves packages between nesting levels, so a one-line package.json bump can produce a thousand-line diff nobody reads. Instead of diffing text, scripts/ci/lockfile_diff.py parses the packages map out of both versions of every tracked lockfile (via git show <ref>:<path>, merge base → HEAD), reduces each to {install path: version}, and set-diffs the maps. Reordering and hash churn produce a literally empty diff; what's left is the actual dependency change. Keying on the full node_modules/... path means the same package deduped at two versions shows as two entries (rendered as react *(nested under foo)*).

The comment is upserted: the body starts with a hidden <!-- hermes-lockfile-diff --> marker, and the workflow finds its previous comment by that prefix (paginated) and PATCHes it, so a PR gets exactly one comment tracking the latest push. If a later push reverts all lockfile changes, the comment is updated to say so rather than left stale or deleted.

Example comment body:

⚠️ package-lock.json changes (2 packages)

Package Before After
apps/bootstrap-installer 0.0.1 99.99.99
➖ apps/desktop 0.17.0 —

see #65316

Why in-house instead of a marketplace action: the existing lockfile-diff actions are unpinned third-party JS that would run with pull-requests: write — exactly what our supply-chain policy exists to avoid. The differ is ~200 lines of stdlib Python with tests.

The job is advisory: it never fails on findings, and fork PRs (read-only GITHUB_TOKEN) degrade to a ::warning::. It's in the all-checks-pass gate only so a crash of the script itself is caught (skipped counts as success).

closes #65316
^ that's just a demo

Related Issue

N/A — direct request.

Type of Change

  • ✨ New feature (non-breaking change that adds functionality)

Changes Made

  • scripts/ci/lockfile_diff.py — new semantic differ: parse_lockfile() / diff_locks() / render_markdown() pure functions + git show-based ref comparison across all tracked package-lock.json files (lockfileVersion 2/3)
  • .github/workflows/lockfile-diff.yml — new workflow_call job: diffs merge base → HEAD, writes the table to the step summary, upserts the PR comment via gh api (create or PATCH by marker)
  • .github/workflows/ci.yml — wires the lockfile-diff job through the orchestrator, gated on PR events + the new npm_lock lane; added to the all-checks-pass gate
  • .github/actions/detect-changes/action.yml — exposes the new npm_lock output
  • scripts/ci/classify_changes.py — new npm_lock lane (any **/package-lock.json; fails open on .github/ changes per the existing contract)
  • tests/ci/test_lockfile_diff.py — new: reorder/hash-churn ⇒ empty diff, add/remove/update detection, nested-dedup tracking, root/link-entry skipping, markdown marker + rendering
  • tests/ci/test_classify_changes.py — npm_lock lane cases (root + nested lockfile, fail-open default)

How to Test

  1. scripts/run_tests.sh tests/ci/ — 28 tests pass (differ + classifier)
  2. Local e2e against real refs: python3 scripts/ci/lockfile_diff.py --base HEAD --head HEAD --output /tmp/d.md ⇒ empty report; committing a simulated version bump + package removal in the root lockfile and diffing HEAD~1..HEAD produces the two-row table above
  3. On a PR touching any package-lock.json: the package-lock.json diff job posts the comment; push again and the same comment updates in place

Checklist

Code

  • I've read the Contributing Guide
  • My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
  • I searched for existing PRs to make sure this isn't a duplicate
  • My PR contains only changes related to this fix/feature (no unrelated commits)
  • I've run pytest tests/ -q and all tests pass (via scripts/run_tests.sh tests/ci/)
  • I've added tests for my changes (required for bug fixes, strongly encouraged for features)
  • I've tested on my platform: NixOS (Linux)

Documentation & Housekeeping

  • I've updated relevant documentation (README, docs/, docstrings) — module docstrings in the new script; no user-facing docs affected
  • I've updated cli-config.yaml.example if I added/changed config keys — N/A
  • I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — N/A (follows the existing orchestrator/lane pattern)
  • I've considered cross-platform impact (Windows, macOS) per the compatibility guide — CI-only, runs on ubuntu-latest; script uses encoding="utf-8" explicitly
  • I've updated tool descriptions/schemas if I changed tool behavior — N/A

Screenshots / Logs

=== Summary: 2 files, 28 tests passed, 0 failed (100% complete) in 0.6s (32 workers) ===

Simulated bump e2e output:

2 package version change(s) — report written to /tmp/ld-diff.md
<!-- hermes-lockfile-diff -->
## ⚠️ `package-lock.json` changes (2 packages)
...
| apps/bootstrap-installer | `0.0.1` | `99.99.99` |
| ➖ apps/desktop | `0.17.0` | — |

@ethernet8023
ethernet8023 force-pushed the ethie/package-lock-compare branch from fcb3a76 to c709d13 Compare July 15, 2026 21:03
@github-actions

github-actions Bot commented Jul 15, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

✅ CI-sensitive file review passed

The ci-reviewed label is present on this PR.

@ethernet8023 ethernet8023 added the ci-reviewed applied to manually approve dangerous changes label Jul 15, 2026
@alt-glitch alt-glitch added type/feature New feature or request P3 Low — cosmetic, nice to have sweeper:risk-automation Sweeper risk: may affect CI, automerge, label sync, or maintainer automation labels Jul 15, 2026
@ethernet8023
feat(ci): semantic package-lock.json diff as an upserted PR comment
08270cf 
git diff on a lockfile is unreadable: npm reorders entries, rewrites
integrity hashes, and moves packages between nesting levels, so a
one-line package.json bump produces a thousand-line textual diff.

scripts/ci/lockfile_diff.py instead parses the `packages` map out of
both versions of every tracked package-lock.json (via `git show`),
reduces each to {install path: version}, and set-diffs the maps —
reorder/hash churn vanishes, leaving only actual version movement
(added / removed / updated, with nested dedup copies tracked
separately).

The lockfile-diff workflow posts the result as a Markdown table in a
PR comment gated behind a hidden marker: subsequent pushes PATCH the
existing comment instead of stacking new ones, and a push that reverts
all lockfile changes updates the comment to say so. Advisory only —
never fails on findings; fork PRs (read-only token) degrade to a
warning.

Wired through the ci.yml orchestrator with a new npm_lock lane in
classify_changes.py (fails open on .github/ changes per the existing
contract).
@ethernet8023
ethernet8023 force-pushed the ethie/package-lock-compare branch from c709d13 to 08270cf Compare July 16, 2026 03:13
@ethernet8023 ethernet8023 mentioned this pull request Jul 16, 2026
Closed
@ethernet8023
ethernet8023 enabled auto-merge (squash) July 16, 2026 03:17
@ethernet8023
ethernet8023 merged commit f8ddf4f into main Jul 16, 2026
39 checks passed
@ethernet8023
ethernet8023 deleted the ethie/package-lock-compare branch July 16, 2026 03:18
Gravezzz pushed a commit to Gravezzz/hermes-agent that referenced this pull request Jul 21, 2026
@ethernet8023
feat(ci): semantic package-lock.json diff as an upserted PR comment (N…
dd9c710 
…ousResearch#65206)

git diff on a lockfile is unreadable: npm reorders entries, rewrites
integrity hashes, and moves packages between nesting levels, so a
one-line package.json bump produces a thousand-line textual diff.

scripts/ci/lockfile_diff.py instead parses the `packages` map out of
both versions of every tracked package-lock.json (via `git show`),
reduces each to {install path: version}, and set-diffs the maps —
reorder/hash churn vanishes, leaving only actual version movement
(added / removed / updated, with nested dedup copies tracked
separately).

The lockfile-diff workflow posts the result as a Markdown table in a
PR comment gated behind a hidden marker: subsequent pushes PATCH the
existing comment instead of stacking new ones, and a push that reverts
all lockfile changes updates the comment to say so. Advisory only —
never fails on findings; fork PRs (read-only token) degrade to a
warning.

Wired through the ci.yml orchestrator with a new npm_lock lane in
classify_changes.py (fails open on .github/ changes per the existing
contract).
randlee pushed a commit to randlee/hermes-agent that referenced this pull request Aug 11, 2026
@ethernet8023
feat(ci): semantic package-lock.json diff as an upserted PR comment (N…
056e71e 
…ousResearch#65206)

git diff on a lockfile is unreadable: npm reorders entries, rewrites
integrity hashes, and moves packages between nesting levels, so a
one-line package.json bump produces a thousand-line textual diff.

scripts/ci/lockfile_diff.py instead parses the `packages` map out of
both versions of every tracked package-lock.json (via `git show`),
reduces each to {install path: version}, and set-diffs the maps —
reorder/hash churn vanishes, leaving only actual version movement
(added / removed / updated, with nested dedup copies tracked
separately).

The lockfile-diff workflow posts the result as a Markdown table in a
PR comment gated behind a hidden marker: subsequent pushes PATCH the
existing comment instead of stacking new ones, and a push that reverts
all lockfile changes updates the comment to say so. Advisory only —
never fails on findings; fork PRs (read-only token) degrade to a
warning.

Wired through the ci.yml orchestrator with a new npm_lock lane in
classify_changes.py (fails open on .github/ changes per the existing
contract).
33hodl pushed a commit to 33hodl/hermes-agent that referenced this pull request Aug 12, 2026
@ethernet8023
feat(ci): semantic package-lock.json diff as an upserted PR comment (N…
7e333ac 
…ousResearch#65206)

git diff on a lockfile is unreadable: npm reorders entries, rewrites
integrity hashes, and moves packages between nesting levels, so a
one-line package.json bump produces a thousand-line textual diff.

scripts/ci/lockfile_diff.py instead parses the `packages` map out of
both versions of every tracked package-lock.json (via `git show`),
reduces each to {install path: version}, and set-diffs the maps —
reorder/hash churn vanishes, leaving only actual version movement
(added / removed / updated, with nested dedup copies tracked
separately).

The lockfile-diff workflow posts the result as a Markdown table in a
PR comment gated behind a hidden marker: subsequent pushes PATCH the
existing comment instead of stacking new ones, and a push that reverts
all lockfile changes updates the comment to say so. Advisory only —
never fails on findings; fork PRs (read-only token) degrade to a
warning.

Wired through the ci.yml orchestrator with a new npm_lock lane in
classify_changes.py (fails open on .github/ changes per the existing
contract).
prmartinow pushed a commit to prmartinow/hermes-agent that referenced this pull request Aug 26, 2026
@ethernet8023
feat(ci): semantic package-lock.json diff as an upserted PR comment (N…
968e941 
…ousResearch#65206)

git diff on a lockfile is unreadable: npm reorders entries, rewrites
integrity hashes, and moves packages between nesting levels, so a
one-line package.json bump produces a thousand-line textual diff.

scripts/ci/lockfile_diff.py instead parses the `packages` map out of
both versions of every tracked package-lock.json (via `git show`),
reduces each to {install path: version}, and set-diffs the maps —
reorder/hash churn vanishes, leaving only actual version movement
(added / removed / updated, with nested dedup copies tracked
separately).

The lockfile-diff workflow posts the result as a Markdown table in a
PR comment gated behind a hidden marker: subsequent pushes PATCH the
existing comment instead of stacking new ones, and a push that reverts
all lockfile changes updates the comment to say so. Advisory only —
never fails on findings; fork PRs (read-only token) degrade to a
warning.

Wired through the ci.yml orchestrator with a new npm_lock lane in
classify_changes.py (fails open on .github/ changes per the existing
contract).
melon-xf added a commit to melon-xf/hermes-agent that referenced this pull request Sep 3, 2026
@melon-xf
feat(ci): semantic package-lock.json diff as an upserted PR comment (N…
c14f0df 
…ousResearch#65206)

git diff on a lockfile is unreadable: npm reorders entries, rewrites
integrity hashes, and moves packages between nesting levels, so a
one-line package.json bump produces a thousand-line textual diff.

scripts/ci/lockfile_diff.py instead parses the `packages` map out of
both versions of every tracked package-lock.json (via `git show`),
reduces each to {install path: version}, and set-diffs the maps —
reorder/hash churn vanishes, leaving only actual version movement
(added / removed / updated, with nested dedup copies tracked
separately).

The lockfile-diff workflow posts the result as a Markdown table in a
PR comment gated behind a hidden marker: subsequent pushes PATCH the
existing comment instead of stacking new ones, and a push that reverts
all lockfile changes updates the comment to say so. Advisory only —
never fails on findings; fork PRs (read-only token) degrade to a
warning.

Wired through the ci.yml orchestrator with a new npm_lock lane in
classify_changes.py (fails open on .github/ changes per the existing
contract).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

ci-reviewed applied to manually approve dangerous changes P3 Low — cosmetic, nice to have sweeper:risk-automation Sweeper risk: may affect CI, automerge, label sync, or maintainer automation type/feature New feature or request

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@ethernet8023 @alt-glitch