Skip to content

ci: route lint-semgrep through install.sh + uv-managed pipx:semgrep (three-way-parity per Aaron 2026-04-27)#653

Merged
AceHack merged 5 commits intomainfrom
acehack/scorecard-pin-semgrep-docker-2026-04-27
Apr 27, 2026
Merged

ci: route lint-semgrep through install.sh + uv-managed pipx:semgrep (three-way-parity per Aaron 2026-04-27)#653
AceHack merged 5 commits intomainfrom
acehack/scorecard-pin-semgrep-docker-2026-04-27

Conversation

@AceHack
Copy link
Copy Markdown
Member

@AceHack AceHack commented Apr 27, 2026

Summary

Drives toward Aaron's three-way-parity invariant (GOVERNANCE §24): dev laptops + CI runners + devcontainers all install semgrep through the same tools/setup/install.sh → mise → declarative .mise.toml. No actions/setup-python, no inline pip install, no GitHub-specific shapes for what install.sh already does.

Resolves Scorecard PinnedDependenciesID alerts #15, #16, #17, #18.

What changed

File Change
.mise.toml Bumped uv = "0.9""0.11.8" (current; verified via gh API today). Added "pipx:semgrep" = "1.161.0". No separate pipx package — mise auto-routes pipx: through uv tool install when uv is in the toolchain.
.github/workflows/gate.yml lint-semgrep job: dropped actions/setup-python + the two pip install lines. Replaced with ./tools/setup/install.sh (same step build-and-test uses) → run semgrep.
tools/setup/common/elan.sh curl master/elan-init.sh | sh → download at v4.2.1 commit SHA, verify SHA256 (4bacca95…9264), then exec.
tools/setup/linux.sh curl mise.run | sh → download mise-v2026.4.24-linux-{x64,arm64}.tar.gz directly from GitHub releases, verify per-arch SHA256, extract.
docs/DECISIONS/2026-04-27-uv-canonical-python-tool-manager.md New ADR: uv is the canonical Python tool manager; pipx:foo mise pins route through uv automatically; lineage from ../scratch.
memory/feedback_three_way_parity_install_scripts_dev_ci_devcontainer_minimize_github_specific_surface_aaron_2026_04_27.md New substrate memory: dev/CI/devcontainer share install scripts; minimize GitHub-specific surface.
MEMORY.md Pointer row to the new memory + ADR.

Aaron's framing

Preserve quality signals (2026-04-27, earlier today):

"code_quality severity=all given we want high quality signals for future AI training and we persision the PRs do you still agree?"

→ This PR is the fix, not a relax.

Three-way parity / minimize GitHub-specific surface (2026-04-27, mid-PR):

"actions/setup-python we should be using our base python that our install scripts install we are trying to not use github stuff unless we have to so it's easy to switch hosts and our dev macchine and build machine setup is the same, that's one of the invariants we want to try to keep as close as possible dev machine / build machines are same/very similar for setup/share the setup/install scripts and post install scripts. this makes CI more deterministic too."

→ Drove the revert from the initial Docker-container approach to install.sh-based.

uv first, no redundant pipx (2026-04-27, late-PR):

"we have uv do we need pipx, isn't there a uvx this should be much faster also +pipx = '1.11.1' is this latest, remember you mode cached latest can't be trusted you have to search the internet this goies for all version numbers you add, let's not start on an older version."

→ Removed redundant pipx = "1.11.1"; bumped uv to current 0.11.8 verified via gh API.

Document uv-as-canonical (2026-04-27):

"the fact that uv is our desired python setup should be documented somewehre this project ../scratch made it first class too"

→ ADR lands documenting it.

Why this is a quality improvement, not just rule-appeasement

  • Content-addressing at every install path (commit SHA + SHA256 for elan, release tarball + SHA256 for mise, mise-managed pin for semgrep, OCI digest could be re-introduced if we ever needed Docker).
  • Smaller attack surface. Removes actions/setup-python + two pip install steps from gate.yml; removes pip-bootstrap surface entirely.
  • Faster CI. uv resolver/installer is materially faster than pip+pipx.
  • Audit trail. Bumping is a single declarative .mise.toml edit; CI inherits.
  • Restored host portability — no GitHub-specific actions in lint-semgrep.
  • First-class dev-laptop parity. semgrep was previously not actually installed by install.sh (despite outdated comments saying so); now it really is.
  • Deterministic CI. mise resolves declarative pins atomically; no pip dep-resolver wandering.

Bumping procedure (documented inline)

  • elan.sh: bump ELAN_INIT_COMMIT (gh api /repos/leanprover/elan/releases/latest) and ELAN_INIT_SHA256 (curl + sha256sum) together.
  • linux.sh: bump MISE_VERSION and both MISE_SHA256_* values together; pull from /repos/jdx/mise/releases/latest.
  • .mise.toml pipx:semgrep: bump version pin; mise's pipx: backend (auto-routed to uv) handles the rest. See ADR docs/DECISIONS/2026-04-27-uv-canonical-python-tool-manager.md.

Test plan

🤖 Generated with Claude Code

…olves Scorecard PinnedDependenciesID #17 #18)

Replaces `pip install --upgrade pip + pip install semgrep` with the
official semgrep/semgrep:1.161.0 image, pinned by multi-arch manifest
digest sha256:326e5f41cc972bb423b764a14febbb62bbad29ee1c01820805d077dd868fea48.

Why this is a quality improvement, not just rule-appeasement:
- Image bytes are content-addressed by digest. Tag mutation cannot
  affect us; the earlier `pip install semgrep` was version-floating
  and would silently pick up any new semgrep release between CI runs.
- Removes the entire pip-bootstrap surface (Setup Python +
  pip --upgrade + pip install semgrep). Smaller attack surface, faster
  CI, fewer pinning surfaces to maintain.
- Multi-arch index digest covers both x86_64 and arm64 runners.

Per Aaron 2026-04-27: "given we want high quality signals for future
AI training and we persision the PRs do you still agree?" — fix, do
not relax. This commit is the fix.

Resolves Scorecard alerts #17, #18 (PinnedDependenciesID, pipCommand).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings April 27, 2026 21:00
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates the semgrep CI job to run semgrep from a SHA-pinned official Docker image instead of installing it via pip, addressing supply-chain pinning concerns raised by Scorecard.

Changes:

  • Switch lint (semgrep) job to a job-level container: using semgrep/semgrep pinned by digest.
  • Remove Python setup and pip install semgrep steps from the workflow.

Comment thread .github/workflows/gate.yml Outdated
…ard PinnedDependenciesID #15 #16)

elan.sh — switched from `curl ... master/elan-init.sh | sh` (latest
master, unpinned) to:
  1. Download elan-init.sh at v4.2.1 commit SHA
     (58e8d545e33641f66dbcbd22c4283109e71757be)
  2. Verify SHA256 (4bacca9502cb89736fe63d2685abc2947cfbf34dc87673504f1bb4c43eda9264)
  3. Execute the verified copy

linux.sh — switched from `curl mise.run | sh` (auto-detects latest at
runtime) to:
  1. Download the pinned tarball mise-v2026.4.24-linux-{x64,arm64}.tar.gz
     directly from github.com/jdx/mise/releases
  2. Verify per-arch SHA256 (x64: de2f924…2c58, arm64: cf5f4899…5727)
  3. Extract mise/bin/mise to ~/.local/bin and source PATH

Why this is a quality improvement, not just rule-appeasement:
- The previous shapes silently absorbed any new elan/mise release
  between CI runs. A compromised upstream master branch (elan) or a
  redirector swap (mise.run) would have shipped to every dev laptop
  + every CI run with no signal. Content-hash pinning makes such an
  event a hard fail with a verification message.
- Bumping is a deliberate two-line change (commit/tarball + hash)
  with a documented procedure in each script's comment block — easier
  to audit than `master`/`mise.run`.
- Portable SHA256 verification (sha256sum/shasum fallback) per
  Otto-235 4-shell target.

Per Aaron 2026-04-27: "preserve quality signals" — fix, don't relax.

Resolves Scorecard alerts #15 (elan downloadThenRun) and #16 (mise
downloadThenRun).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@AceHack AceHack changed the title ci: SHA-pin semgrep via official Docker image (resolves Scorecard #17 #18) ci: SHA-pin semgrep + elan + mise by content hash (resolves Scorecard #15 #16 #17 #18) Apr 27, 2026
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9c222ce4ee

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread tools/setup/linux.sh
AceHack and others added 2 commits April 27, 2026 17:11
… Aaron 2026-04-27)

Reverts the Docker-container approach in favour of the GOVERNANCE §24
three-way-parity invariant: dev laptops + CI runners + devcontainers
all install semgrep through the same `tools/setup/install.sh` via
mise. CI was the odd one out (had its own actions/setup-python + pip
install), which would have made switching CI hosts costly and broke
host-portability.

Aaron 2026-04-27:

> "actions/setup-python we should be using our base python that our
> install scripts install we are trying to not use github stuff
> unless we have to so it's easy to switch hosts and our dev machine
> and build machine setup is the same, that's one of the invariants
> we want to try to keep as close as possible dev machine / build
> machines are same/very similar for setup/share the setup/install
> scripts and post install scripts. this makes CI more deterministic
> too."

What changed:
- `.mise.toml`: added `pipx = "1.11.1"` (aqua-backed; same SHA-pinned
  release path as actionlint / shellcheck / uv) and
  `"pipx:semgrep" = "1.161.0"`. Now any machine that runs install.sh
  gets the same semgrep version pinned the same way.
- `.github/workflows/gate.yml` lint-semgrep: dropped the
  `container: semgrep/semgrep@sha256:...` block from the previous
  iteration; dropped `actions/setup-python` and the two `pip install`
  steps. Job is now: checkout → install.sh → semgrep. Same shape as
  build-and-test.

This still resolves Scorecard PinnedDependenciesID #17 + #18 (the
unpinned pip-bootstrap surface goes away entirely) and additionally:
- Restores host-portability — no GitHub-specific actions in this job
- First-class dev-laptop parity (semgrep was previously not actually
  installed by install.sh; now it is)
- More deterministic CI (mise resolves declarative pins; no
  pip dependency-resolver wandering)

Verified locally: `mise install` + `mise exec -- semgrep --version`
produces 1.161.0 on the same shape.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
… decision (Aaron 2026-04-27)

Removes `pipx = "1.11.1"` from `.mise.toml` — mise's `pipx:` backend
auto-routes through `uv tool install` when uv is available, so adding
pipx as a separate package was redundant. Bumps `uv` from 0.9 to
0.11.8 (latest, verified via gh API today 2026-04-27 — current
training data is stale per Otto-247 version-currency rule).

Adds:
- `docs/DECISIONS/2026-04-27-uv-canonical-python-tool-manager.md` —
  ADR documenting uv as the canonical Python tool manager for Zeta,
  the lineage from `../scratch` (where uv was made first-class
  first), the mise pipx:→uv auto-route behaviour, and the rules for
  using `pipx:foo` pins without an actual pipx package.
- `memory/feedback_three_way_parity_install_scripts_dev_ci_devcontainer_minimize_github_specific_surface_aaron_2026_04_27.md` —
  substrate memory naming Aaron's host-portability invariant; what
  three-way parity buys; what was almost violated by the
  Docker-container draft; the right-fix decision flow.
- MEMORY.md row pointing at the new memory + ADR.

Aaron 2026-04-27 verbatim:
- "we have uv do we need pipx, isn't there a uvx this should be much
  faster also +pipx = '1.11.1' is this latest, remember you mode
  cached latest can't be trusted you have to search the internet
  this goies for all version numbers you add, let's not start on an
  older version."
- "the fact that uv is our desired python setup should be documented
  somewehre this project ../scratch made it first class too"

Verified locally: `mise install` with `uv = "0.11.8"` +
`pipx:semgrep = "1.161.0"` (no separate pipx) installs semgrep
1.161.0 via uv tool install.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings April 27, 2026 21:16
@AceHack AceHack changed the title ci: SHA-pin semgrep + elan + mise by content hash (resolves Scorecard #15 #16 #17 #18) ci: route lint-semgrep through install.sh + uv-managed pipx:semgrep (three-way-parity per Aaron 2026-04-27) Apr 27, 2026
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 5 comments.

Comment thread tools/setup/linux.sh Outdated
Comment thread tools/setup/common/elan.sh Outdated
Comment thread .github/workflows/gate.yml Outdated
Comment thread .mise.toml Outdated
Comment thread .github/workflows/gate.yml
…-refs, stale comment)

7 review threads:

1. Copilot P1 — name attribution in gate.yml + .mise.toml comments
   ("Per Aaron 2026-04-27") violates the role-ref-on-current-state
   discipline (history surfaces only). Replaced with role-refs:
   "the host-portability invariant" / "per the three-way-parity
   invariant (GOVERNANCE §24)".

2. Codex P2 — preserve armv7 support that `curl mise.run | sh`
   used to give us implicitly. Added the armv7 case + SHA256
   to the per-arch dispatch in linux.sh.

3. Copilot P1 — temp dir leak on failure in linux.sh: `mktemp -d`
   only cleaned up on the success path. Added `trap 'rm -rf
   "${MISE_TMP}"' EXIT` so the dir is removed even on download /
   SHA / extract failure.

4. Copilot P1 — same pattern in elan.sh tmp file. Added EXIT
   trap.

5. Copilot P1 — gate.yml install-toolchain step comment claimed
   "mise installs python + pipx + semgrep" but pipx was dropped
   in commit d62fc6d (mise auto-routes pipx: through uv).
   Updated the comment to reflect current state and to point at
   the uv-canonical ADR.

6. Copilot P1 — same name-attribution issue in .mise.toml
   semgrep block. Replaced.

7. Copilot P1 — PR description was already updated; thread
   should be re-evaluated by reviewer with the current
   description in view.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@AceHack AceHack merged commit d3e5c68 into main Apr 27, 2026
22 checks passed
@AceHack AceHack deleted the acehack/scorecard-pin-semgrep-docker-2026-04-27 branch April 27, 2026 21:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants