-
Notifications
You must be signed in to change notification settings - Fork 1
ci: route lint-semgrep through install.sh + uv-managed pipx:semgrep (three-way-parity per Aaron 2026-04-27) #653
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
AceHack
merged 5 commits into
main
from
acehack/scorecard-pin-semgrep-docker-2026-04-27
Apr 27, 2026
Merged
Changes from all commits
Commits
Show all changes
5 commits
Select commit
Hold shift + click to select a range
2d18429
ci: run lint-semgrep via SHA-pinned semgrep/semgrep Docker image (res…
AceHack 9c222ce
ci/setup: pin elan + mise installers by content hash (resolves Scorec…
AceHack 4332c03
ci: switch semgrep to mise-managed pipx:semgrep (three-way-parity per…
AceHack d62fc6d
ci: drop redundant pipx pin, bump uv to 0.11.8; document uv-canonical…
AceHack 0a6e0f9
ci/setup: address review feedback on #653 (cleanup traps, armv7, role…
AceHack File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
71 changes: 71 additions & 0 deletions
71
docs/DECISIONS/2026-04-27-uv-canonical-python-tool-manager.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,71 @@ | ||
| # ADR: uv is the canonical Python tool manager for Zeta — pipx-named mise tools route through uv | ||
|
|
||
| **Date:** 2026-04-27 | ||
| **Status:** *Decision: `uv` (Astral) is the canonical Python tool installer / runtime / venv manager for the Zeta factory. Tools that the mise registry exposes via the `pipx:` backend (e.g. `pipx:semgrep`) are pinned in `.mise.toml` with the `pipx:` prefix because that is mise's registry naming; the actual installer mise invokes is `uv tool install` (mise auto-routes `pipx:` through uv when uv is in the toolchain — no separate `pipx` package is required). No `pip install` / `actions/setup-python` paths in CI; no parallel pipx-vs-uv camps in the install scripts.* | ||
| **Owner:** architect (synthesis); human maintainer (shaping-decision owner). | ||
| **Decision confidence:** *high* — choice is dictated by composing two prior commitments (three-way-parity per GOVERNANCE §24 + ../scratch first-class adoption of uv) plus a verified mise behaviour; no novel tradeoff to weigh. | ||
|
|
||
| ## Context | ||
|
|
||
| 1. **../scratch made uv first-class first.** `../scratch` is the maintainer's adjacent project where Python tooling decisions get burned in before this factory absorbs them. uv was made first-class there before any Zeta-side commitment, and the round-34 .mise.toml pull-in here cited *"`../scratch`"* explicitly as the source. The lineage matters: this ADR is documenting an *already-made* decision that was sitting implicit in `.mise.toml` comments rather than a fresh choice. | ||
| 2. **Three-way parity per GOVERNANCE §24.** Dev laptops + CI runners + devcontainers all bootstrap toolchain via `tools/setup/install.sh` → mise → declarative `.mise.toml`. Any second Python tool installer (pipx, bare pip, conda) creates a fork in the install graph and breaks the parity invariant. | ||
| 3. **mise routes pipx: through uv automatically.** Per upstream mise docs (`mise.jdx.dev/dev-tools/backends/pipx.html`): *"If you have uv installed, mise will use uv tool install under the hood and you don't need to install pipx to run the commands containing 'pipx:'."* Verified locally on 2026-04-27 with `uv = "0.11.8"` + `pipx:semgrep = "1.161.0"` (no separate pipx in the toolchain) — mise installs semgrep via `uv tool install` and `semgrep --version` reports 1.161.0. | ||
| 4. **Aaron 2026-04-27 explicit course-correction.** During the Scorecard PinnedDependenciesID fix work, Aaron pushed back on an initial draft that added `pipx = "1.11.1"` alongside `uv`: | ||
| > "we have uv do we need pipx, isn't there a uvx this should be much faster" | ||
|
|
||
| Plus the prior framing: | ||
| > "the fact that uv is our desired python setup should be documented somewehre this project ../scratch made it first class too" | ||
|
|
||
| This ADR is the documenting move. | ||
|
|
||
| ## Decision | ||
|
|
||
| 1. **`uv` is canonical.** Pinned in `.mise.toml` `[tools]` section. All Python-language CLI tooling routes through it. | ||
| 2. **`pipx:` prefix in mise registry calls is fine.** It is just mise's naming for the namespace; the actual installer is uv. Do NOT add `pipx = "X.Y.Z"` to `.mise.toml` to "satisfy" the prefix — that would install a redundant package that mise will not use. | ||
| 3. **No `actions/setup-python` in CI.** CI gets Python from the same `mise install` pass that dev laptops run. | ||
| 4. **No `pip install ...` lines in CI workflows.** Anything that wants a Python CLI tool gets it via `.mise.toml` pin (`pipx:foo = "X.Y.Z"`) or via `uv tool install` invoked from a script that ran through `install.sh`. | ||
|
|
||
| ## Consequences | ||
|
|
||
| **Positive:** | ||
|
|
||
| - **Dev/CI parity.** Same Python toolchain, same versions, same install path, same provenance. | ||
| - **Speed.** uv's resolver and installer are materially faster than pip+pipx. | ||
| - **GitHub artifact attestation verification** for `uv` itself (mise's aqua: backend handles it). One fewer tool in the toolchain (no pipx) means one fewer attestation surface. | ||
| - **Resolves Scorecard PinnedDependenciesID** for any future `pip install foo` patterns by removing the pattern entirely from CI. | ||
| - **Single-bump surface.** Bumping a Python CLI tool is a single `.mise.toml` edit. CI inherits. | ||
|
|
||
| **Negative:** | ||
|
|
||
| - **Naming surprise.** `pipx:semgrep` reads as if pipx is involved; the actual installer is uv. The `.mise.toml` comment block on `uv` and on each `pipx:*` entry must call this out (they do, post this ADR). Mitigation: this ADR is linked from those comments. | ||
| - **Tied to mise's auto-route behaviour.** If a future mise release changes the auto-route preference (e.g. requires `uvx:` prefix instead), all `pipx:*` entries need a rename. Mitigation: pinned mise version in `.mise.toml`; bump deliberately, with a smoke test. | ||
|
|
||
| **Neutral (deferred):** | ||
|
|
||
| - A potential future `uvx:` first-class backend in mise would let us drop the `pipx:` prefix entirely. Not blocking; cosmetic. | ||
|
|
||
| ## What this ADR does NOT decide | ||
|
|
||
| - Does NOT decide whether Zeta itself ships Python code (it does not; F# / C# / TS). | ||
| - Does NOT decide build-system choice for any future Python *project* (we do not have one). This ADR is about *tooling* — installing Python-language CLI tools (semgrep, ruff, etc.) — not about authoring Python. | ||
| - Does NOT decide between `uv tool install` and `uv venv` — both are valid for their respective use cases (CLI tools vs project venvs); they both flow through uv. | ||
|
|
||
| ## Lineage | ||
|
|
||
| - `../scratch` `.mise.toml` (Aaron's adjacent project) — uv made first-class here first | ||
| - Round-34 pull-in — Zeta `.mise.toml` cited `../scratch` as the source of the uv pin | ||
| - 2026-04-27 — Aaron's explicit ask to document the decision; this ADR | ||
|
|
||
| ## Composes with | ||
|
|
||
| - **GOVERNANCE.md §24** (three-way parity) | ||
| - **`docs/DECISIONS/2026-04-20-tools-scripting-language.md`** (no Python authored in Zeta tooling — this ADR is the orthogonal "but Python *tools* are fine, installed via uv" companion) | ||
| - **`memory/feedback_three_way_parity_install_scripts_dev_ci_devcontainer_minimize_github_specific_surface_aaron_2026_04_27.md`** (the substrate memory naming Aaron's host-portability invariant) | ||
| - **#653** — the PR landing the first concrete `pipx:semgrep` use, removing pip install + actions/setup-python from gate.yml lint-semgrep | ||
|
|
||
| ## Forward-action | ||
|
|
||
| - Land this ADR alongside the `.mise.toml` change in #653 | ||
| - Update `.mise.toml` comments on `uv` and on each `pipx:*` entry to point at this ADR | ||
| - Future `pipx:*` additions cite this ADR in their comment, not the verbose rationale | ||
| - If `actions/setup-python` is found anywhere in `.github/workflows/`, file a follow-up issue to migrate to `install.sh + .mise.toml` |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
77 changes: 77 additions & 0 deletions
77
...cripts_dev_ci_devcontainer_minimize_github_specific_surface_aaron_2026_04_27.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,77 @@ | ||
| --- | ||
| name: Three-way-parity invariant — dev/CI/devcontainer share install scripts; minimize GitHub-specific surface so switching CI hosts is cheap (Aaron 2026-04-27) | ||
| description: Aaron 2026-04-27 explicit course-correction during Scorecard PinnedDependencies fix. Otto's first attempt switched gate.yml lint-semgrep job to a SHA-pinned semgrep/semgrep Docker image. Aaron pushed back: this still uses GitHub-specific stuff (Docker container action) AND breaks the dev/CI parity invariant (dev laptops don't run semgrep via Docker). Right answer: install semgrep through the same `tools/setup/install.sh` that dev laptops + devcontainers use — i.e. add `pipx:semgrep` to `.mise.toml`. Composes GOVERNANCE §24 (already-codified three-way parity) + Otto's authority to push toolchain through install.sh + Aaron's host-portability invariant ("easy to switch hosts"). Triggered by Scorecard PinnedDependenciesID #17/#18 work where the convenience-fix (Docker container) would have introduced a real divergence. | ||
| type: feedback | ||
| --- | ||
|
|
||
| # Three-way-parity invariant — dev/CI/devcontainer share install scripts | ||
|
|
||
| ## Verbatim quote (Aaron 2026-04-27) | ||
|
|
||
| > "actions/setup-python we should be using our base python that our install scripts install we are trying to not use github stuff unless we have to so it's easy to switch hosts and our dev macchine and build machine setup is the same, that's one of the invariants we want to try to keep as close as possible dev machine / build machines are same/very similar for setup/share the setup/install scripts and post install scripts. this makes CI more deterministic too. if i'm off base here just let me know. but SHA pinning github actions is a supply change risk mitigation good idea." | ||
|
|
||
| ## The invariant (codified) | ||
|
|
||
| GOVERNANCE.md §24 already names this as the "three-way parity" rule: | ||
|
|
||
| - **Dev laptop** runs `tools/setup/install.sh` to bootstrap toolchain | ||
| - **CI runner** runs `tools/setup/install.sh` to bootstrap toolchain | ||
| - **Devcontainer image** runs `tools/setup/install.sh` to bootstrap toolchain | ||
|
|
||
| Same script. Same `.mise.toml`. Same versions. No GitHub-specific actions stand in for what install.sh already does (no `actions/setup-python`, no `actions/setup-node`, no `setup-dotnet`). | ||
|
|
||
| ## What this BUYS | ||
|
|
||
| - **Host portability.** If a future CI host swap happens (GitHub Actions → GitLab CI → self-hosted Forgejo runners → BuildKite → ...), the install path is the same script. Only the YAML wrapper differs. | ||
| - **Dev/CI parity for debugging.** A failure on CI reproduces locally because the toolchain is bit-identical (same mise pins, same install.sh path). | ||
| - **Determinism.** mise resolves declarative pins atomically; pip's dependency-resolver wandering or `actions/setup-python` cache poisoning don't enter the picture. | ||
| - **One bump surface.** Bumping a tool version is a single `.mise.toml` edit; CI inherits. | ||
|
|
||
| ## What was almost violated (today's failure mode) | ||
|
|
||
| Otto's first attempt to fix Scorecard PinnedDependenciesID #17/#18 (pip install in lint-semgrep) was to switch the job to a SHA-pinned `semgrep/semgrep:1.161.0@sha256:...` Docker image at the `container:` job level. | ||
|
|
||
| That fix: | ||
|
|
||
| - **Did** resolve the Scorecard alerts (image bytes are content-addressed) | ||
| - **Did NOT** install semgrep on dev laptops via the install path | ||
| - **Did** introduce a GitHub-Actions-specific shape (`container:` block) that isn't portable across CI hosts | ||
| - **Did** break the dev/CI parity invariant — a dev laptop running semgrep manually wouldn't be running the same version that CI runs | ||
|
|
||
| Aaron caught this immediately. The invariant is more important than the immediate fix. | ||
|
|
||
| ## The right fix (post-correction) | ||
|
|
||
| 1. Add `pipx = "1.11.1"` to `.mise.toml` (aqua-backed; same SHA-pinned release path as `actionlint` / `shellcheck` / `uv`) | ||
| 2. Add `"pipx:semgrep" = "1.161.0"` to `.mise.toml` (mise's pipx: backend installs the named PyPI package at the pinned version) | ||
| 3. Drop `actions/setup-python` + the two `pip install` steps from gate.yml | ||
| 4. Replace with `./tools/setup/install.sh` (same step build-and-test already uses) | ||
|
|
||
| Now: dev laptops + CI + devcontainers all install semgrep 1.161.0 the same way, through the same tooling, pinned in the same `.mise.toml`. | ||
|
|
||
| ## Operational rule | ||
|
|
||
| When fixing a CI-side problem, before reaching for a GitHub-specific solution (custom action, container: block, runner-side feature), check: | ||
|
|
||
| - **Does this tool exist in `.mise.toml` already?** If so, use it via install.sh. | ||
| - **Could this tool live in `.mise.toml`?** If yes (mise registry has it via aqua / pipx / npm / cargo / etc.), add it there and route CI through install.sh. | ||
| - **Is this genuinely GitHub-specific?** (e.g. workflow triggering, GITHUB_TOKEN scope, code-scanning upload) — only then is a GitHub-specific shape correct. | ||
|
|
||
| ## What this rule does NOT mean | ||
|
|
||
| - Does NOT mean "never use GitHub Actions" — `actions/checkout`, `actions/cache`, `github/codeql-action/*`, `actions/upload-artifact` are all genuinely GitHub-specific and stay | ||
| - Does NOT mean "never SHA-pin GitHub actions" — Aaron explicitly affirmed: *"SHA pinning github actions is a supply change risk mitigation good idea"*. SHA-pinning the actions we DO use is correct | ||
| - Does NOT mean "rewrite all of CI today" — apply the rule going forward; existing breaches are tech-debt to fix opportunistically when touching the surrounding code | ||
|
|
||
| ## Composes with | ||
|
|
||
| - **GOVERNANCE.md §24** — three-way parity rule (this memory is the lived rationale, not just a re-citation) | ||
| - **Otto-247 (version-currency)** — version pins in `.mise.toml` get the same WebSearch-first discipline | ||
| - **#71 (Otto owns settings)** — install-path decisions are within Otto's authority; the binding decision is "share via install.sh", not "let CI drift" | ||
| - **#652 (block-only-on-Aaron-must-do)** — Aaron's earlier framing today: drive forward with best long-term judgment. The course-correction today refines what "best long-term" looks like for CI design | ||
|
|
||
| ## Forward-action | ||
|
|
||
| - File this memory + MEMORY.md row | ||
| - Apply the rule to the in-flight fix: replace Docker-container approach with `pipx:semgrep` in `.mise.toml` + install.sh in gate.yml | ||
| - Future-self check: when adding a CI step, default-check `.mise.toml` first; reach for a GitHub-specific shape only when no parity-preserving option exists |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.