CNTRLPLANE-3944: Regenerate requirements.txt files with pinned --hash entries - #9135
Conversation
Regenerate hypershift-ci-python/requirements.txt via `uv pip compile --generate-hashes` so every dependency carries --hash=sha256 entries for its published artifacts. This enables pip's hash-checking mode (pip install --require-hashes), protecting the CI Python image build against a compromised or tampered package matching a version pin but not the originally published artifact. Part of CNTRLPLANE-3944.
Normalize the top-level pins from PEP 440 arbitrary-equality (===) to standard version matching (==), then regenerate the full dependency lock via `uv pip compile --generate-hashes`. Pip's hash-checking mode requires every installed package -- including transitive dependencies of mkdocs-material, mkdocs-mermaid2-plugin, and mkdocs-glightbox -- to be pinned and hashed, so the file now lists the complete resolved closure instead of just the three top-level packages. Verified with `pip install --require-hashes` and a real `mkdocs build --strict` run against the regenerated lock. Part of CNTRLPLANE-3944.
Replace the open-ended boto3/botocore version ranges with exact pins resolved via `uv pip compile --generate-hashes`, and add sha256 hashes for every package in the resolved closure (boto3, botocore, jmespath, python-dateutil, s3transfer, six, urllib3). Pip's hash-checking mode requires exact pins, so this also satisfies that precondition for the OIDC/Dex demo playbook setup described in contrib/oidc/README.md. Part of CNTRLPLANE-3944.
Add a new hypershift-ci-python/README.md and extend docs/README.md and contrib/oidc/README.md with instructions for regenerating each requirements.txt via `uv pip compile --generate-hashes` when adding a new dependency or bumping an existing one, plus how to validate the result with `pip install --require-hashes`. Part of CNTRLPLANE-3944.
|
Pipeline controller notification For optional jobs, comment This repository is configured in: LGTM mode |
|
@mgencur: This pull request references CNTRLPLANE-3944 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the task to target the "5.0.0" version, but no target version was set. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
Skipping CI for Draft Pull Request. |
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughThe change converts OIDC, documentation, and HyperShift CI Python requirements into generated exact-version dependency sets with SHA-256 hashes and transitive dependency provenance. README files document regeneration with Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 11✅ Passed checks (11 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (1)
contrib/oidc/requirements.txt (1)
1-35: 🔒 Security & Privacy | 🔵 TrivialVerify SBOM, provenance, signing, and license checks in the consuming pipeline.
The lockfiles improve artifact integrity, but the supplied changes do not demonstrate SBOM generation, provenance attestations, Sigstore/cosign signing, or license compatibility checks for the expanded dependency closures.
contrib/oidc/requirements.txt#L1-L35: verify these controls cover the OIDC dependency set.docs/requirements.txt#L1-L451: verify these controls cover the expanded documentation dependency closure.hypershift-ci-python/requirements.txt#L1-L237: verify these controls cover the HyperShift CI dependency set.As per path instructions, “New deps: justify need, check license compatibility,” “SBOM: ensure build produces provenance attestations,” and “Signing: artifacts signed with Sigstore/cosign.”
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@contrib/oidc/requirements.txt` around lines 1 - 35, Verify the consuming pipeline applies SBOM generation, provenance attestations, Sigstore/cosign signing, and license compatibility checks to every dependency closure: contrib/oidc/requirements.txt lines 1-35, docs/requirements.txt lines 1-451, and hypershift-ci-python/requirements.txt lines 1-237. Update the relevant build or release workflow so all three dependency sets are covered, and justify or validate the licenses of newly introduced dependencies.Source: Path instructions
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@hypershift-ci-python/README.md`:
- Around line 52-53: Update the README’s pip-compile fallback command to include
the --generate-hashes option, ensuring regenerated requirements remain
hash-pinned while preserving the existing uv pip compile guidance.
In `@hypershift-ci-python/requirements.txt`:
- Around line 146-152: Update the pinned packages in requirements.txt to
httplib2 0.32.0, protobuf 5.29.6, and pyasn1 0.6.4. Regenerate each package’s
sha256 hashes and rerun the hash-checked dependency install to validate the
lockfile.
---
Nitpick comments:
In `@contrib/oidc/requirements.txt`:
- Around line 1-35: Verify the consuming pipeline applies SBOM generation,
provenance attestations, Sigstore/cosign signing, and license compatibility
checks to every dependency closure: contrib/oidc/requirements.txt lines 1-35,
docs/requirements.txt lines 1-451, and hypershift-ci-python/requirements.txt
lines 1-237. Update the relevant build or release workflow so all three
dependency sets are covered, and justify or validate the licenses of newly
introduced dependencies.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository YAML (base), Central YAML (inherited)
Review profile: CHILL
Plan: Enterprise
Run ID: 612e181d-5a2c-4464-af00-29f690706090
📒 Files selected for processing (6)
contrib/oidc/README.mdcontrib/oidc/requirements.txtdocs/README.mddocs/requirements.txthypershift-ci-python/README.mdhypershift-ci-python/requirements.txt
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #9135 +/- ##
==========================================
+ Coverage 44.56% 44.62% +0.05%
==========================================
Files 774 775 +1
Lines 97003 97105 +102
==========================================
+ Hits 43228 43329 +101
+ Misses 50783 50781 -2
- Partials 2992 2995 +3 see 8 files with indirect coverage changes
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
Bump httplib2 (0.22.0 -> 0.32.0) and pyasn1 (0.6.3 -> 0.6.4) in hypershift-ci-python/requirements.txt. While regenerating, `uv pip compile --generate-hashes` was found to reuse a package's previous hash under its new version's pin when the compile output path is the same file as its input -- confirmed against the real published PyPI artifacts, since neither --refresh nor --no-cache prevented it. Fix the requirements.txt hashes for the two bumped packages, and correct all three README instructions to compile to a temporary path and move it into place afterward, instead of compiling directly onto the existing file. Part of CNTRLPLANE-3944.
|
Pushed an additional commit: bumped While doing this, found that |
|
The Docs build depends on #9133 . This PR will need a rebase after the merge. |
Regenerate via the temp-file-and-move workflow (compile to a fresh path, then move it into place) to avoid the stale-hash issue found when compiling directly onto the existing file. Hashes verified against the real published PyPI artifacts for protobuf 5.29.6. Part of CNTRLPLANE-3944.
|
/hold |
Add --require-hashes to the pip install steps in docs/Dockerfile and hypershift-ci-python/Dockerfile so the images actually enforce the sha256 verification the requirements.txt files now provide, rather than just carrying unused hash metadata. Also fix docs/image-version.awk, which derives the mkdocs-material image tag from docs/requirements.txt. It relied on the old mkdocs-material===X.Y.Z (arbitrary-equality) format and a bare substring match on "mkdocs-material", which broke once that line became mkdocs-material==X.Y.Z inside a much larger hashed dependency list containing other matching lines (mkdocs-material-extensions, "# via mkdocs-material" comments). Anchor the match to the start of the line and split on "==" instead. Verified both Dockerfiles build successfully with --require-hashes, and that `mkdocs build --strict` runs correctly against the resulting docs image. Also verified `make image` now correctly derives the tag quay.io/hypershift/mkdocs-material:9.6.8 again. Part of CNTRLPLANE-3944.
|
/unhold |
celebdor
left a comment
There was a problem hiding this comment.
It is missing the "--require-hashes" in the .github/workflows/docs-build-reusable.yaml
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/docs-build.yaml:
- Line 11: Update the reusable workflow reference in the docs-build workflow to
use the PR-local .github/workflows/docs-build-reusable.yaml path instead of the
openshift/hypershift repository reference pinned to main, ensuring the workflow
tests the version changed in the current PR.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository YAML (base), Central YAML (inherited)
Review profile: CHILL
Plan: Enterprise
Run ID: 91645b0f-1457-4376-b28f-eadf0cc437cd
📒 Files selected for processing (2)
.github/workflows/docs-build-reusable.yaml.github/workflows/docs-build.yaml
actions/setup-python removed the pip-install input in v7.0.0 (it installed into the global environment, conflicted with virtualenvs and tools like uv, and wasn't picked up by Dependabot). Since this workflow was already pinned to v7.0.0, its pip-install: '-r docs/requirements.txt' input silently became a no-op, meaning docs dependencies were never actually installed here. Replace it with an explicit "pip install -r docs/requirements.txt" step, matching the fix already proposed in openshift#9133.
Add --require-hashes to the docs dependency install step, matching docs/Dockerfile, so CI actually enforces the sha256 verification that docs/requirements.txt now provides instead of just carrying unused hash metadata. Part of CNTRLPLANE-3944.
18850b0 to
acf25af
Compare
|
/test images |
|
/approve |
|
/label acknowledge-critical-fixes-only |
|
/verify bypass Looks like this PR would be verified after merge with main via @mgencur. |
Env vars are step-scoped; PIP_CACHE_DIR was attached to the wrong step and had no effect on pip's cache location.
|
/lgtm |
|
Scheduling tests matching the |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: celebdor, jparrill, mgencur The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
/jira backport release-4.22 release-4.21 release-4.20 |
|
@PoornimaSingour: The following backport issues have been created: Queuing cherrypicks to the requested branches to be created after this PR merges: DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
@openshift-ci-robot: #9135 failed to apply on top of branch "release-4.22": DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
|
I commented on the wrong PR. Please ignore my previous comment. |
What this PR does / why we need it:
Regenerates all three Python
requirements.txtfiles in the repo so every dependency is pinned to an exact version and carries--hash=sha256:...entries for its published artifacts, enabling pip's hash-checking mode (pip install --require-hashes). This protects against a compromised or tampered package matching a version pin but not the originally published artifact — a practice increasingly adopted across OpenShift repositories.hypershift-ci-python/requirements.txt— was already==pinned; hashes added viauv pip compile --generate-hashes.docs/requirements.txt— normalized from PEP 440 arbitrary-equality (===) to standard==, then regenerated to the full transitive closure with hashes. Verified with a realmkdocs build --strictrun.contrib/oidc/requirements.txt— open-endedboto3>=/botocore>=ranges resolved to exact pins and hashed in one step.Also adds/updates READMEs (new
hypershift-ci-python/README.md, updates todocs/README.mdandcontrib/oidc/README.md) documenting how to add a new dependency or update an existing one's hash going forward, usinguv pip compile --generate-hashes.Which issue(s) this PR fixes:
Fixes CNTRLPLANE-3944
Special notes for your reviewer:
--require-hashesmode requires every installed package — including transitive dependencies — to be pinned and hashed,docs/requirements.txtandcontrib/oidc/requirements.txtgrew substantially (they previously listed only top-level packages). This is expected/standard behavior ofpip-compile/uv pip compile --generate-hasheslock files, not scope creep.pip install --dry-run --require-hashes -r <file>, anddocs/requirements.txtwas additionally verified with a realmkdocs build --strict.requirements.txtfiles, but this hasn't been observed against this repo's setup yet.Checklist:
🤖 Generated with Claude Code via
/jira:solve https://redhat.atlassian.net/browse/CNTRLPLANE-3944 mgencurSummary by CodeRabbit
requirements.txt.pip --require-hashes.mkdocs-material.