forked from opendatahub-io/notebooks
-
Notifications
You must be signed in to change notification settings - Fork 29
chore(rhoai-3.3): cherry-pick developer tooling QoL from main #2055
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
jiridanek
wants to merge
7
commits into
rhoai-3.3
Choose a base branch
from
cherry-pick-qol-rhoai-3.3
base: rhoai-3.3
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
7 commits
Select commit
Hold shift + click to select a range
99cfa91
fix(CVE): centralize CVE constraints in dependencies/cve-constraints.…
jiridanek f1922b3
Moved keras from override-dependencies to cve-constraints.txt
mtchoum1 2670fbd
ISSUE #3032: chore(uv): pin uv version to fix CI check-generated-code…
jiridanek e5e0be1
ISSUE #3036: chore(scripts/): rewrite `scripts/pylocks_generator.sh` …
jiridanek 6e33d01
NO-JIRA: chore: fix Makefile `test` target to use correct `./uv` scri…
jiridanek 7124110
docs: update pylocks_generator.sh references to .py
jiridanek d9b3697
fix: pre-commit fixes and CI workflow cleanup
jiridanek File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,99 +1,240 @@ | ||
| --- | ||
| # This GitHub action is meant to update the pipfile.locks | ||
| name: Pipfile.locks Renewal Action | ||
| # This GitHub action is meant to update the lock files (pylock.toml) | ||
| name: Lock Files Renewal Action | ||
|
|
||
| on: # yamllint disable-line rule:truthy | ||
| # Triggers the workflow every Wednesday at 1am UTC | ||
| schedule: | ||
| - cron: "0 1 * * 3" | ||
| - cron: "0 1 * * 3" # Weekly lockfile update | ||
| - cron: "0 9,15 * * 1-5" # Auto-merge check at 9am and 3pm UTC on weekdays | ||
| workflow_dispatch: # for manual trigger workflow from GH Web UI | ||
| inputs: | ||
| operation: | ||
| description: 'Which operation to run' | ||
| required: true | ||
| default: 'update-lockfiles' | ||
| type: choice | ||
| options: | ||
| - 'update-lockfiles' | ||
| - 'auto-merge' | ||
| branch: | ||
| description: 'Specify branch' | ||
| description: 'Specify branch (for update-lockfiles)' | ||
| required: false | ||
| default: 'main' | ||
| python_version: | ||
| description: 'Select a Python version to update Pipfile.lock' | ||
| index_mode: | ||
| description: 'Index mode for lock file generation (for update-lockfiles)' | ||
| required: false | ||
| default: '["3.11", "3.12"]' | ||
| default: 'auto' | ||
| type: choice | ||
| options: | ||
| - '["3.11", "3.12"]' | ||
| - '["3.12"]' | ||
| - '["3.11"]' | ||
| - '["3.9"]' | ||
| - '["3.8"]' | ||
| lockfiles_upgrade: | ||
| description: 'Force full relock and upgrades for pylock.toml (manual runs)' | ||
| - 'auto' | ||
| - 'public-index' | ||
| - 'rh-index' | ||
| force_upgrade: | ||
| description: 'Force upgrade all packages to latest versions (for update-lockfiles)' | ||
| required: false | ||
| default: 'false' | ||
| type: choice | ||
| options: | ||
| - 'true' | ||
| - 'false' | ||
| update_optional_dirs: | ||
| description: 'Include optional directories in update' | ||
| required: false | ||
| default: 'false' | ||
| type: choice | ||
| options: | ||
| - 'true' | ||
| - 'false' | ||
|
|
||
| jobs: | ||
| refresh-pipfile-locks: | ||
| refresh-lock-files: | ||
| # Only run on Wednesday schedule or manual dispatch with 'update-lockfiles' operation | ||
| if: (github.event_name == 'workflow_dispatch' && github.event.inputs.operation == 'update-lockfiles') || (github.event_name == 'schedule' && github.event.schedule == '0 1 * * 3') | ||
| runs-on: ubuntu-latest | ||
| concurrency: | ||
| group: refresh-pipfile-locks-${{ matrix.python-version }}-${{ github.ref }} | ||
| group: refresh-lock-files-${{ github.ref }} | ||
| cancel-in-progress: false | ||
| strategy: | ||
| fail-fast: false | ||
| matrix: | ||
| python-version: >- | ||
| ${{ fromJSON( github.event.inputs.python_version || '["3.11", "3.12"]' ) }} | ||
| permissions: | ||
| contents: write | ||
| pull-requests: write | ||
| env: | ||
| BRANCH: ${{ github.event.inputs.branch || 'main' }} | ||
| INCLUDE_OPT_DIRS: ${{ github.event.inputs.update_optional_dirs || 'false' }} | ||
| # Force full relock on scheduled runs. For manual runs, use the input toggle. | ||
| FORCE_LOCKFILES_UPGRADE: ${{ github.event_name == 'schedule' && '1' || (github.event.inputs.lockfiles_upgrade == 'true' && '1' || '0') }} | ||
| INDEX_MODE: ${{ github.event.inputs.index_mode || 'auto' }} | ||
| # Force upgrade on scheduled runs, or when explicitly requested | ||
| FORCE_LOCKFILES_UPGRADE: ${{ github.event_name == 'schedule' && '1' || (github.event.inputs.force_upgrade == 'true' && '1' || '0') }} | ||
|
|
||
| steps: | ||
| - name: Checkout code | ||
| uses: actions/checkout@v6 | ||
| with: | ||
| ref: ${{ env.BRANCH }} | ||
| token: ${{ secrets.GH_ACCESS_TOKEN }} | ||
| persist-credentials: true | ||
|
|
||
| - name: Configure Git | ||
| run: | | ||
| git config --global user.email "github-actions[bot]@users.noreply.github.com" | ||
| git config --global user.name "GitHub Actions" | ||
|
|
||
| - name: Set up Python ${{ matrix.python-version }} | ||
| - name: Set up Python | ||
| uses: actions/setup-python@v6 | ||
| with: | ||
| python-version: ${{ matrix.python-version }} | ||
|
|
||
| - name: Install pipenv | ||
| run: pip install "pipenv==2025.0.4" | ||
| python-version: '3.12' | ||
|
|
||
| - name: Install uv | ||
| run: pip install "uv==0.9.6" | ||
| uses: astral-sh/setup-uv@v7 | ||
| with: | ||
| version-file: uv.toml | ||
|
|
||
| - name: Run make refresh-pipfilelock-files | ||
| - name: Run make refresh-lock-files | ||
| run: | | ||
| make refresh-pipfilelock-files PYTHON_VERSION=${{ matrix.python-version }} INCLUDE_OPT_DIRS=${{ env.INCLUDE_OPT_DIRS }} | ||
| make refresh-lock-files INDEX_MODE=${{ env.INDEX_MODE }} | ||
| env: | ||
| FORCE_LOCKFILES_UPGRADE: ${{ env.FORCE_LOCKFILES_UPGRADE }} | ||
|
|
||
| - name: Commit changes (if any) | ||
| - name: Create Pull Request | ||
| env: | ||
| GH_TOKEN: ${{ secrets.GH_ACCESS_TOKEN }} | ||
| run: | | ||
| git add . | ||
| git diff --cached --quiet && echo "No changes to commit." || git commit -m "Update Pipfile.lock for Python ${{ matrix.python-version }}" | ||
| if git diff --cached --quiet; then | ||
| echo "No changes to commit." | ||
| exit 0 | ||
| fi | ||
|
|
||
| - name: Pull and push changes | ||
| BRANCH_NAME="lockfile-update-$(date +%Y%m%d-%H%M)" | ||
| git checkout -b "$BRANCH_NAME" | ||
| git commit -m "Update lock files" | ||
| git push -u origin "$BRANCH_NAME" | ||
|
|
||
| gh pr create \ | ||
| --title "Update lock files" \ | ||
| --body "$(cat <<'EOF' | ||
| Automated lock file update. | ||
|
|
||
| **Auto-merge policy:** This PR will be automatically merged after 1 working day unless: | ||
| - Moved to draft status | ||
| - Labeled with `do-not-merge/*` | ||
| - Manually merged or closed | ||
| EOF | ||
| )" \ | ||
| --label "automated-lockfile-update" \ | ||
| --base "${{ env.BRANCH }}" | ||
|
|
||
| auto-merge-lockfile-prs: | ||
| # Run on auto-merge schedule or manual dispatch with 'auto-merge' operation | ||
| if: (github.event_name == 'workflow_dispatch' && github.event.inputs.operation == 'auto-merge') || (github.event_name == 'schedule' && github.event.schedule == '0 9,15 * * 1-5') | ||
| runs-on: ubuntu-latest | ||
| permissions: | ||
| contents: write | ||
| pull-requests: write | ||
| steps: | ||
| - name: Auto-merge eligible lockfile PRs | ||
| env: | ||
| GH_TOKEN: ${{ secrets.GH_ACCESS_TOKEN }} | ||
| GITHUB_TOKEN_FOR_APPROVAL: ${{ github.token }} | ||
| run: | | ||
| git pull --rebase origin ${{ env.BRANCH }} | ||
| git push origin ${{ env.BRANCH }} | ||
| set -euo pipefail | ||
|
|
||
| REPO="${{ github.repository }}" | ||
|
|
||
| echo "Searching for PRs with label 'automated-lockfile-update'..." | ||
|
|
||
| # Get all open PRs with the automated-lockfile-update label | ||
| PRS=$(gh pr list --repo "$REPO" --label "automated-lockfile-update" --state open --json number,title,createdAt,isDraft,labels --limit 50) | ||
|
|
||
| if [ "$PRS" = "[]" ] || [ -z "$PRS" ]; then | ||
| echo "No open PRs found with label 'automated-lockfile-update'" | ||
| exit 0 | ||
| fi | ||
|
|
||
| echo "Found PRs: $PRS" | ||
|
|
||
| # Process each PR | ||
| echo "$PRS" | jq -c '.[]' | while read -r pr; do | ||
| PR_NUM=$(echo "$pr" | jq -r '.number') | ||
| PR_TITLE=$(echo "$pr" | jq -r '.title') | ||
| CREATED_AT=$(echo "$pr" | jq -r '.createdAt') | ||
| IS_DRAFT=$(echo "$pr" | jq -r '.isDraft') | ||
| LABELS=$(echo "$pr" | jq -r '.labels[].name' 2>/dev/null || echo "") | ||
|
|
||
| echo "" | ||
| echo "=== Processing PR #$PR_NUM: $PR_TITLE ===" | ||
| echo "Created at: $CREATED_AT" | ||
| echo "Is draft: $IS_DRAFT" | ||
| echo "Labels: $LABELS" | ||
|
|
||
| # Skip drafts | ||
| if [ "$IS_DRAFT" = "true" ]; then | ||
| echo "SKIP: PR #$PR_NUM is a draft" | ||
| continue | ||
| fi | ||
|
|
||
| # Skip if has do-not-merge/* label | ||
| if echo "$LABELS" | grep -q "^do-not-merge/"; then | ||
| echo "SKIP: PR #$PR_NUM has a do-not-merge/* label" | ||
| continue | ||
|
coderabbitai[bot] marked this conversation as resolved.
|
||
| fi | ||
|
|
||
| # Check if PR is at least 1 working day old | ||
| # Working day = Monday-Friday, so: | ||
| # - If created Mon-Thu, eligible next day | ||
| # - If created Fri, eligible Mon | ||
| # - If created Sat, eligible Mon | ||
| # - If created Sun, eligible Tue | ||
|
|
||
| CREATED_TS=$(date -d "$CREATED_AT" +%s 2>/dev/null || date -j -f "%Y-%m-%dT%H:%M:%SZ" "$CREATED_AT" +%s 2>/dev/null || date -j -f "%Y-%m-%dT%H:%M:%S" "${CREATED_AT%Z}" +%s) | ||
| NOW_TS=$(date +%s) | ||
| CREATED_DOW=$(date -d "$CREATED_AT" +%u 2>/dev/null || date -j -f "%Y-%m-%dT%H:%M:%SZ" "$CREATED_AT" +%u 2>/dev/null || date -j -f "%Y-%m-%dT%H:%M:%S" "${CREATED_AT%Z}" +%u) | ||
|
|
||
| if [ -z "$CREATED_TS" ] || [ -z "$CREATED_DOW" ]; then | ||
| echo "WARNING: Failed to parse date '$CREATED_AT' for PR #$PR_NUM. Skipping." | ||
| continue | ||
| fi | ||
|
|
||
| # Calculate minimum age in seconds for 1 working day | ||
| # Base: 24 hours = 86400 seconds | ||
| # If created on Friday (5), add weekend: 72 hours = 259200 seconds | ||
| # If created on Saturday (6), need to wait till Monday + 1 day: 48 + 24 = 72 hours | ||
| # If created on Sunday (7), need to wait till Tuesday: 24 + 24 = 48 hours... actually Mon+1day = Tue | ||
|
|
||
| case "$CREATED_DOW" in | ||
| 5) MIN_AGE_SECONDS=$((72 * 3600)) ;; # Friday -> Monday (3 days) | ||
| 6) MIN_AGE_SECONDS=$((48 * 3600)) ;; # Saturday -> Monday (2 days) | ||
| 7) MIN_AGE_SECONDS=$((48 * 3600)) ;; # Sunday -> Tuesday (2 days, Mon is working day 0) | ||
| *) MIN_AGE_SECONDS=$((24 * 3600)) ;; # Mon-Thu -> next day (1 day) | ||
| esac | ||
|
|
||
| AGE_SECONDS=$((NOW_TS - CREATED_TS)) | ||
| AGE_HOURS=$((AGE_SECONDS / 3600)) | ||
|
|
||
| echo "PR age: ${AGE_HOURS} hours (minimum required: $((MIN_AGE_SECONDS / 3600)) hours)" | ||
|
|
||
| if [ "$AGE_SECONDS" -lt "$MIN_AGE_SECONDS" ]; then | ||
| echo "SKIP: PR #$PR_NUM is not old enough (created $AGE_HOURS hours ago, need $((MIN_AGE_SECONDS / 3600)) hours)" | ||
| continue | ||
| fi | ||
|
|
||
| echo "Checking review status for PR #$PR_NUM..." | ||
|
|
||
| # Check if PR has an approving review | ||
| REVIEWS=$(gh pr view "$PR_NUM" --repo "$REPO" --json reviews --jq '.reviews[] | select(.state == "APPROVED")' 2>/dev/null || echo "") | ||
|
|
||
| if [ -z "$REVIEWS" ]; then | ||
| echo "No approving review found. Adding approval using github-actions bot..." | ||
|
|
||
| # Get PR author to ensure we don't approve our own PR | ||
| PR_AUTHOR=$(gh pr view "$PR_NUM" --repo "$REPO" --json author --jq '.author.login') | ||
| echo "PR author: $PR_AUTHOR" | ||
|
|
||
| # Add approving review using GITHUB_TOKEN (github-actions bot) | ||
| # This is different from GH_ACCESS_TOKEN which created the PR | ||
| GH_TOKEN="$GITHUB_TOKEN_FOR_APPROVAL" gh pr review "$PR_NUM" --repo "$REPO" --approve --body "Auto-approved by lockfile renewal workflow after 1 working day waiting period." || { | ||
| echo "WARNING: Failed to add approval for PR #$PR_NUM. May need manual approval." | ||
| continue | ||
| } | ||
| echo "Approval added successfully." | ||
| else | ||
| echo "PR #$PR_NUM already has an approving review." | ||
| fi | ||
|
|
||
| echo "MERGING: PR #$PR_NUM meets all criteria" | ||
| gh pr merge "$PR_NUM" --repo "$REPO" --merge --admin || echo "WARNING: Failed to merge PR #$PR_NUM" | ||
|
|
||
| done | ||
|
|
||
| echo "" | ||
| echo "Auto-merge check complete." | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Don't persist
GH_ACCESS_TOKENbefore executing branch code.This checks out a user-selected
branchwith a write-scoped PAT andpersist-credentials: true, then executesmakefrom that branch. A malicious branch can read the stored remote credentials and exfiltrate the PAT. Checkout without persisted credentials and use the PAT only for the final push/PR call.Suggested fix
- name: Checkout code uses: actions/checkout@v6 with: ref: ${{ env.BRANCH }} - token: ${{ secrets.GH_ACCESS_TOKEN }} - persist-credentials: true + persist-credentials: false @@ BRANCH_NAME="lockfile-update-$(date +%Y%m%d-%H%M)" git checkout -b "$BRANCH_NAME" git commit -m "Update lock files" - git push -u origin "$BRANCH_NAME" + git push -u "https://x-access-token:${GH_TOKEN}@github.com/${GITHUB_REPOSITORY}.git" "$BRANCH_NAME"Also applies to: 82-101
🤖 Prompt for AI Agents
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@jiridanek, thanks for pointing to the upstream PR opendatahub-io/notebooks#3202. Since this is a cherry-pick branch targetingrhoai-3.3, and the security fix is being handled upstream, the concern remains open in this PR until either the fix is cherry-picked here as well, or the team decides to accept the current behavior for this branch.The risk of
persist-credentials: truewithGH_ACCESS_TOKENon a user-controlled branch checkout is still present in the current state of this file — worth keeping in mind ifrhoai-3.3also runs this workflow against user-submitted branches.🧠 Learnings used