CNTRLPLANE-2579: feat(konflux): add script to update pipeline task bundles to latest trusted versions - #7553
Conversation
…rusted versions This script fetches trusted tasks data from the Konflux data-acceptable-bundles OCI artifact and updates pipeline YAML files to use the latest trusted task bundle digests. Features: - Updates task bundle digests to latest trusted versions - Shows available version upgrades (e.g., 0.3 → 0.4) - With --upgrade-versions, also applies version upgrades - Supports dry-run mode for CI checks - JSON output for automation Jira: CNTRLPLANE-2579 Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
|
Skipping CI for Draft Pull Request. |
|
@celebdor: This pull request references CNTRLPLANE-2579 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.22.0" version, but no target version was set. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
openshift-ci
Bot
added
the
do-not-merge/work-in-progress
openshift-ci-robot
added
the
jira/valid-reference
WalkthroughA new Python tool is implemented to update Tekton Pipeline task bundles to latest trusted versions. The script fetches trusted tasks data from OCI artifacts, parses pipeline YAML files to identify bundle references, analyzes them against trusted data to detect updates or upgrades, and provides diff/apply/JSON output modes with dry-run support. Changes
Estimated code review effort🎯 4 (Complex) | ⏱️ ~50 minutes ✨ Finishing touches
Comment |
openshift-ci
Bot
added
the
area/ci-tooling
|
/area ci-tooling |
openshift-ci
Bot
removed
the
do-not-merge/work-in-progress
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Fix all issues with AI agents
In `@hack/tools/scripts/update_trusted_task_bundles.py`:
- Around line 188-216: The fetch_trusted_tasks_data function currently calls
subprocess.run without a timeout and ignores the cache_dir parameter; update the
subprocess.run call in fetch_trusted_tasks_data to include a reasonable timeout
(e.g., seconds value) and add an except subprocess.TimeoutExpired handler that
raises a clear RuntimeError with stderr/timeout context; also either implement
caching by using the provided cache_dir to persist the skopeo output (use
cache_dir as the destination instead of a tempdir when provided, e.g.,
Path(cache_dir)/"data") or remove the unused cache_dir parameter and related
function signature to avoid dead code—modify only fetch_trusted_tasks_data and
its error handling to reference subprocess.run, subprocess.TimeoutExpired, and
the cache_dir usage (or removal).
| def fetch_trusted_tasks_data(data_source: str, cache_dir: Optional[str] = None) -> TrustedTasksData: | ||
| """Fetch trusted tasks data from OCI artifact using skopeo. | ||
|
|
||
| Args: | ||
| data_source: OCI image reference (e.g., 'quay.io/konflux-ci/tekton-catalog/data-acceptable-bundles:latest') | ||
| cache_dir: Optional directory to cache the fetched data | ||
|
|
||
| Returns: | ||
| TrustedTasksData object with parsed trusted_tasks | ||
| """ | ||
| # Create temp directory for skopeo output | ||
| with tempfile.TemporaryDirectory() as tmpdir: | ||
| dest_dir = Path(tmpdir) / "data" | ||
|
|
||
| # Use skopeo to copy the OCI artifact | ||
| cmd = [ | ||
| "skopeo", "copy", "--preserve-digests", | ||
| f"docker://{data_source}", | ||
| f"dir:{dest_dir}" | ||
| ] | ||
|
|
||
| print(f"Fetching trusted tasks data from {data_source}...", file=sys.stderr) | ||
| try: | ||
| result = subprocess.run(cmd, capture_output=True, text=True, check=True) | ||
| except subprocess.CalledProcessError as e: | ||
| print(f"Error fetching data: {e.stderr}", file=sys.stderr) | ||
| raise RuntimeError(f"Failed to fetch trusted tasks data: {e}") | ||
| except FileNotFoundError: | ||
| raise RuntimeError("skopeo not found. Please install skopeo.") |
There was a problem hiding this comment.
Add timeout to subprocess call to prevent hangs.
The subprocess.run call on line 211 has no timeout. If skopeo hangs (e.g., network issues, unresponsive registry), this script will block indefinitely, which is problematic for CI pipelines.
Also, the cache_dir parameter is declared but never used—consider removing it or implementing the caching logic.
Proposed fix
-def fetch_trusted_tasks_data(data_source: str, cache_dir: Optional[str] = None) -> TrustedTasksData:
+def fetch_trusted_tasks_data(data_source: str) -> TrustedTasksData:
"""Fetch trusted tasks data from OCI artifact using skopeo.
Args:
data_source: OCI image reference (e.g., 'quay.io/konflux-ci/tekton-catalog/data-acceptable-bundles:latest')
- cache_dir: Optional directory to cache the fetched data
Returns:
TrustedTasksData object with parsed trusted_tasks
""" try:
- result = subprocess.run(cmd, capture_output=True, text=True, check=True)
+ subprocess.run(cmd, capture_output=True, text=True, check=True, timeout=120)
except subprocess.CalledProcessError as e:
print(f"Error fetching data: {e.stderr}", file=sys.stderr)
- raise RuntimeError(f"Failed to fetch trusted tasks data: {e}")
+ raise RuntimeError(f"Failed to fetch trusted tasks data: {e}") from e
except FileNotFoundError:
- raise RuntimeError("skopeo not found. Please install skopeo.")
+ raise RuntimeError("skopeo not found. Please install skopeo.") from None
+ except subprocess.TimeoutExpired as e:
+ raise RuntimeError(f"Timeout fetching trusted tasks data after {e.timeout}s") from e📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| def fetch_trusted_tasks_data(data_source: str, cache_dir: Optional[str] = None) -> TrustedTasksData: | |
| """Fetch trusted tasks data from OCI artifact using skopeo. | |
| Args: | |
| data_source: OCI image reference (e.g., 'quay.io/konflux-ci/tekton-catalog/data-acceptable-bundles:latest') | |
| cache_dir: Optional directory to cache the fetched data | |
| Returns: | |
| TrustedTasksData object with parsed trusted_tasks | |
| """ | |
| # Create temp directory for skopeo output | |
| with tempfile.TemporaryDirectory() as tmpdir: | |
| dest_dir = Path(tmpdir) / "data" | |
| # Use skopeo to copy the OCI artifact | |
| cmd = [ | |
| "skopeo", "copy", "--preserve-digests", | |
| f"docker://{data_source}", | |
| f"dir:{dest_dir}" | |
| ] | |
| print(f"Fetching trusted tasks data from {data_source}...", file=sys.stderr) | |
| try: | |
| result = subprocess.run(cmd, capture_output=True, text=True, check=True) | |
| except subprocess.CalledProcessError as e: | |
| print(f"Error fetching data: {e.stderr}", file=sys.stderr) | |
| raise RuntimeError(f"Failed to fetch trusted tasks data: {e}") | |
| except FileNotFoundError: | |
| raise RuntimeError("skopeo not found. Please install skopeo.") | |
| def fetch_trusted_tasks_data(data_source: str) -> TrustedTasksData: | |
| """Fetch trusted tasks data from OCI artifact using skopeo. | |
| Args: | |
| data_source: OCI image reference (e.g., 'quay.io/konflux-ci/tekton-catalog/data-acceptable-bundles:latest') | |
| Returns: | |
| TrustedTasksData object with parsed trusted_tasks | |
| """ | |
| # Create temp directory for skopeo output | |
| with tempfile.TemporaryDirectory() as tmpdir: | |
| dest_dir = Path(tmpdir) / "data" | |
| # Use skopeo to copy the OCI artifact | |
| cmd = [ | |
| "skopeo", "copy", "--preserve-digests", | |
| f"docker://{data_source}", | |
| f"dir:{dest_dir}" | |
| ] | |
| print(f"Fetching trusted tasks data from {data_source}...", file=sys.stderr) | |
| try: | |
| subprocess.run(cmd, capture_output=True, text=True, check=True, timeout=120) | |
| except subprocess.CalledProcessError as e: | |
| print(f"Error fetching data: {e.stderr}", file=sys.stderr) | |
| raise RuntimeError(f"Failed to fetch trusted tasks data: {e}") from e | |
| except FileNotFoundError: | |
| raise RuntimeError("skopeo not found. Please install skopeo.") from None | |
| except subprocess.TimeoutExpired as e: | |
| raise RuntimeError(f"Timeout fetching trusted tasks data after {e.timeout}s") from e |
🧰 Tools
🪛 Ruff (0.14.13)
188-188: Unused function argument: cache_dir
(ARG001)
211-211: Local variable result is assigned to but never used
Remove assignment to unused variable result
(F841)
211-211: subprocess call: check for execution of untrusted input
(S603)
214-214: Within an except clause, raise exceptions with raise ... from err or raise ... from None to distinguish them from errors in exception handling
(B904)
214-214: Avoid specifying long messages outside the exception class
(TRY003)
216-216: Within an except clause, raise exceptions with raise ... from err or raise ... from None to distinguish them from errors in exception handling
(B904)
216-216: Avoid specifying long messages outside the exception class
(TRY003)
🤖 Prompt for AI Agents
In `@hack/tools/scripts/update_trusted_task_bundles.py` around lines 188 - 216,
The fetch_trusted_tasks_data function currently calls subprocess.run without a
timeout and ignores the cache_dir parameter; update the subprocess.run call in
fetch_trusted_tasks_data to include a reasonable timeout (e.g., seconds value)
and add an except subprocess.TimeoutExpired handler that raises a clear
RuntimeError with stderr/timeout context; also either implement caching by using
the provided cache_dir to persist the skopeo output (use cache_dir as the
destination instead of a tempdir when provided, e.g., Path(cache_dir)/"data") or
remove the unused cache_dir parameter and related function signature to avoid
dead code—modify only fetch_trusted_tasks_data and its error handling to
reference subprocess.run, subprocess.TimeoutExpired, and the cache_dir usage (or
removal).
|
/approve |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: bryan-cox, celebdor The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
openshift-ci
Bot
added
the
approved
|
@celebdor: all tests passed! Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
|
/lgtm |
openshift-ci-robot
added
the
verified
|
@bryan-cox: The DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
3 participants
What this PR does / why we need it:
Adds a Python script to automate updating Tekton pipeline task bundle references to the latest trusted versions from the Konflux catalog.
Tekton pipeline task bundles in
.tekton/pipelines/need to be periodically updated to use the latest trusted digests from thedata-acceptable-bundlesOCI artifact. This ensures:The script
hack/tools/scripts/update_trusted_task_bundles.pyfetches trusted tasks data fromquay.io/konflux-ci/tekton-catalog/data-acceptable-bundles:latestand updates pipeline YAML files.Features
0.3 → 0.4)--upgrade-versions, also applies version upgradesUsage
Which issue(s) this PR fixes:
Fixes https://issues.redhat.com/browse/CNTRLPLANE-2579
Special notes for your reviewer:
Checklist: