Expose SLSA source level from new verify-source task#2867
Conversation
|
@ralphbean: The following test has Failed, say /retest to rerun failed tests.
Inspecting Test ArtifactsTo inspect your test artifacts, follow these steps:
mkdir -p oras-artifacts
cd oras-artifacts
oras pull quay.io/konflux-test-storage/konflux-team/build-definitions:ebacc93ac8cf5da11af128f48cd4244ca1fc1cd3Test results analysis🚨 Error occurred while running the E2E tests, list of failed Spec(s): ➡️ [ Click to view logsExpected success, but got an error:
<*errors.errorString | 0xc000ec3bc0>:
pod: test-comp-krpn-on-pull-request-2mvl6-clone-repository-pod | init container: prepare
2025/10/09 02:09:39 Entrypoint initialization
pod: test-comp-krpn-on-pull-request-2mvl6-clone-repository-pod | init container: place-scripts
2025/10/09 02:09:39 Decoded script /tekton/scripts/script-0-mpphz
2025/10/09 02:09:39 Decoded script /tekton/scripts/script-1-65sd5
2025/10/09 02:09:39 Decoded script /tekton/scripts/script-2-bg228
pod: test-comp-krpn-on-pull-request-2mvl6-clone-repository-pod | container step-clone:
INFO: Using mounted CA bundle: /mnt/trusted-ca/ca-bundle.crt
{"level":"info","ts":1759975785.8784237,"caller":"git/git.go:380","msg":"Retrying operation (attempt 1)"}
{"level":"info","ts":1759975786.0381322,"caller":"git/git.go:217","msg":"Successfully cloned https://github.com/konflux-qe-bd/multiarch-sample-repo-clone @ ccec5c62f54b1f5babfa664bf91af0b0e5f2e44d (grafted, HEAD) in path /var/workdir/source"}
{"level":"info","ts":1759975786.0381913,"caller":"git/git.go:380","msg":"Retrying operation (attempt 1)"}
{"level":"info","ts":1759975786.070147,"caller":"git/git.go:263","msg":"Successfully initialized and updated submodules in path /var/workdir/source"}
Merge option disabled. Using checked-out revision ccec5c62f54b1f5babfa664bf91af0b0e5f2e44d directly.
pod: test-comp-krpn-on-pull-request-2mvl6-clone-repository-pod | container step-symlink-check:
Running symlink check
pod: test-comp-krpn-on-pull-request-2mvl6-clone-repository-pod | container step-slsa-verify:
=== SLSA Source Verification ===
Repository: github.com/konflux-qe-bd/multiarch-sample-repo-clone
Commit: ccec5c62f54b1f5babfa664bf91af0b0e5f2e44d
/tekton/scripts/script-2-bg228: line 27: SLSA_ARTIFACTS_DIR: unbound variable
pod: test-comp-krpn-on-pull-request-2mvl6-clone-repository-pod | container step-create-trusted-artifact:
2025/10/09 02:09:47 Skipping step because a previous step failed
pod: test-comp-krpn-on-pull-request-2mvl6-init-pod | init container: prepare
2025/10/09 02:09:33 Entrypoint initialization
pod: test-comp-krpn-on-pull-request-2mvl6-init-pod | init container: place-scripts
2025/10/09 02:09:34 Decoded script /tekton/scripts/script-0-jrb2s
pod: test-comp-krpn-on-pull-request-2mvl6-init-pod | container step-init:
Build Initialize: quay.io/redhat-user-workloads/build-templates-e2e/test-comp-krpn:on-pr-ccec5c62f54b1f5babfa664bf91af0b0e5f2e44d
Determine if Image Already Exists
pod: test-comp-krpn-on-pull-request-hvj6q-clone-repository-pod | init container: prepare
2025/10/09 02:09:03 Entrypoint initialization
pod: test-comp-krpn-on-pull-request-hvj6q-clone-repository-pod | init container: place-scripts
2025/10/09 02:09:05 Decoded script /tekton/scripts/script-0-lxwrg
2025/10/09 02:09:05 Decoded script /tekton/scripts/script-1-ts9bv
2025/10/09 02:09:05 Decoded script /tekton/scripts/script-2-nlsw9
pod: test-comp-krpn-on-pull-request-hvj6q-clone-repository-pod | container step-clone:
INFO: Using mounted CA bundle: /mnt/trusted-ca/ca-bundle.crt
{"level":"info","ts":1759975749.1212752,"caller":"git/git.go:380","msg":"Retrying operation (attempt 1)"}
{"level":"info","ts":1759975749.2948987,"caller":"git/git.go:217","msg":"Successfully cloned https://github.com/konflux-qe-bd/multiarch-sample-repo-clone @ 7cca8c27efce838fc70018f73bd76f4c7201e527 (grafted, HEAD) in path /var/workdir/source"}
{"level":"info","ts":1759975749.2949588,"caller":"git/git.go:380","msg":"Retrying operation (attempt 1)"}
{"level":"info","ts":1759975749.3274295,"caller":"git/git.go:263","msg":"Successfully initialized and updated submodules in path /var/workdir/source"}
Merge option disabled. Using checked-out revision 7cca8c27efce838fc70018f73bd76f4c7201e527 directly.
pod: test-comp-krpn-on-pull-request-hvj6q-clone-repository-pod | container step-symlink-check:
Running symlink check
pod: test-comp-krpn-on-pull-request-hvj6q-clone-repository-pod | container step-slsa-verify:
=== SLSA Source Verification ===
Repository: github.com/konflux-qe-bd/multiarch-sample-repo-clone
Commit: 7cca8c27efce838fc70018f73bd76f4c7201e527
/tekton/scripts/script-2-nlsw9: line 27: SLSA_ARTIFACTS_DIR: unbound variable
pod: test-comp-krpn-on-pull-request-hvj6q-clone-repository-pod | container step-create-trusted-artifact:
2025/10/09 02:09:10 Skipping step because a previous step failed
pod: test-comp-krpn-on-pull-request-hvj6q-init-pod | init container: prepare
2025/10/09 02:08:58 Entrypoint initialization
pod: test-comp-krpn-on-pull-request-hvj6q-init-pod | init container: place-scripts
2025/10/09 02:08:58 Decoded script /tekton/scripts/script-0-5cmtq
pod: test-comp-krpn-on-pull-request-hvj6q-init-pod | container step-init:
Build Initialize: quay.io/redhat-user-workloads/build-templates-e2e/test-comp-krpn:on-pr-7cca8c27efce838fc70018f73bd76f4c7201e527
Determine if Image Already Exists
pod: test-comp-krpn-on-pull-request-pzxdp-clone-repository-pod | init container: prepare
2025/10/09 02:08:00 Entrypoint initialization
pod: test-comp-krpn-on-pull-request-pzxdp-clone-repository-pod | init container: place-scripts
2025/10/09 02:08:00 Decoded script /tekton/scripts/script-0-j9jp9
2025/10/09 02:08:00 Decoded script /tekton/scripts/script-1-4rlf4
2025/10/09 02:08:00 Decoded script /tekton/scripts/script-2-jk7px
pod: test-comp-krpn-on-pull-request-pzxdp-clone-repository-pod | container step-clone:
INFO: Using mounted CA bundle: /mnt/trusted-ca/ca-bundle.crt
{"level":"info","ts":1759975692.0150113,"caller":"git/git.go:380","msg":"Retrying operation (attempt 1)"}
{"level":"info","ts":1759975692.1878688,"caller":"git/git.go:217","msg":"Successfully cloned https://github.com/konflux-qe-bd/multiarch-sample-repo-clone @ 4e6397236defba588c637cb0876bc546cbd51ca1 (grafted, HEAD) in path /var/workdir/source"}
{"level":"info","ts":1759975692.187935,"caller":"git/git.go:380","msg":"Retrying operation (attempt 1)"}
{"level":"info","ts":1759975692.2203214,"caller":"git/git.go:263","msg":"Successfully initialized and updated submodules in path /var/workdir/source"}
Merge option disabled. Using checked-out revision 4e6397236defba588c637cb0876bc546cbd51ca1 directly.
pod: test-comp-krpn-on-pull-request-pzxdp-clone-repository-pod | container step-symlink-check:
Running symlink check
pod: test-comp-krpn-on-pull-request-pzxdp-clone-repository-pod | container step-slsa-verify:
=== SLSA Source Verification ===
Repository: github.com/konflux-qe-bd/multiarch-sample-repo-clone
Commit: 4e6397236defba588c637cb0876bc546cbd51ca1
/tekton/scripts/script-2-jk7px: line 27: SLSA_ARTIFACTS_DIR: unbound variable
pod: test-comp-krpn-on-pull-request-pzxdp-clone-repository-pod | container step-create-trusted-artifact:
2025/10/09 02:08:13 Skipping step because a previous step failed
pod: test-comp-krpn-on-pull-request-pzxdp-init-pod | init container: prepare
2025/10/09 02:07:51 Entrypoint initialization
pod: test-comp-krpn-on-pull-request-pzxdp-init-pod | init container: place-scripts
2025/10/09 02:07:51 Decoded script /tekton/scripts/script-0-k8x4c
pod: test-comp-krpn-on-pull-request-pzxdp-init-pod | container step-init:
Build Initialize: quay.io/redhat-user-workloads/build-templates-e2e/test-comp-krpn:on-pr-4e6397236defba588c637cb0876bc546cbd51ca1
Determine if Image Already Exists
{
s: "\n pod: test-comp-krpn-on-pull-request-2mvl6-clone-repository-pod | init container: prepare\n2025/10/09 02:09:39 Entrypoint initialization\n\n pod: test-comp-krpn-on-pull-request-2mvl6-clone-repository-pod | init container: place-scripts\n2025/10/09 02:09:39 Decoded script /tekton/scripts/script-0-mpphz\n2025/10/09 02:09:39 Decoded script /tekton/scripts/script-1-65sd5\n2025/10/09 02:09:39 Decoded script /tekton/scripts/script-2-bg228\n\npod: test-comp-krpn-on-pull-request-2mvl6-clone-repository-pod | container step-clone: \nINFO: Using mounted CA bundle: /mnt/trusted-ca/ca-bundle.crt\n{\"level\":\"info\",\"ts\":1759975785.8784237,\"caller\":\"git/git.go:380\",\"msg\":\"Retrying operation (attempt 1)\"}\n{\"level\":\"info\",\"ts\":1759975786.0381322,\"caller\":\"git/git.go:217\",\"msg\":\"Successfully cloned https://github.com/konflux-qe-bd/multiarch-sample-repo-clone @ ccec5c62f54b1f5babfa664bf91af0b0e5f2e44d (grafted, HEAD) in path /var/workdir/source\"}\n{\"level\":\"info\",\"ts\":1759975786.0381913,\"caller\":\"git/git.go:380\",\"msg\":\"Retrying operation (attempt 1)\"}\n{\"level\":\"info\",\"ts\":1759975786.070147,\"caller\":\"git/git.go:263\",\"msg\":\"Successfully initialized and updated submodules in path /var/workdir/source\"}\nMerge option disabled. Using checked-out revision ccec5c62f54b1f5babfa664bf91af0b0e5f2e44d directly.\n\npod: test-comp-krpn-on-pull-request-2mvl6-clone-repository-pod | container step-symlink-check: \nRunning symlink check\n\npod: test-comp-krpn-on-pull-request-2mvl6-clone-repository-pod | container step-slsa-verify: \n=== SLSA Source Verification ===\nRepository: github.com/konflux-qe-bd/multiarch-sample-repo-clone\nCommit: ccec5c62f54b1f5babfa664bf91af0b0e5f2e44d\n/tekton/scripts/script-2-bg228: line 27: SLSA_ARTIFACTS_DIR: unbound variable\n\npod: test-comp-krpn-on-pull-request-2mvl6-clone-repository-pod | container step-create-trusted-artifact: \n2025/10/09 02:09:47 Skipping step because a previous step failed\n\n pod: test-comp-krpn-on-pull-request-2mvl6-init-pod | init container: prepare\n2025/10/09 02:09:33 Entrypoint initialization\n\n pod: test-comp-krpn-on-pull-request-2mvl6-init-pod | init container... the content is too long - please download the artifact to see the full content
OCI Artifact Browser URL<not enabled> |
There was a problem hiding this comment.
Do you have some more context for this? I'm not sure I understand the motivation for adding this and especially for doing this verification in the clone task.
I just tried running source-tool locally, and I have some observations:
- The repo says that it's a PoC, in-development, shouldn't be used in production
- It looks like the verification can be done at any time from anywhere and doesn't need to be part of the clone task
- I always get 403 API rate limit exceeded unless I pass a github token
The clone task may run into rate limits as well
./source-tool verifycommit \ --commit 134593d9158efd253e979e2e8d87b939945d091e \ --owner slsa-framework \ --repo source-tool \ --github_token $(gh auth token) - It only supports github.com
|
We wanted to add this in order to better support showing Konflux as and end-to-end example, i.e. https://slsa.dev/blog/2025/07/slsa-e2e The alternative to this would be to create a custom task for dedicated use in our end to end demo. Ralph and I were discussing this, saying that it would be easier for those following along at home to reproduce our demo if we introduced the logic into the task that we have instead of keeping it separate. If we have it in this task, I think we should add information to it in the task description, especially calling out that the tool integration is a POC. An alternative would be to kustomize the clone task, producing a separate one for the SLSA verification but this would have an additional maintenance overhead. |
If I'm reading the blog post right, shouldn't VSA verification happen during the Verification phase? Are we doing it at build time because that's the quicker way to set up a demo? I don't think this should go into a task that's part of actual production pipelines. |
|
FWIW, I'm working on gitlab support in source-tool right now. PR forthcoming: |
|
Source provenance is a pretty new idea. I suppose we should write an ADR about it to make sure the concepts are clear, away from the implementation. edit: ADR draft posted at konflux-ci/architecture#268 I'm going to move this into draft for the moment. |
That would be great 💯 |
This task verifies the SLSA source level of a git commit by checking for a Verification Summary Attestation (VSA) stored as a git note. Unlike git-clone, this task does not clone the repository - it only performs verification using the sourcetool verifycommit command. The task accepts a repository URL and commit SHA as parameters and outputs the achieved SLSA source level and test results. Includes tests for both positive (with VSA) and negative (without VSA) cases. Assisted-by: Claude Code Signed-off-by: Ralph Bean <rbean@redhat.com>
e54d78a to
662aec2
Compare
- Add warning to task description noting that source-tool is a proof-of-concept, not production-ready - Mention GitHub-only support and potential rate limiting - Fix CODEOWNERS entry from /task/run-script-oci-ta to /task/verify-source Assisted-by: Claude Code Signed-off-by: Ralph Bean <rbean@redhat.com>
Add optional basic-auth workspace to provide GitHub authentication token to avoid API rate limits. The task looks for a 'token' file in the workspace and passes it to sourcetool via --github_token. Includes documentation and examples in the README showing how to create a secret and use it with the task. Assisted-by: Claude Code Signed-off-by: Ralph Bean <rbean@redhat.com>
Instead of passing tokens as command-line arguments (which may be logged), the task now sets environment variables (GITHUB_TOKEN or GITLAB_TOKEN) based on the repository host. This allows sourcetool to passively pick up the credentials without exposing them in logs. Changes: - Detect repo host and set appropriate env var (GITHUB_TOKEN/GITLAB_TOKEN) - Remove command-line flag approach - Update task description to mention GitLab support - Update workspace description for both GitHub and GitLab - Update README with examples for both platforms Assisted-by: Claude Code Signed-off-by: Ralph Bean <rbean@redhat.com>
| WARNING: This task uses source-tool (https://github.com/slsa-framework/source-tool) | ||
| which is currently a proof-of-concept and under active development. It should | ||
| not be used in production environments. It supports GitHub and GitLab repositories, | ||
| and may encounter API rate limits without authentication. |
There was a problem hiding this comment.
It isn't necessarily under active development. Just the fact that it is a POC is enough for us to say that you might not want to use it in production.
There was a problem hiding this comment.
Yeah - and the "GitHub and GitLab" statement here is not true, yet. I'm still cleaning up https://github.com/ralphbean/source-tool/blob/gitlab-support/docs/VCS.md
| optional: true | ||
| steps: | ||
| - name: slsa-verify | ||
| image: quay.io/konflux-ci/git-clone@sha256:bd303d16e9d9b01622d69deff77c583ebdea36611b15dc243da658d93763e8de |
| the verification using the sourcetool verifycommit command. | ||
|
|
||
| WARNING: This task uses source-tool (https://github.com/slsa-framework/source-tool) |
There was a problem hiding this comment.
we should consistently use sourcetool or source-tool. @puerco, is one generally preferred? :)
There was a problem hiding this comment.
I think it should be source-tool everywhere.
|
|
||
| # Initialize test results | ||
| VERIFICATION_RESULT="PASSED" | ||
| ACHIEVED_LEVEL="SLSA_SOURCE_LEVEL_1" |
There was a problem hiding this comment.
We should state that this is an assumption based on the fact that we have a repository url and revision. Do we have any checks to ensure that the repository/revision are valid or is that just an assumption too?
| # Fallback to basic level 1 | ||
| ACHIEVED_LEVEL="SLSA_SOURCE_LEVEL_1" |
There was a problem hiding this comment.
If we are explicitly setting this here, should we not have an explicit level in the initialization? Or should we just remove these, letting us fall back to the default?
| if grep -q "FAILED:" "$VERIFICATION_LOG"; then | ||
| echo "Source-tool verification failed - no VSA found in repository" |
There was a problem hiding this comment.
Are there valid failure cases that might exist which would make us be level 0? For example, if there is an invalid reference, we wouldn't be able to claim that it is in a modern source tool.
|
|
||
| # Extract the actual SLSA level from sourcetool output | ||
| # Format: "SUCCESS: commit ... verified with [SLSA_SOURCE_LEVEL_3 TAG_HYGIENE]" | ||
| EXTRACTED_LEVEL=$(grep -oP '\[SLSA_SOURCE_LEVEL_\d+' "$VERIFICATION_LOG" | sed 's/\[//') |
There was a problem hiding this comment.
Instead of performing log parsing, would it be possible to parse the VSA for the source level?
There was a problem hiding this comment.
It looked a lot simpler with source-tool 😅
But if you both prefer this solution, I don't mind. Side note: it would be great if source-tool could return machine-readable output
There was a problem hiding this comment.
Side note: let's do that. :)
Update all references from sourcetool to source-tool to match the correct command name throughout the verify-source task. Assisted-by: Claude Code Signed-off-by: Ralph Bean <rbean@redhat.com>
Update verify-source task to fetch git notes and parse the VSA JSON directly to extract SLSA source level, instead of relying on parsing source-tool command output. This approach is more reliable and maintainable. The task now: - Fetches git notes (refs/notes/commits) from the repository - Parses the VSA JSON structure - Decodes the base64 payload to extract verifiedLevels Assisted-by: Claude Code Signed-off-by: Ralph Bean <rbean@redhat.com>
Reduce deep nesting in the verification script: - Inverted git fetch condition for early exit - Flattened VSA parsing by chaining operations linearly - Removed redundant SLSA_SOURCE_LEVEL_1 fallback assignments (already set as default at initialization) - Consolidated cleanup commands This reduces nesting from 5-6 levels down to 2-3 levels max, making the code more readable and maintainable. Tests pass: both with-vsa and no-vsa scenarios work correctly. Assisted-by: Claude Code Signed-off-by: Ralph Bean <rbean@redhat.com>
chmeliik
left a comment
There was a problem hiding this comment.
The behavior of the task doesn't match the proposed flow from konflux-ci/architecture#268
I guess this a quick PoC for demo purposes which would later be reworked to follow the proposal?
Yeah, I think that's right. I'd like to get this merged soon as long as its not a total dead end so that we can demo. We can write down the real expectations in that ADR and then revise the task to match. |
chmeliik
left a comment
There was a problem hiding this comment.
In the context of #2867 (comment), this looks good enough to me. Some comments that would need to be addressed in a proper implementation, non-blocking for now
Consider squashing the commits, since their sequence will more likely confuse a reviewer rather than help
| - name: basic-auth | ||
| description: | | ||
| A Workspace containing a token file for API authentication. | ||
| The workspace should contain a file named 'token' with a GitHub | ||
| personal access token, GitLab personal access token, or other | ||
| authentication token. The task will automatically set the appropriate |
There was a problem hiding this comment.
It's a shame that this doesn't work with the same kind of credentials as the git-clone task. That feels like a point against the source-tool approach.
If source-tool supported verifying a locally cloned commit, that might be the best of both worlds. Clone using whatever auth you want (or just fetch from a trusted artifact), use source-tool to hide away the complexities of source VSA verification
| case "$REPO_HOST" in | ||
| *github.com) | ||
| export GITHUB_TOKEN="$TOKEN" | ||
| echo "Using GitHub token from workspace for authentication" | ||
| ;; | ||
| *gitlab.com|*gitlab.*) | ||
| export GITLAB_TOKEN="$TOKEN" | ||
| echo "Using GitLab token from workspace for authentication" | ||
| ;; |
There was a problem hiding this comment.
I'll just note that this doesn't work with the current approach (no source-tool, raw git operations and manual verification)
| set -eux | ||
|
|
||
| # Strip trailing newlines from results | ||
| LEVEL=$(echo "$SLSA_LEVEL_ACHIEVED" | tr -d '\n') |
There was a problem hiding this comment.
Consider not removing the newline here - I think we want the Task to return a newline-less result and the test should verify that
Modify Task to write results using printf instead of echo to avoid trailing newlines. Update tests to expect newline-less results instead of stripping them. Addresses review comment from PR konflux-ci#2867. Assisted-by: Claude Code Signed-off-by: Ralph Bean <rbean@redhat.com>
Modify Task to write results using printf instead of echo to avoid trailing newlines. Update tests to expect newline-less results instead of stripping them. Addresses review comment from PR #2867. Assisted-by: Claude Code Signed-off-by: Ralph Bean <rbean@redhat.com>
Expose SLSA source level from the git clone task
This depends on another process outside of konflux producing a VSA that attests to the SLSA source level of the commit, and pushing it as a git note for the commit.
In the absence of a VSA, we default to SLSA source level 1 which is just "use source control", which is unavoidable with Konflux.
Note the new tests for git-clone to demonstrate both a positive and negative case.