-
Notifications
You must be signed in to change notification settings - Fork 118
feat: Report validation errors on pull requests as comment #1963
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: Report validation errors on pull requests as comment #1963
Conversation
1a30d06 to
b63beda
Compare
vdemeester
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👼🏼
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
PR Overview
This PR adds functionality to report PipelineRun validation errors as comments on pull requests and introduces a new provider interface function CreateComment. Key changes include:
- Reporting PipelineRun YAML validation errors as a markdown-formatted comment on pull requests.
- Adding a CreateComment function implementation across multiple provider packages (GitHub, GitLab, Gitea, Bitbucket Cloud, Bitbucket Server).
- Updating tests and documentation to reflect the new error reporting behavior.
Reviewed Changes
| File | Description |
|---|---|
| test/testdata/failures/pipeline-validation.yaml | New invalid PipelineRun template for testing validation errors |
| test/gitea_test.go | Added tests for reporting bad YAML errors on pull requests |
| docs/content/docs/guide/running.md | Added documentation for YAML parsing error reporting |
| pkg/provider/github/github.go | Added CreateComment implementation for GitHub |
| pkg/pipelineascode/match.go | Enhanced error reporting by creating a markdown comment with YAML errors |
| pkg/provider/gitlab/gitlab.go, gitea.go, etc. | Added CreateComment implementations for other providers |
| pkg/provider/interface.go | Updated provider interface to include CreateComment |
Copilot reviewed 21 out of 21 changed files in this pull request and generated 1 comment.
06c2a02 to
4c16336
Compare
|
Once this is merged, I think it will also lay a good foundation for alerting users to warnings such as |
4c16336 to
fc81197
Compare
|
/test doc-generation |
I'd like to get a full blown linter available in tkn pac for this, maybe we can add the output in the comment... or inline as annotations on the code |
|
but let's discuss this on the jira/issue |
|
/retest |
1 similar comment
|
/retest |
f529b6d to
1c9584e
Compare
|
Testing |
|
/cancel go-testing |
|
/retest |
|
and also if I send |
|
ah this is weird |
|
@chmouel can you please update this? |
9148f7e to
97a1902
Compare
|
I don't think i can quite reproduce your issue chmouel/scratchmyback#352 it seems to work for me, can you give me a clear path to reproduce this ? (by recording a video for example) |
@chmouel in your PR if you see |
c5da652 to
f9cdcd4
Compare
|
/retest |
f9cdcd4 to
aea95cd
Compare
|
This should be fixed (it was just a regexp quote issue)! see chmouel/scratchmyback#358 cc @zakisk |
|
AI: Spelling Mistakes & Corrections
Bugs & Potential Issues
Code Improvements & Idiomatic Go
Documentations & Notes
Edge Cases & Considerations
Suggestions for Future Work
SummaryThe code changes are well-structured, address the intended functionality (YAML error reporting via PR comments), and include comprehensive tests. The documentation updates (mostly correcting typos) are appropriate. Potential enhancements focus on improving clarity and robustness for edge cases, particularly around unsupported features in some providers and regex parsing.
|
|
@chmouel tested it working fine now /lgtm |
|
/lgtm |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Congrats @chmouel your PR Has been approved 🎉
✅ Pull Request Approved
Approval Status:
- Required Approvals: 1
- Current Approvals: 2
👥 Reviewers Who Approved:
| Reviewer | Permission Level | Approval Status |
|---|---|---|
| @vdemeester | admin |
✅ |
| @zakisk | admin |
✅ |
📝 Next Steps
- Ensure all required checks pass
- Comply with branch protection rules
- Request a maintainer to merge using the
/mergecommand (or merge it
directly if you have repository permission).
Automated by the PAC Boussole 🧭
LGTM Not working with this 😕 |
🤖 AI Review provided by openrouter:x-ai/grok-3-beta:After carefully analyzing the changes in the provided git diff, I'll summarize my findings regarding potential bugs, spelling mistakes, and opportunities for code improvement. The changes primarily focus on enhancing error reporting for invalid YAML in PipelineRun templates, adding comments on pull requests when errors are detected, and updating documentation. Observations and Analysis1. Documentation Changes
2. Code Changes for Error Reporting
3. Provider Interface and Implementations
4. End-to-End Tests
5. General Observations
Potential Bugs
Spelling Mistakes
Code Improvements for Idiomatic Go
ConclusionThe changes introduce a valuable feature for user feedback on YAML errors in PipelineRun templates via pull request comments. The implementation is robust across supported providers with good test coverage. However, there are minor areas of concern regarding regex fragility and unsupported providers (Bitbucket) that could be mitigated with additional logging or fallback mechanisms. Documentation updates are clear and helpful, and code improvements suggested focus on maintainability and idiomatic Go practices. If you'd like me to elaborate on any specific part or provide a code snippet for suggested improvements (e.g., utility function for Markdown tables or structured error handling), please let me know! |
🤖 AI Review provided by gemini:gemini-2.5-pro-exp-03-25:Okay, let's break down these changes carefully. Overall Goal of the Change: The primary goal of this change appears to be enhancing the user experience when Pipelines-as-Code encounters errors while parsing or validating PipelineRun definitions found in the
Analysis by File:
Summary & Verdict:
This is a well-implemented feature enhancement with good documentation and test coverage. The change significantly improves feedback for users encountering malformed or invalid PipelineRun definitions in their repositories. |
This commit introduces the capability to report PipelineRun validation errors as comments on pull requests. This functionality is triggered when validation errors are detected and the event type is a pull request. The errors are formatted into a markdown table and posted as a comment on the pull request, allowing users to easily identify and address issues in their PipelineRun templates. The commit also includes updates to the provider interfaces to include a CreateComment function, and implementations for GitHub, GitLab, Bitbucket Cloud, Bitbucket Server and Gitea providers. Signed-off-by: Chmouel Boudjnah <[email protected]>
eca4596 to
6ef4357
Compare
- Corrected the script path from `./hack/gh-workflow-ci` to `./hack/gh-workflow-ci.sh` for the log output step. - Ensured the log output step always runs using `if: always()` to aid debugging failures. Signed-off-by: Chmouel Boudjnah <[email protected]>
6ef4357 to
ac7ef5d
Compare

Changes
This commit introduces the capability to report PipelineRun validation errors as comments on pull requests. This functionality is triggered when validation errors are detected and the event type is a pull request. The errors are formatted into a markdown table and posted as a comment on the pull request, allowing users to easily identify and address issues in their PipelineRun templates.
The commit also includes updates to the provider interfaces to include a CreateComment function, and implementations for GitHub, GitLab, Bitbucket Cloud, Bitbucket Server and Gitea providers.
Jira: https://issues.redhat.com/browse/SRVKP-7075
Submitter Checklist
📝 Ensure your commit message is clear and informative. Refer to the How to write a git commit message guide. Include the commit message in the PR body rather than linking to an external site (e.g., Jira ticket).
♽ Run make test lint before submitting a PR to avoid unnecessary CI processing. Consider installing pre-commit and running pre-commit install in the repository root for an efficient workflow.
✨ We use linters to maintain clean and consistent code. Run make lint before submitting a PR. Some linters offer a --fix mode, executable with make fix-linters (ensure markdownlint and golangci-lint are installed).
📖 Document any user-facing features or changes in behavior.
🧪 While 100% coverage isn't required, we encourage unit tests for code changes where possible.
🎁 If feasible, add an end-to-end test. See README for details.
🔎 Address any CI test flakiness before merging, or provide a valid reason to bypass it (e.g., token rate limitations).
If adding a provider feature, fill in the following details: