Stabilize review_trivyignore Bats tests in CI#704
Conversation
review_trivyignore Bats tests in CI
|
Testing script results from test/test.sh Test script outputs from test/test.sh
docker: invalid reference format Run 'docker run --help' for more information Run 'docker run --help' for more information Run 'docker run --help' for more information Run 'docker run --help' for more information Run 'docker run --help' for more information Run 'docker run --help' for more information |
🛡️ Trivy Scan Report🔧 Config Scan: ✅ Clean🐳 Image Scan: ❌ Vulnerabilities FoundShow Findings |
PR smoke test of comment with security scanning resultsResults
Suppressed ResultsNothing here. Rules informationRules detailsDetails{
"driver": {
"name": "dependency-check",
"version": "7.1.0",
"informationUri": "https://jeremylong.github.io/DependencyCheck/",
"rules": [
{
"id": "CVE-2022-24823",
"shortDescription": {
"text": "Medium severity - CVE-2022-24823 Exposure of Resource to Wrong Sphere vulnerability in pkg:maven/io.netty/netty-transport@4.1.76.Final"
},
"fullDescription": {
"text": "Netty is an open-source, asynchronous event-driven network application framework. The package `io.netty:netty-codec-http` prior to version 4.1.77.Final contains an insufficient fix for CVE-2021-21290. When Netty's multipart decoders are used local information disclosure can occur via the local system temporary directory if temporary storing uploads on the disk is enabled. This only impacts applications running on Java version 6 and lower. Additionally, this vulnerability impacts code running on Unix-like systems, and very old versions of Mac OSX and Windows as they all share the system temporary directory between all users. Version 4.1.77.Final contains a patch for this vulnerability. As a workaround, specify one's own `java.io.tmpdir` when starting the JVM or use DefaultHttpDataFactory.setBaseDir(...) to set the directory to something that is only readable by the current user."
},
"help": {
"text": "",
"markdown": "For more information see [CVE-2022-24823](http://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2022-24823).\n\n\nIf this is a false positive - consider using the HTML report to generate a suppression file. For more information see [How dependency-check works](https://jeremylong.github.io/DependencyCheck/general/internals.html), [How to read the HTML report](https://jeremylong.github.io/DependencyCheck/general/thereport.html), and [Suppressing false positives](https://jeremylong.github.io/DependencyCheck/general/suppression.html)."
},
"properties": {
"cvssv2_score": 1.9,
"cvssv2_accessVector": "LOCAL",
"cvssv2_accessComplexity": "MEDIUM",
"cvssv2_authenticationr": "NONE",
"cvssv2_confidentialImpact": "$enc.json($rule.cvssv2ConfidentialityImpact)",
"cvssv2_integrityImpact": "NONE",
"cvssv2_availabilityImpact": "NONE",
"cvssv2_severity": "LOW",
"cvssv3_baseScore": 5.5,
"cvssv3_attackVector": "LOCAL",
"cvssv3_attackComplexity": "LOW",
"cvssv3_privilegesRequired": "LOW",
"cvssv3_userInteraction": "NONE",
"cvssv3_scope": "UNCHANGED",
"cvssv3_confidentialityImpact": "HIGH",
"cvssv3_integrityImpact": "NONE",
"cvssv3_availabilityImpact": "NONE",
"cvssv3_baseSeverity": "MEDIUM",
"cvssv3_exploitabilityScore": "1.8",
"cvssv3_impactScore": "3.6",
"cvssv3_version": "3.1",
"source": "NVD"
},
"defaultConfiguration": {
"level": "error"
}
}
],
"properties": {
"disclaimer": "Dependency-Check is an open source tool performing a best effort analysis of 3rd party dependencies; false positives and false negatives may exist in the analysis performed by the tool. Use of the tool and the reporting provided constitutes acceptance for use in an AS IS condition, and there are NO warranties, implied or otherwise, with regard to the analysis or its use. Any use of the tool and the reporting provided is at the user's risk. In no event shall the copyright holder or OWASP be held liable for any damages whatsoever arising out of or in connection with the use of this tool, the analysis performed, or the resulting report.",
"nvd": "This report contains data retrieved from the National Vulnerability Database: http://nvd.nist.gov",
"npm": "This report may contain data retrieved from the NPM Public Advisories: https://www.npmjs.com/advisories",
"retirejs": "This report may contain data retrieved from the RetireJS community: https://retirejs.github.io/retire.js/",
"ossindex": "This report may contain data retrieved from the Sonatype OSS Index: https://ossindex.sonatype.org",
"NVD CVE Checked": "2022-05-25T15:58:07",
"NVD CVE Modified": "2022-05-25T14:00:01",
"VersionCheckOn": "2022-05-25T15:58:07"
}
}
}Tool information
|
|
All this is still broken! |
|
All this is still broken! github-actions Bot Test script outputs from test/test.sh docker: invalid reference format Run 'docker run --help' for more information Run 'docker run --help' for more information Run 'docker run --help' for more information Run 'docker run --help' for more information Run 'docker run --help' for more information Run 'docker run --help' for more information |
|
The |
Fixed in |
|
Testing script results from test/test.sh Test script outputs from test/test.sh
docker: invalid reference format Run 'docker run --help' for more information Run 'docker run --help' for more information Run 'docker run --help' for more information Run 'docker run --help' for more information Run 'docker run --help' for more information Run 'docker run --help' for more information |
PR smoke test of comment with security scanning resultsResults
Suppressed ResultsNothing here. Rules informationRules detailsDetails{
"driver": {
"name": "dependency-check",
"version": "7.1.0",
"informationUri": "https://jeremylong.github.io/DependencyCheck/",
"rules": [
{
"id": "CVE-2022-24823",
"shortDescription": {
"text": "Medium severity - CVE-2022-24823 Exposure of Resource to Wrong Sphere vulnerability in pkg:maven/io.netty/netty-transport@4.1.76.Final"
},
"fullDescription": {
"text": "Netty is an open-source, asynchronous event-driven network application framework. The package `io.netty:netty-codec-http` prior to version 4.1.77.Final contains an insufficient fix for CVE-2021-21290. When Netty's multipart decoders are used local information disclosure can occur via the local system temporary directory if temporary storing uploads on the disk is enabled. This only impacts applications running on Java version 6 and lower. Additionally, this vulnerability impacts code running on Unix-like systems, and very old versions of Mac OSX and Windows as they all share the system temporary directory between all users. Version 4.1.77.Final contains a patch for this vulnerability. As a workaround, specify one's own `java.io.tmpdir` when starting the JVM or use DefaultHttpDataFactory.setBaseDir(...) to set the directory to something that is only readable by the current user."
},
"help": {
"text": "",
"markdown": "For more information see [CVE-2022-24823](http://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2022-24823).\n\n\nIf this is a false positive - consider using the HTML report to generate a suppression file. For more information see [How dependency-check works](https://jeremylong.github.io/DependencyCheck/general/internals.html), [How to read the HTML report](https://jeremylong.github.io/DependencyCheck/general/thereport.html), and [Suppressing false positives](https://jeremylong.github.io/DependencyCheck/general/suppression.html)."
},
"properties": {
"cvssv2_score": 1.9,
"cvssv2_accessVector": "LOCAL",
"cvssv2_accessComplexity": "MEDIUM",
"cvssv2_authenticationr": "NONE",
"cvssv2_confidentialImpact": "$enc.json($rule.cvssv2ConfidentialityImpact)",
"cvssv2_integrityImpact": "NONE",
"cvssv2_availabilityImpact": "NONE",
"cvssv2_severity": "LOW",
"cvssv3_baseScore": 5.5,
"cvssv3_attackVector": "LOCAL",
"cvssv3_attackComplexity": "LOW",
"cvssv3_privilegesRequired": "LOW",
"cvssv3_userInteraction": "NONE",
"cvssv3_scope": "UNCHANGED",
"cvssv3_confidentialityImpact": "HIGH",
"cvssv3_integrityImpact": "NONE",
"cvssv3_availabilityImpact": "NONE",
"cvssv3_baseSeverity": "MEDIUM",
"cvssv3_exploitabilityScore": "1.8",
"cvssv3_impactScore": "3.6",
"cvssv3_version": "3.1",
"source": "NVD"
},
"defaultConfiguration": {
"level": "error"
}
}
],
"properties": {
"disclaimer": "Dependency-Check is an open source tool performing a best effort analysis of 3rd party dependencies; false positives and false negatives may exist in the analysis performed by the tool. Use of the tool and the reporting provided constitutes acceptance for use in an AS IS condition, and there are NO warranties, implied or otherwise, with regard to the analysis or its use. Any use of the tool and the reporting provided is at the user's risk. In no event shall the copyright holder or OWASP be held liable for any damages whatsoever arising out of or in connection with the use of this tool, the analysis performed, or the resulting report.",
"nvd": "This report contains data retrieved from the National Vulnerability Database: http://nvd.nist.gov",
"npm": "This report may contain data retrieved from the NPM Public Advisories: https://www.npmjs.com/advisories",
"retirejs": "This report may contain data retrieved from the RetireJS community: https://retirejs.github.io/retire.js/",
"ossindex": "This report may contain data retrieved from the Sonatype OSS Index: https://ossindex.sonatype.org",
"NVD CVE Checked": "2022-05-25T15:58:07",
"NVD CVE Modified": "2022-05-25T14:00:01",
"VersionCheckOn": "2022-05-25T15:58:07"
}
}
}Tool information
|
|
The primary tests on this repo are still failing Testing script results from test/test.sh Test script outputs from test/test.sh docker: invalid reference format Run 'docker run --help' for more information Run 'docker run --help' for more information Run 'docker run --help' for more information Run 'docker run --help' for more information Run 'docker run --help' for more information Run 'docker run --help' for more information |
Addressed in
Addressed in |
PR smoke test of comment with security scanning resultsResults
Suppressed ResultsNothing here. Rules informationRules detailsDetails{
"driver": {
"name": "dependency-check",
"version": "7.1.0",
"informationUri": "https://jeremylong.github.io/DependencyCheck/",
"rules": [
{
"id": "CVE-2022-24823",
"shortDescription": {
"text": "Medium severity - CVE-2022-24823 Exposure of Resource to Wrong Sphere vulnerability in pkg:maven/io.netty/netty-transport@4.1.76.Final"
},
"fullDescription": {
"text": "Netty is an open-source, asynchronous event-driven network application framework. The package `io.netty:netty-codec-http` prior to version 4.1.77.Final contains an insufficient fix for CVE-2021-21290. When Netty's multipart decoders are used local information disclosure can occur via the local system temporary directory if temporary storing uploads on the disk is enabled. This only impacts applications running on Java version 6 and lower. Additionally, this vulnerability impacts code running on Unix-like systems, and very old versions of Mac OSX and Windows as they all share the system temporary directory between all users. Version 4.1.77.Final contains a patch for this vulnerability. As a workaround, specify one's own `java.io.tmpdir` when starting the JVM or use DefaultHttpDataFactory.setBaseDir(...) to set the directory to something that is only readable by the current user."
},
"help": {
"text": "",
"markdown": "For more information see [CVE-2022-24823](http://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2022-24823).\n\n\nIf this is a false positive - consider using the HTML report to generate a suppression file. For more information see [How dependency-check works](https://jeremylong.github.io/DependencyCheck/general/internals.html), [How to read the HTML report](https://jeremylong.github.io/DependencyCheck/general/thereport.html), and [Suppressing false positives](https://jeremylong.github.io/DependencyCheck/general/suppression.html)."
},
"properties": {
"cvssv2_score": 1.9,
"cvssv2_accessVector": "LOCAL",
"cvssv2_accessComplexity": "MEDIUM",
"cvssv2_authenticationr": "NONE",
"cvssv2_confidentialImpact": "$enc.json($rule.cvssv2ConfidentialityImpact)",
"cvssv2_integrityImpact": "NONE",
"cvssv2_availabilityImpact": "NONE",
"cvssv2_severity": "LOW",
"cvssv3_baseScore": 5.5,
"cvssv3_attackVector": "LOCAL",
"cvssv3_attackComplexity": "LOW",
"cvssv3_privilegesRequired": "LOW",
"cvssv3_userInteraction": "NONE",
"cvssv3_scope": "UNCHANGED",
"cvssv3_confidentialityImpact": "HIGH",
"cvssv3_integrityImpact": "NONE",
"cvssv3_availabilityImpact": "NONE",
"cvssv3_baseSeverity": "MEDIUM",
"cvssv3_exploitabilityScore": "1.8",
"cvssv3_impactScore": "3.6",
"cvssv3_version": "3.1",
"source": "NVD"
},
"defaultConfiguration": {
"level": "error"
}
}
],
"properties": {
"disclaimer": "Dependency-Check is an open source tool performing a best effort analysis of 3rd party dependencies; false positives and false negatives may exist in the analysis performed by the tool. Use of the tool and the reporting provided constitutes acceptance for use in an AS IS condition, and there are NO warranties, implied or otherwise, with regard to the analysis or its use. Any use of the tool and the reporting provided is at the user's risk. In no event shall the copyright holder or OWASP be held liable for any damages whatsoever arising out of or in connection with the use of this tool, the analysis performed, or the resulting report.",
"nvd": "This report contains data retrieved from the National Vulnerability Database: http://nvd.nist.gov",
"npm": "This report may contain data retrieved from the NPM Public Advisories: https://www.npmjs.com/advisories",
"retirejs": "This report may contain data retrieved from the RetireJS community: https://retirejs.github.io/retire.js/",
"ossindex": "This report may contain data retrieved from the Sonatype OSS Index: https://ossindex.sonatype.org",
"NVD CVE Checked": "2022-05-25T15:58:07",
"NVD CVE Modified": "2022-05-25T14:00:01",
"VersionCheckOn": "2022-05-25T15:58:07"
}
}
}Tool information
|
|
Testing script results from test/test.sh Test script outputs from test/test.sh
Convert SARIF file ./test/fixtures/codeql.sarif Results
Suppressed ResultsNothing here. Rules informationRules detailsDetails{
"driver": {
"name": "CodeQL command-line toolchain",
"organization": "GitHub",
"semanticVersion": "2.2.4",
"rules": [
{
"id": "js/xss",
"name": "js/xss",
"shortDescription": {
"text": "Client-side cross-site scripting"
},
"fullDescription": {
"text": "Writing user input directly to the DOM allows for a cross-site scripting vulnerability."
},
"defaultConfiguration": {
"level": "error"
},
"properties": {
"tags": [
"security",
"external/cwe/cwe-079",
"external/cwe/cwe-116"
],
"kind": "path-problem",
"precision": "high",
"name": "Client-side cross-site scripting",
"description": "Writing user input directly to the DOM allows for\n a cross-site scripting vulnerability.",
"id": "js/xss",
"problem.severity": "error"
}
}
]
}
}Tool information
(node:30) [DEP0040] DeprecationWarning: The Results
Suppressed ResultsNothing here. Rules informationRules detailsDetails{
"driver": {
"name": "dependency-check",
"version": "7.1.0",
"informationUri": "https://jeremylong.github.io/DependencyCheck/",
"rules": [
{
"id": "CVE-2022-24823",
"shortDescription": {
"text": "Medium severity - CVE-2022-24823 Exposure of Resource to Wrong Sphere vulnerability in pkg:maven/io.netty/netty-transport@4.1.76.Final"
},
"fullDescription": {
"text": "Netty is an open-source, asynchronous event-driven network application framework. The package `io.netty:netty-codec-http` prior to version 4.1.77.Final contains an insufficient fix for CVE-2021-21290. When Netty's multipart decoders are used local information disclosure can occur via the local system temporary directory if temporary storing uploads on the disk is enabled. This only impacts applications running on Java version 6 and lower. Additionally, this vulnerability impacts code running on Unix-like systems, and very old versions of Mac OSX and Windows as they all share the system temporary directory between all users. Version 4.1.77.Final contains a patch for this vulnerability. As a workaround, specify one's own `java.io.tmpdir` when starting the JVM or use DefaultHttpDataFactory.setBaseDir(...) to set the directory to something that is only readable by the current user."
},
"help": {
"text": "",
"markdown": "For more information see [CVE-2022-24823](http://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2022-24823).\n\n\nIf this is a false positive - consider using the HTML report to generate a suppression file. For more information see [How dependency-check works](https://jeremylong.github.io/DependencyCheck/general/internals.html), [How to read the HTML report](https://jeremylong.github.io/DependencyCheck/general/thereport.html), and [Suppressing false positives](https://jeremylong.github.io/DependencyCheck/general/suppression.html)."
},
"properties": {
"cvssv2_score": 1.9,
"cvssv2_accessVector": "LOCAL",
"cvssv2_accessComplexity": "MEDIUM",
"cvssv2_authenticationr": "NONE",
"cvssv2_confidentialImpact": "$enc.json($rule.cvssv2ConfidentialityImpact)",
"cvssv2_integrityImpact": "NONE",
"cvssv2_availabilityImpact": "NONE",
"cvssv2_severity": "LOW",
"cvssv3_baseScore": 5.5,
"cvssv3_attackVector": "LOCAL",
"cvssv3_attackComplexity": "LOW",
"cvssv3_privilegesRequired": "LOW",
"cvssv3_userInteraction": "NONE",
"cvssv3_scope": "UNCHANGED",
"cvssv3_confidentialityImpact": "HIGH",
"cvssv3_integrityImpact": "NONE",
"cvssv3_availabilityImpact": "NONE",
"cvssv3_baseSeverity": "MEDIUM",
"cvssv3_exploitabilityScore": "1.8",
"cvssv3_impactScore": "3.6",
"cvssv3_version": "3.1",
"source": "NVD"
},
"defaultConfiguration": {
"level": "error"
}
}
],
"properties": {
"disclaimer": "Dependency-Check is an open source tool performing a best effort analysis of 3rd party dependencies; false positives and false negatives may exist in the analysis performed by the tool. Use of the tool and the reporting provided constitutes acceptance for use in an AS IS condition, and there are NO warranties, implied or otherwise, with regard to the analysis or its use. Any use of the tool and the reporting provided is at the user's risk. In no event shall the copyright holder or OWASP be held liable for any damages whatsoever arising out of or in connection with the use of this tool, the analysis performed, or the resulting report.",
"nvd": "This report contains data retrieved from the National Vulnerability Database: http://nvd.nist.gov",
"npm": "This report may contain data retrieved from the NPM Public Advisories: https://www.npmjs.com/advisories",
"retirejs": "This report may contain data retrieved from the RetireJS community: https://retirejs.github.io/retire.js/",
"ossindex": "This report may contain data retrieved from the Sonatype OSS Index: https://ossindex.sonatype.org",
"NVD CVE Checked": "2022-05-25T15:58:07",
"NVD CVE Modified": "2022-05-25T14:00:01",
"VersionCheckOn": "2022-05-25T15:58:07"
}
}
}Tool information
(node:32) [DEP0040] DeprecationWarning: The |
Reviewer's guide (collapsed on small PRs)Reviewer's GuideStabilizes the File-Level Changes
Assessment against linked issues
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
There was a problem hiding this comment.
Hey - I've found 1 issue, and left some high level feedback:
- In the Bats tests, the
bash -c "... '$SCRIPT_PATH'"pattern will pass a literal$SCRIPT_PATHinto the subshell rather than the expanded path; consider removing the inner single quotes and quoting the variable itself (e.g.,bash -c "unset GITHUB_TOKEN && \"$SCRIPT_PATH\"") or exportingSCRIPT_PATHand referencing it without quotes inside thebash -cstring so the script actually runs as intended.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- In the Bats tests, the `bash -c "... '$SCRIPT_PATH'"` pattern will pass a literal `$SCRIPT_PATH` into the subshell rather than the expanded path; consider removing the inner single quotes and quoting the variable itself (e.g., `bash -c "unset GITHUB_TOKEN && \"$SCRIPT_PATH\""`) or exporting `SCRIPT_PATH` and referencing it without quotes inside the `bash -c` string so the script actually runs as intended.
## Individual Comments
### Comment 1
<location path="test/unit/test_review_trivyignore.bats" line_range="31-35" />
<code_context>
- # Run without GITHUB_TOKEN
- run python "$SCRIPT_PATH"
+ # Run without GITHUB_TOKEN, even if CI injects it into the environment
+ run bash -c "unset GITHUB_TOKEN && '$SCRIPT_PATH'"
[ "$status" -eq 1 ]
[[ $output =~ "GITHUB_TOKEN environment variable not set" ]]
</code_context>
<issue_to_address>
**suggestion (testing):** Consider also unsetting `GITHUB_REPOSITORY` in the "no GITHUB_TOKEN" test to fully isolate the environment
That way the test only validates the "missing token" behavior and won’t be affected by any current or future CI defaults. For example:
```bash
run bash -c "unset GITHUB_TOKEN GITHUB_REPOSITORY && '$SCRIPT_PATH'"
```
```suggestion
# Run without GITHUB_TOKEN (and without GITHUB_REPOSITORY), even if CI injects them into the environment
run bash -c "unset GITHUB_TOKEN GITHUB_REPOSITORY && '$SCRIPT_PATH'"
[ "$status" -eq 1 ]
[[ $output =~ "GITHUB_TOKEN environment variable not set" ]]
}
```
</issue_to_address>Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
| # Run without GITHUB_TOKEN, even if CI injects it into the environment | ||
| run bash -c "unset GITHUB_TOKEN && '$SCRIPT_PATH'" | ||
| [ "$status" -eq 1 ] | ||
| [[ $output =~ "GITHUB_TOKEN environment variable not set" ]] | ||
| } |
There was a problem hiding this comment.
suggestion (testing): Consider also unsetting GITHUB_REPOSITORY in the "no GITHUB_TOKEN" test to fully isolate the environment
That way the test only validates the "missing token" behavior and won’t be affected by any current or future CI defaults. For example:
run bash -c "unset GITHUB_TOKEN GITHUB_REPOSITORY && '$SCRIPT_PATH'"| # Run without GITHUB_TOKEN, even if CI injects it into the environment | |
| run bash -c "unset GITHUB_TOKEN && '$SCRIPT_PATH'" | |
| [ "$status" -eq 1 ] | |
| [[ $output =~ "GITHUB_TOKEN environment variable not set" ]] | |
| } | |
| # Run without GITHUB_TOKEN (and without GITHUB_REPOSITORY), even if CI injects them into the environment | |
| run bash -c "unset GITHUB_TOKEN GITHUB_REPOSITORY && '$SCRIPT_PATH'" | |
| [ "$status" -eq 1 ] | |
| [[ $output =~ "GITHUB_TOKEN environment variable not set" ]] | |
| } |
The CI bash test suite was failing because the
review_trivyignoreunit tests depended on environment assumptions that do not hold reliably in GitHub Actions. This change tightens those tests so they exercise the intended failure paths consistently.Test invocation
review_trivyignore.pythroughpython.pythonalias being present.Environment isolation
GITHUB_TOKENin the negative-path test.GITHUB_REPOSITORYunset behavior so both tests validate missing-env handling instead of inheriting CI-provided variables.Scope
test/unit/test_review_trivyignore.bats.Summary by Sourcery
Stabilize security-related test execution and improve CI robustness.
CI:
production.cloudfront.docker.cominstead of the deprecatedproduction.cloudflare.docker.com.Tests:
review_trivyignoreBats tests to invoke the script via its shebang and explicitly unset CI-injected environment variables for deterministic failure-path coverage.