Skip to content

[red-knot] Run py-fuzzer in CI to check for new panics #17719

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
AlexWaygood merged 3 commits into from
Apr 29, 2025
Merged

[red-knot] Run py-fuzzer in CI to check for new panics #17719

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
36f2766
Upload red-knot binaries in CI on completion of linux tests
AlexWaygood Apr 29, 2025
426ca70
[red-knot] Run py-fuzzer in CI to check for new panics
AlexWaygood Apr 29, 2025
8c563cc
Update .github/workflows/ci.yaml
AlexWaygood Apr 29, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 51 additions & 0 deletions .github/workflows/ci.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -276,6 +276,10 @@ jobs:
with:
name: ruff
path: target/debug/ruff
- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: red_knot
path: target/debug/red_knot

cargo-test-linux-release:
name: "cargo test (linux, release)"
Expand Down Expand Up @@ -632,6 +636,53 @@ jobs:
name: ecosystem-result
path: ecosystem-result

fuzz-redknot:
name: "Fuzz for new red-knot panics"
runs-on: depot-ubuntu-22.04-16
needs:
- cargo-test-linux
- determine_changes
# Only runs on pull requests, since that is the only we way we can find the base version for comparison.
if: ${{ !contains(github.event.pull_request.labels.*.name, 'no-test') && github.event_name == 'pull_request' && needs.determine_changes.outputs.red_knot == 'true' }}
Comment on lines +645 to +646
Copy link
Member

@dhruvmanila dhruvmanila Apr 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, and it's most useful to run it on a PR. We also don't run the py-fuzzer on parser on main.

timeout-minutes: 20
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- uses: actions/download-artifact@95815c38cf2ff2164869cbab79da8d1f422bc89e # v4.2.1
name: Download new red-knot binary
id: redknot-new
with:
name: red_knot
path: target/debug
- uses: dawidd6/action-download-artifact@20319c5641d495c8a52e688b7dc5fada6c3a9fbc # v8
name: Download baseline red-knot binary
with:
name: red_knot
branch: ${{ github.event.pull_request.base.ref }}
workflow: "ci.yaml"
check_artifacts: true
- uses: astral-sh/setup-uv@d4b2f3b6ecc6e67c4457f6d3e41ec42d3d0fcb86 # v5.4.2
- name: Fuzz
env:
FORCE_COLOR: 1
NEW_REDKNOT: ${{ steps.redknot-new.outputs.download-path }}
run: |
# Make executable, since artifact download doesn't preserve this
chmod +x "${PWD}/red_knot" "${NEW_REDKNOT}/red_knot"

(
uvx \
--python="${PYTHON_VERSION}" \
--from=./python/py-fuzzer \
fuzz \
--test-executable="${NEW_REDKNOT}/red_knot" \
--baseline-executable="${PWD}/red_knot" \
--only-new-bugs \
--bin=red_knot \
0-500
)

cargo-shear:
name: "cargo shear"
runs-on: ubuntu-latest
Expand Down
Loading