Skip to content

Conversation

@AlexWaygood
Copy link
Member

Summary

Now that we've moved the corpus tests to the ty_python_semantic crate, running cargo test -p ty_python_semantic takes an annoying amount of time locally. (This also hasn't been helped by the fact that we run a lot more corpus tests after #18531.) We can speed things up quite a bit by splitting the corpus tests up more, so that they run in parallel.

Test Plan

On main locally:

~/dev/ruff (main)⚡ % cargo test -p ty_python_semantic --test corpus              
   Compiling ty_python_semantic v0.0.0 (/Users/alexw/dev/ruff/crates/ty_python_semantic)
    Finished `test` profile [unoptimized + debuginfo] target(s) in 0.81s
     Running tests/corpus.rs (target/debug/deps/corpus-1d987a322669ccb0)

running 6 tests
test corpus_no_panic ... ok
test linter_stubs_no_panic ... ok
test parser_no_panic ... ok
test linter_af_no_panic ... ok
test linter_gz_no_panic ... ok
test typeshed_no_panic ... ok

test result: ok. 6 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 8.63s

On this PR branch locally:

~/dev/ruff (alex/split-corpus-tests-more)⚡ % cargo test -p ty_python_semantic --test corpus
    Finished `test` profile [unoptimized + debuginfo] target(s) in 0.08s
     Running tests/corpus.rs (target/debug/deps/corpus-1d987a322669ccb0)

running 12 tests
test corpus_no_panic ... ok
test linter_no_panic::_a_e_expects ... ok
test linter_no_panic::_g_o_expects ... ok
test linter_stubs_no_panic ... ok
test parser_no_panic ... ok
test typeshed_no_panic::_f_k_expects ... ok
test typeshed_no_panic::_l_p_expects ... ok
test typeshed_no_panic::_a_e_expects ... ok
test linter_no_panic::_q_z_expects ... ok
test typeshed_no_panic::_q_z_expects ... ok
test linter_no_panic::_p_expects ... ok
test linter_no_panic::_f_expects ... ok

test result: ok. 12 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 3.77s

I.e., a speedup of around 5s

@AlexWaygood AlexWaygood added testing Related to testing Ruff itself ty Multi-file analysis & type inference labels Jun 16, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Jun 16, 2025

mypy_primer results

No ecosystem changes detected ✅

@MichaReiser
Copy link
Member

This could make the tests on ci (or systems with fewer cores in general) somewhat slower because we now have to infer typeshed more often but I think this change still makes sense.

@AlexWaygood AlexWaygood enabled auto-merge (squash) June 16, 2025 17:36
@AlexWaygood AlexWaygood merged commit 5e57e46 into main Jun 16, 2025
34 checks passed
@AlexWaygood AlexWaygood deleted the alex/split-corpus-tests-more branch June 16, 2025 17:38
dcreager added a commit that referenced this pull request Jun 16, 2025
* main: (38 commits)
  [`pyupgrade`] Suppress `UP008` diagnostic if `super` symbol is not builtin (#18688)
  [pylint] Fix `PLW0128` to check assignment targets in square brackets and after asterisks (#18665)
  [`refurb`] Make the fix for `FURB163` unsafe for `log2`, `log10`, `*args`, and deleted comments (#18645)
  [ty] allow `T: Never` as subtype of `Never` (#18687)
  [ty] Use more parallelism when running corpus tests (#18711)
  [ty] Support `dataclasses.KW_ONLY` (#18677)
  [`ruff`] Check for non-context-manager use of `pytest.raises`, `pytest.warns`, and `pytest.deprecated_call` (`RUF061`) (#17368)
  Add syntax error when conversion flag does not immediately follow exclamation mark (#18706)
  [`flake8-pyi`] Fix `custom-typevar-for-self` with string annotations (`PYI019`) (#18311)
  Drop confusing second `*` from glob pattern example (#18709)
  [ty] Stabilize completions (#18650)
  [ty] Correctly label typeshed-sync PRs (#18702)
  Update Rust crate memchr to v2.7.5 (#18696)
  Update dependency react-resizable-panels to v3.0.3 (#18691)
  Update Rust crate clap to v4.5.40 (#18692)
  Update Rust crate libcst to v1.8.2 (#18695)
  Update Rust crate jiff to v0.2.15 (#18693)
  Update Rust crate libc to v0.2.173 (#18694)
  Update Rust crate syn to v2.0.103 (#18698)
  Update Rust crate toml to v0.8.23 (#18699)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

testing Related to testing Ruff itself ty Multi-file analysis & type inference

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants