Skip to content

[ty] Batch changes to watched paths#24045

Merged
MichaReiser merged 2 commits intomainfrom
micha/batch-watcher-paths
Mar 20, 2026
Merged

[ty] Batch changes to watched paths#24045
MichaReiser merged 2 commits intomainfrom
micha/batch-watcher-paths

Conversation

@MichaReiser
Copy link
Member

@MichaReiser MichaReiser commented Mar 19, 2026

Summary

The file_watching tests are notoriously slow on macOS. This PR improves the performance by:

  • Fixing a debouncer bug where it never flushed the change events when there's a constant stream of changes, because it incorrectly reset the "start" time in every loop iteration (It only flushed the batch if there were no changes after 3s seconds instead of forcing a flush after 3 seconds even if there are more changes)
  • Use notify's paths_mut API to batch changes to the watched paths.

Test Plan

It wasn't uncommon that the file watcher tests take close to 10s to run:

		PASS [   5.328s] ty::file_watching new_file_in_included_out_of_project_directory
        PASS [   3.844s] ty::file_watching new_files_with_explicit_included_paths
        PASS [   3.805s] ty::file_watching new_ignored_file
        PASS [   4.806s] ty::file_watching new_non_project_file
        PASS [   8.525s] ty::file_watching remove_search_path
        PASS [   3.406s] ty::file_watching rename_file
        PASS [   4.471s] ty::file_watching search_path
        PASS [   3.558s] ty::file_watching submodule_cache_invalidation_after_pyproject_created
        PASS [   3.432s] ty::file_watching submodule_cache_invalidation_created
        PASS [   3.525s] ty::file_watching submodule_cache_invalidation_created_then_deleted
        PASS [   3.479s] ty::file_watching submodule_cache_invalidation_deleted
        PASS [   1.654s] ty::file_watching unix::changed_metadata
        PASS [   1.329s] ty::file_watching unix::symlink_inside_project
        PASS [   1.701s] ty::file_watching unix::symlinked_module_search_path

Most file watching tests now complete in under 1 second:

  PASS [   1.841s] ty::file_watching change_python_version_and_platform
  PASS [   0.784s] ty::file_watching hard_links_in_project
  PASS [   0.771s] ty::file_watching new_file
  PASS [   1.606s] ty::file_watching remove_search_path
  PASS [   0.841s] ty::file_watching rename_file
  PASS [   0.789s] ty::file_watching search_path
  PASS [   1.103s] ty::file_watching submodule_cache_invalidation_after_pyproject_created
  PASS [   0.989s] ty::file_watching submodule_cache_invalidation_created
  PASS [   0.822s] ty::file_watching unix::changed_metadata 
  PASS [   0.820s] ty::file_watching unix::symlink_inside_project

@MichaReiser MichaReiser added testing Related to testing Ruff itself ty Multi-file analysis & type inference labels Mar 19, 2026
@astral-sh-bot
Copy link

astral-sh-bot bot commented Mar 19, 2026

Typing conformance results

No changes detected ✅

Current numbers
The percentage of diagnostics emitted that were expected errors held steady at 85.29%. The percentage of expected errors that received a diagnostic held steady at 78.13%. The number of fully passing files held steady at 64/132.

@astral-sh-bot
Copy link

astral-sh-bot bot commented Mar 19, 2026

Memory usage report

Memory usage unchanged ✅

@astral-sh-bot
Copy link

astral-sh-bot bot commented Mar 19, 2026

ecosystem-analyzer results

No diagnostic changes detected ✅

Full report with detailed diff (timing results)

@MichaReiser MichaReiser marked this pull request as ready for review March 19, 2026 09:26
@astral-sh-bot astral-sh-bot bot requested a review from BurntSushi March 19, 2026 09:26
@MichaReiser MichaReiser reopened this Mar 19, 2026

if let Err(error) = watcher_paths.commit() {
tracing::warn!(
"Failed to apply file watcher updates: {error}. You have to restart Ruff after making changes to watched files or you might see stale results."
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is in ty_project -- should it mention Ruff here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nope. I wanted to blame codex, but the warning message that I added when I first implemented this already mentioned Ruff.

@MichaReiser MichaReiser enabled auto-merge (squash) March 20, 2026 08:02
@MichaReiser MichaReiser merged commit a213a6e into main Mar 20, 2026
48 checks passed
@MichaReiser MichaReiser deleted the micha/batch-watcher-paths branch March 20, 2026 08:03
carljm added a commit that referenced this pull request Mar 25, 2026
* main:
  [`flake8-bandit`] Check tuple arguments for partial paths in `S607` (#24080)
  [ty] Update Salsa (#24081)
  Update Rust toolchain to 1.94 and MSRV to 1.92 (#24076)
  [ty] Move ruffen-docs formatting config to a `ruff.toml` config file (#24074)
  [ty] `reveal_type` diagnostics in unreachable code (#24070)
  [ty] Improve keyword argument narrowing for nested dictionaries (#24010)
  [ty] Preserve blank lines between comments and imports in add-import action (#24066)
  [ty] Add diagnostic hint for invalid assignments involving invariant generics (#24032)
  Clarify `extend-ignore` and `extend-select` settings documentation (#24064)
  [ty] Batch changes to watched paths (#24045)
  replace deprecated `std::f64::EPSILON` with `f64::EPSILON` (#24067)
  [ty] Fix untracked reads in Salsa queries that can lead to backdating panics (#24051)
  [ty] Unions/intersections of gradual types should be assignable to `Never` (#24056)
  Fix incorrect path for ty_python_semantic in fuzzer (#24052)
  Bump 0.15.7 (#24049)
  [ty] ecosystem-analyzer: Fail on newly panicking projects (#24043)
  Don't show noqa hover for non-Python documents (#24040)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

testing Related to testing Ruff itself ty Multi-file analysis & type inference

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants