[ty] Install more dependencies before benchmarking hydra-zen#18757
[ty] Install more dependencies before benchmarking hydra-zen#18757AlexWaygood wants to merge 1 commit intomainfrom
Conversation
|
You might want to update mypy primer too 😅 Hydra is already one of the slowest to install. I'm curious by how much performance degrades due to adding the new dependencies. |
I will! |
|
| commit: "dd2b50a9614c6f8c46c5866f283c8f7e7a960aa8", | ||
| paths: vec![SystemPath::new("src")], | ||
| dependencies: vec!["pydantic", "beartype", "hydra-core"], | ||
| dependencies: vec!["pydantic", "beartype", "hydra-core", "pytest", "hypothesis"], |
There was a problem hiding this comment.
I can't comment but you might want to lower the number of the max expected diagnostics on line 441.
That makes me realise that we may want to enable all rules, including possibly-* rules
There was a problem hiding this comment.
That makes me realise that we may want to enable all rules, including possibly-* rules
that makes sense but I'll leave it out of this PR
There was a problem hiding this comment.
Oh yeah, that was more a note for myself
|
I realised that the benchmarks only run ty on the |
Summary
This makes the number of
unresolved-importdiagnostics on hydra-zen go down from 136 to 16. The overall number of diagnostics goes down from 703 to 588.That seems worth the cost of installing them considering that hydra-zen would likely have these imports installed if they were running ty in CI, and we want the benchmark to accurately mimic the experience users would have if they ran ty on these projects in CI. (In CI, gathering suggestions for an
unresolved-importdiagnostic would usually be a very cold path, for example, because there would usually be a very small number of diagnostics like this.)Test Plan
I:
<hydra-zen repo root>/.venvpath/to/ty/binary checkand observed that there were 136 `unresolved-import diagnosticsuv pip install pytest hypothesispath/to/ty/binary checkand observed that there were now only 16 `unresolved-import diagnostics