-
Notifications
You must be signed in to change notification settings - Fork 38
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Run formatters in parallel #17
Conversation
All the formatters are run in parallel using rayon's powerful par_iter which uses a thread pool under the hood.
That was easy! Got to love the rust library ecosystem. The threads seem to default on the number of CPUs which is fine. We can always tune this later with some user feedback. At some point, we'll need to forward the exit status to the parent process exit status if any of those fail, although it can happen at a later stage. |
Thank you for helping us to implement this. It wasn't easy for me to choose the library (threadpool, rayon, crossbeam) for this issue. I've been researching which library to use while working on another issue. Have you tried crossbeam? In my research, crossbeam provides various queues that allow pushing and popping items from several threads without a mutex. The code is LGTM. And I agree with what @zimbatm said, we can change it later if we need to. Regarding the error, I haven't think of displaying multiple errors when the |
Yes, and it's a great library. I think for this use case
I will try to come up with something 😄 (probably a different PR) |
Oh, this is great, I didn't notice it. We should stick with
Yes, I agree with this.
Yes, agree that it should be in a different PR. |
Since we discussed this in https://git.numtide.com/numtide/treefmt/pulls/14#issuecomment-609 It doesn't really matter. Reviewed-on: https://git.numtide.com/numtide/treefmt/pulls/17 Co-authored-by: zimbatm <[email protected]> Co-committed-by: zimbatm <[email protected]>
Closes #15
All the formatters are run in parallel using
rayon
's powerfulpar_iter
which uses a thread pool under the hood.Should I setup a CLI flag/env var to specify the number of threads to use?
Currently errors are not reported, but I can implement this in this PR. Do we have a preferred way to report multiple errors? I'm not sure that's something
anyhow
exposes, and we might want to format the errors ourselves anyway.