[ENG-351] Tokio git patch

[brxken128]

This PR forces all uses of tokio to be from the workspace, and the workspace uses tokio v1.24.2 with the commits listed in this branch.

The Linear issue contains more information, but I have tested that things build on Linux.

Windows, MacOS, Android and iOS have yet to be tested, but probably should be before this is merged.

[linear]

ENG-351 Use tokio as a git dependency

Currently, tokio uses a 16KB internal buffer size for IO operations - this means crypto performance heavily regressed in ENG-348.

I have opened a PR upstream, but they may not accept it and we'd have to wait for the next release.

Until this is resolved, I think we should use a git dependency of tokio that increases the IO buffer to 2MiB. In my testing, 2MiB offered the best performance/memory usage balance.

We will have to ensure that things don't break on other OSes, especially on Windows due to the absurdly low stack size. I don't think it's ever stack-allocated, but we should still double check.

This will result in better performance for all tokio::fs::File read/writes, as read, read_exact, etc all use this 16KB buffer by default I believe. It may also not be limited just to File.

The difference between the 16KB buffer and 2MiB is astronomical. Encrypting a 10GiB took 24 seconds with 16KB, and took 8.7 seconds with 2MiB.

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

2 Ignored Deployments

Name	Status	Preview	Comments	Updated
spacedrive-landing	⬜️ Ignored (Inspect)			Jan 25, 2023 at 3:01PM (UTC)
spacedrive-web	⬜️ Ignored (Inspect)			Jan 25, 2023 at 3:01PM (UTC)