-
-
Notifications
You must be signed in to change notification settings - Fork 131
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Make ListBody<T>
lazily deserializable
#2310
Comments
How would we even do lazy deserialzation? Or do you simply mean to defer the deserialzation to when it is actually looked at? |
its possible in theory but i doubt serde supports it and im not sure the complexity this brings is worth the performance gain, in some cases it may even lead to worse performance when people repeatedly deserialize the stream's items |
Hyper streams the response body by default so we just need two streaming adapters:
Serde does not support streaming adapters but it's trivial to do so ourselves. We deserialize upon hitting a comma if our counter of braces and brackets are zero (i.e. the comma separates elements in the top level array).
The stream ( |
Would this not cause potentially more memory to be used since the string has to valid for as long as the deserilizer is? |
Instead of
Response::models
being anasync fn -> Result<Vec<T>, E>
it should return a type implementingStream<Item = Result<T, E>>
andIntoFuture<Output = Result<Vec<T>, E>>
. This allows for lazy deserializing the response and potentially skip the (often unused)Vec
buffer.Inspiration: https://github.com/abdolence/axum-streams-rs
The text was updated successfully, but these errors were encountered: