Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Limit http solver response size #158

Merged
merged 2 commits into from
Apr 20, 2022
Merged

Limit http solver response size #158

merged 2 commits into from
Apr 20, 2022

Conversation

vkgnosis
Copy link
Contributor

Part of #151 .

A large response could exhaust our memory. We protect against this by limiting the size of the response to 10 MB.

Test Plan

new unit test

@vkgnosis vkgnosis requested a review from a team as a code owner April 18, 2022 12:10
@codecov-commenter
Copy link

codecov-commenter commented Apr 18, 2022

Codecov Report

Merging #158 (9b72aaa) into main (10a0cbe) will increase coverage by 0.15%.
The diff coverage is 0.00%.

❗ Current head 9b72aaa differs from pull request most recent head 33dc1bf. Consider uploading reports for the commit 33dc1bf to get more accurate results

@@            Coverage Diff             @@
##             main     #158      +/-   ##
==========================================
+ Coverage   64.90%   65.05%   +0.15%     
==========================================
  Files         184      184              
  Lines       38321    38228      -93     
==========================================
  Hits        24871    24871              
+ Misses      13450    13357      -93     

Comment on lines +7 to +11
pub async fn response_body_with_size_limit(
response: &mut Response,
limit: usize,
) -> Result<Vec<u8>> {
let mut bytes = Vec::new();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would make sense to offer a high performance API which allows you to pass in the buffer to avoid copying the data out of the function.
Then response_body_with_size_limit() could be a simple wrapper function around that more performant API. I believe if response_body_with_size_limit() is tagged with #[inline] we should have a simple API while still avoiding unnecessary copies.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is no copying going on when returning the data out of the function. The data lives on the heap and Vec stores a pointer to it and the size so it's like returning two integers. For high performance passing in the buffer would still be advantageous because it allows you to reuse the allocation in the future but I don't want to make this more complicated before specifically wanting to use that.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, of course. You're right. 🤦‍♂️
It's all good then. 👍

@vkgnosis vkgnosis enabled auto-merge (squash) April 20, 2022 08:12
@vkgnosis vkgnosis merged commit 1eb2ea3 into main Apr 20, 2022
@vkgnosis vkgnosis deleted the http-size-limit branch April 20, 2022 08:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants