Conversation
jvff
force-pushed
the
limit-http-response-size
branch
from
cb2b864 to
22962d6
Compare
jvff
force-pushed
the
limit-http-response-size
branch
2 times, most recently
from
3b3e909 to
7ea2944
Compare
Allow configuring a policy with a limit to HTTP response sizes.
jvff
force-pushed
the
limit-http-response-size
branch
from
7ea2944 to
5bc3cbf
Compare
| #[error("Serialized size of the executed block exceeds limit")] | ||
| ExecutedBlockTooLarge, | ||
| #[error("HTTP response exceeds the size limit of {limit} bytes, having at least {size} bytes")] | ||
| HttpResponseSizeLimitExceeded { limit: u64, size: u64 }, |
There was a problem hiding this comment.
Should we make this a limit for all types of oracle responses?
There was a problem hiding this comment.
Oh right, I'll open an issue to limit the oracle response size as well.
deuszx
left a comment
deuszx
left a comment
There was a problem hiding this comment.
Let's not panic if the header is misformatted.
Prepare to enforce a size limit to the response.
It's no longer needed.
Prepare to abort early if too many bytes are received.
jvff
force-pushed
the
limit-http-response-size
branch
from
5bc3cbf to
791e1be
Compare
| let total_header_size = headers | ||
| .iter() | ||
| .map(|header| (header.name.as_bytes().len() + header.value.len()) as u64) | ||
| .sum(); |
There was a problem hiding this comment.
I wonder - could we collect the sizes when we map response.headers() already? we have the data there (name, value, etc.).
There was a problem hiding this comment.
We could, but I think doing it separately is cleaner.
There was a problem hiding this comment.
Premature optimization is the root of all evil ;)
I'll insist. It did not panic if the header is misformatted. It would have panicked if the http crate had a bug that would return https://docs.rs/http/0.2.12/src/http/header/map.rs.html#3039 Note that |
jvff
force-pushed
the
limit-http-response-size
branch
from
791e1be to
7af077e
Compare
Return an error as soon as it is detected that the size will exceed the limit.
jvff
force-pushed
the
limit-http-response-size
branch
from
7af077e to
6f3ffe2
Compare
3 participants
Motivation
Applications could cause excessive bandwidth costs to validators by performing HTTP requests to obtain large amounts of data as fast as possible.
Proposal
Ensure that HTTP responses are limited in size to a value agreed among the validators.
Test Plan
Wrote some unit tests in a separate branch, using the code from #3509:
https://github.com/jvff/linera-protocol-archive/blob/test-http-response-limits-p1/examples/how-to/perform-http-requests/tests/response_size_tests.rs
Release Plan
devnetand release a newSDK soon, because this contains changes to the consensus critical
ResourceControlPolicy.Links