Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Parse entire HTTP chunk size #396 #528

Merged
merged 3 commits into from
Mar 31, 2024
Merged

Parse entire HTTP chunk size #396 #528

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
91a2b45
Parse entire HTTP chunk size #396
SkyTrix Mar 28, 2024
8c1c220
Add reference to Integer.MAX_VALUE in the error message
SkyTrix Mar 28, 2024
c0d4775
Fix formatting
SkyTrix Mar 28, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 6 additions & 3 deletions http-core/src/main/scala/org/apache/pekko/http/impl/engine/parsing/HttpMessageParser.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -307,17 +307,20 @@ private[http] trait HttpMessageParser[Output >: MessageOutput <: ParserOutput] {
s"HTTP chunk extension length exceeds configured limit of ${settings.maxChunkExtLength} characters")

@tailrec def parseSize(cursor: Int, size: Long): StateResult =
if (size <= settings.maxChunkSize) {
if (size <= Int.MaxValue) {
Copy link
Member

@samueleresca samueleresca Mar 29, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My personal preference (it feels more clear) would be to have:

if (size > Int.MaxValue) {
// early exit here with failEntityStream
}

// everything else

But I don't feel is binding.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

guard clauses. 👍🏻

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm trying to keep the structure consistent with the rest of the file. I also think the above suggestion would be incorrect in the sense that everything else would still be executed after calling failEntityStream? An else block would still be required.

byteChar(input, cursor) match {
case c if CharacterClasses.HEXDIG(c) => parseSize(cursor + 1, size * 16 + CharUtils.hexValue(c))
case ';' if cursor > offset => parseChunkExtensions(size.toInt, cursor + 1)()
case c if size > settings.maxChunkSize =>
failEntityStream(
s"HTTP chunk of $size bytes exceeds the configured limit of ${settings.maxChunkSize} bytes")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see now, It was try decoding how big the chunk size is, and it will stop parsing how if the size already exceed MaxValue.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about HTTP chunk size:$size exceeds the configured limit of ${settings.maxChunkSize} bytes

case ';' if cursor > offset => parseChunkExtensions(size.toInt, cursor + 1)()
case '\r' if cursor > offset && byteChar(input, cursor + 1) == '\n' =>
parseChunkBody(size.toInt, "", cursor + 2)
case '\n' if cursor > offset => parseChunkBody(size.toInt, "", cursor + 1)
case c if CharacterClasses.WSP(c) => parseSize(cursor + 1, size) // illegal according to the spec but can happen, see issue #1812
case c => failEntityStream(s"Illegal character '${escape(c)}' in chunk start")
}
} else failEntityStream(s"HTTP chunk size exceeds the configured limit of ${settings.maxChunkSize} bytes")
} else failEntityStream(s"HTTP chunk size exceeds Integer.MAX_VALUE (${Int.MaxValue}) bytes")
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that this does not start parsing the actual chunk body. It only reads the size value that's written in the chunk header and bails if that's larger than Int.MaxValue. It will never attempt to parse anything larger than settings.maxChunkSize.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did another around of check locally,it's ok.

Copy link
Member

@He-Pin He-Pin Mar 28, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In netty, when oversize, it invoke a protect method handleOversizedMessage, maybe we can introduce something like that too.
see: io.netty.handler.codec.http.HttpObjectAggregator#handleOversizedMessage


try parseSize(offset, 0)
catch {
Expand Down
15 changes: 13 additions & 2 deletions http-core/src/test/scala/org/apache/pekko/http/impl/engine/parsing/RequestParserSpec.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -531,13 +531,24 @@ abstract class RequestParserSpec(mode: String, newLine: String) extends AnyFreeS
closeAfterResponseCompletion shouldEqual Seq(false)
}

"too-large chunk size" in new Test {
"too-large chunk size (> Int.MaxValue)" in new Test {
Seq(
start,
"""1a2b3c4d5e
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"1a2b3c4d5e"

|""") should generalMultiParseTo(
Right(baseRequest),
Left(EntityStreamError(ErrorInfo("HTTP chunk size exceeds the configured limit of 1048576 bytes"))))
Left(EntityStreamError(ErrorInfo("HTTP chunk size exceeds Integer.MAX_VALUE (2147483647) bytes"))))
closeAfterResponseCompletion shouldEqual Seq(false)
}

"too-large chunk size" in new Test {
Seq(
start,
"""400000
|""") should generalMultiParseTo(
Comment on lines +547 to +548
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"""400000
|""") should generalMultiParseTo(
"400000") should generalMultiParseTo(

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found that the trailing newline is significant in this and many other tests in the file.

Right(baseRequest),
Left(
EntityStreamError(ErrorInfo("HTTP chunk of 4194304 bytes exceeds the configured limit of 1048576 bytes"))))
closeAfterResponseCompletion shouldEqual Seq(false)
}

Expand Down
Loading