Improve memory use and performance of multipart parsing #745

quinnj · 2021-08-31T06:47:25Z

Memory usage and performance noted in
https://discourse.julialang.org/t/http-parse-multipart-form-does-a-lot-of-memory-allocations/66661.
This uses some of the same regex tricks we use in the other http parsing
code that operates on SubStrings and avoids excessive allocations. We
also streamline things to avoid allocating when comparing byte buffers
and the unnecessary allocated arrays of the content disposition
processing.

On my machine for current master, I see 35.022 μs (866 allocations: 70.27 KiB) for parsing a simple multipart request from the tests, and
with this PR, I see 3.603 μs (42 allocations: 2.55 KiB).

Memory usage and performance noted in https://discourse.julialang.org/t/http-parse-multipart-form-does-a-lot-of-memory-allocations/66661. This uses some of the same regex tricks we use in the other http parsing code that operates on SubStrings and avoids excessive allocations. We also streamline things to avoid allocating when comparing byte buffers and the unnecessary allocated arrays of the content disposition processing. On my machine for current master, I see `35.022 μs (866 allocations: 70.27 KiB)` for parsing a simple multipart request from the tests, and with this PR, I see `3.603 μs (42 allocations: 2.55 KiB)`.

KristofferC · 2021-08-31T13:48:50Z

src/parsemultipart.jl


 """
    find_multipart_boundary(bytes, boundaryDelimiter; start::Int=1)

 Find the first and last index of the next boundary delimiting a part, and if
 the discovered boundary is the terminating boundary.
 """
-function find_multipart_boundary(bytes::AbstractVector{UInt8}, boundaryDelimiter::AbstractVector{UInt8}; start::Int=1)
+@inline function find_multipart_boundary(bytes::AbstractVector{UInt8}, boundaryDelimiter::AbstractVector{UInt8}; start::Int=1)


I'm kind of amazed that inlining this function has any performance improvement.

Oh, it doesn't. This was from an iteration where I was returning a SubString instead of indices and it could avoid the allocation if it was inlined.

So should it have been removed?

mbauman · 2021-08-31T16:55:39Z

I can confirm this significantly improves the original request that was being tested/reported on Discourse — it is much larger and has an even larger benefit.

Before:

julia> @benchmark HTTP.parse_multipart_form($r)
BenchmarkTools.Trial: 26 samples with 1 evaluation.
 Range (min … max):  181.312 ms … 235.950 ms  ┊ GC (min … max): 3.11% … 3.33%
 Time  (median):     195.296 ms               ┊ GC (median):    3.16%
 Time  (mean ± σ):   197.687 ms ±  13.029 ms  ┊ GC (mean ± σ):  3.20% ± 0.16%

 Memory estimate: 584.19 MiB, allocs estimate: 4785830.

After:

julia> @benchmark HTTP.parse_multipart_form($r)
BenchmarkTools.Trial: 917 samples with 1 evaluation.
 Range (min … max):  5.131 ms …   9.564 ms  ┊ GC (min … max): 0.00% … 0.00%
 Time  (median):     5.309 ms               ┊ GC (median):    0.00%
 Time  (mean ± σ):   5.447 ms ± 448.252 μs  ┊ GC (mean ± σ):  0.00% ± 0.00%

 Memory estimate: 2.03 KiB, allocs estimate: 32.

KristofferC · 2021-08-31T17:09:25Z

src/parsemultipart.jl

 const CR_BYTE = 0x0d # \r
 const LF_BYTE = 0x0a # \n
 const DASH_BYTE = 0x2d # -
 const HTAB_BYTE = 0x09 # \t
 const SPACE_BYTE = 0x20
-const RETURN_BYTES = [CR_BYTE, LF_BYTE]
+const SEMICOLON_BYTE = UInt8(';')
+const CRLFCRLF = [CR_BYTE, LF_BYTE, CR_BYTE, LF_BYTE]


Make into a tuple (allows the compiler to know it's length etc)? I don't think it is mutated anywhere?

quinnj · 2021-08-31T17:37:27Z

Ok, I unexported closewrite (and friends) so nightly isn't broken due to Julia Base now exporting closewrite. I also fixed the websocket tests which were failing because echo.websocket.org doesnt' exist anymore. Let's see how CI fares now.

codecov-commenter · 2021-08-31T17:38:54Z

Codecov Report

Merging #745 (bbff1de) into master (7a3d9c9) will decrease coverage by 0.04%.
The diff coverage is 78.78%.

@@            Coverage Diff             @@
##           master     #745      +/-   ##
==========================================
- Coverage   76.89%   76.84%   -0.05%     
==========================================
  Files          38       38              
  Lines        2445     2514      +69     
==========================================
+ Hits         1880     1932      +52     
- Misses        565      582      +17

Impacted Files	Coverage Δ
src/HTTP.jl	`97.87% <ø> (+0.04%)`	⬆️
src/IOExtras.jl	`72.22% <0.00%> (-9.03%)`	⬇️
src/parsemultipart.jl	`90.90% <89.47%> (-3.92%)`	⬇️
src/Messages.jl	`87.94% <100.00%> (+0.17%)`	⬆️
src/TimeoutRequest.jl	`80.00% <0.00%> (-4.22%)`	⬇️
src/debug.jl	`46.42% <0.00%> (-3.58%)`	⬇️
src/ConnectionRequest.jl	`80.88% <0.00%> (-2.46%)`	⬇️
src/CookieRequest.jl	`85.29% <0.00%> (-1.81%)`	⬇️
src/Servers.jl	`85.97% <0.00%> (-0.36%)`	⬇️
src/Handlers.jl	`62.65% <0.00%> (-0.32%)`	⬇️
... and 19 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a2b467e...bbff1de. Read the comment docs.

Also clean up usages of startread, closeread, startwrite, closewrite to quality to the IOExtras module. Also fix the websocket tests that were hanging since echo.websocket.org doesn't exist anymore. I found pie socket that offers a free websocket testing server if you sign up and get an api key, so that's been added as a repository secret now. If anyone wants the api key to run the tests on their local machine, I'm happy to share.

fredrikekre · 2021-08-31T19:31:30Z

src/IOExtras.jl

-closeread(io) = nothing
+if isdefined(Base, :startwrite)
+    "$_doc"
+    Base.startwrite(io) = nothing


This is type piracy now

Yeah, we'll need to make a plan around this since Base doesn't define a fallback for closewrite. Either we add closewrite(io) = nothing to Base and we can check if it's defined here, or it's also possible we don't need the fallback anymore. I left this as-is for now since it ensures non-breaking.

Why extend the function at all? You can just keep using it as a separate function and qualify it (or the Base one).

Because then that's breaking; we have all these code examples all over the package where we tell people to write streaming servers doing closewrite(stream).

You can still export it. Downstreams users just must then decide if they want to use Base.closewrite or HTTP.closewrite.

Personally, I would strongly prefer that we remove the type piracy and just make a breaking release of HTTP.jl.

(I don't think it's actually breaking. I think the burden is on the downstream users to make sure that they properly qualify any names that they use.

But if this is considering breaking, then I'd rather just tag the breaking release so that we can get rid of the type piracy.)

… considered harmful") (#42080) I feel we are heading up against a "`using` crisis" where any new feature that is implemented by exporting a new name (either in Base or a package) becomes a breaking change. This is already happening (JuliaGPU/CUDA.jl#1097, JuliaWeb/HTTP.jl#745) and as projects get bigger and more names are exported, the likelihood of this rapidly increases. The flaw in `using Foo` is fundamental in that you cannot lexically see where a name comes from so when two packages export the same name, you are screwed. Any code that relies on `using Foo` and then using an exported name from `Foo` is vulnerable to another dependency exporting the same name. Therefore, I think we should start to strongly discourage the use of `using Foo` and only recommend `using Foo` for ephemeral work (e.g. REPL work). --------- Co-authored-by: Dilum Aluthge <[email protected]> Co-authored-by: Mason Protter <[email protected]> Co-authored-by: Max Horn <[email protected]> Co-authored-by: Matt Bauman <[email protected]> Co-authored-by: Alex Arslan <[email protected]> Co-authored-by: Ian Butterworth <[email protected]> Co-authored-by: Neven Sajko <[email protected]>

… considered harmful") (#42080) I feel we are heading up against a "`using` crisis" where any new feature that is implemented by exporting a new name (either in Base or a package) becomes a breaking change. This is already happening (JuliaGPU/CUDA.jl#1097, JuliaWeb/HTTP.jl#745) and as projects get bigger and more names are exported, the likelihood of this rapidly increases. The flaw in `using Foo` is fundamental in that you cannot lexically see where a name comes from so when two packages export the same name, you are screwed. Any code that relies on `using Foo` and then using an exported name from `Foo` is vulnerable to another dependency exporting the same name. Therefore, I think we should start to strongly discourage the use of `using Foo` and only recommend `using Foo` for ephemeral work (e.g. REPL work). --------- Co-authored-by: Dilum Aluthge <[email protected]> Co-authored-by: Mason Protter <[email protected]> Co-authored-by: Max Horn <[email protected]> Co-authored-by: Matt Bauman <[email protected]> Co-authored-by: Alex Arslan <[email protected]> Co-authored-by: Ian Butterworth <[email protected]> Co-authored-by: Neven Sajko <[email protected]> (cherry picked from commit ee09ae7)

KristofferC reviewed Aug 31, 2021

View reviewed changes

quinnj force-pushed the jq/multipart_parsing_perf branch from 4d8c65f to a52b672 Compare August 31, 2021 18:40

quinnj merged commit 0d037f6 into master Aug 31, 2021

quinnj deleted the jq/multipart_parsing_perf branch August 31, 2021 19:24

quinnj mentioned this pull request Aug 31, 2021

Set timeout limit for tests #739

Closed

fredrikekre reviewed Aug 31, 2021

View reviewed changes

KristofferC mentioned this pull request Sep 1, 2021

recommend explicit using Foo: Foo, ... in package code (was: "using considered harmful") JuliaLang/julia#42080

Merged

DilumAluthge mentioned this pull request Sep 2, 2021

[Breaking] Remove the type piracy of several Base functions #749

Closed

fredrikekre mentioned this pull request Sep 30, 2021

Feature request: parse mulitpart form data into a dict #263

Closed

giordano mentioned this pull request Nov 25, 2022

UndefVarError: closewrite not defined #969

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improve memory use and performance of multipart parsing #745

Improve memory use and performance of multipart parsing #745

quinnj commented Aug 31, 2021

KristofferC Aug 31, 2021

quinnj Aug 31, 2021

KristofferC Aug 31, 2021

mbauman commented Aug 31, 2021

KristofferC Aug 31, 2021

quinnj commented Aug 31, 2021

codecov-commenter commented Aug 31, 2021 •

edited

Loading

fredrikekre Aug 31, 2021

quinnj Aug 31, 2021

KristofferC Aug 31, 2021

quinnj Aug 31, 2021

KristofferC Sep 1, 2021

DilumAluthge Sep 2, 2021 •

edited

Loading

DilumAluthge Sep 2, 2021

Improve memory use and performance of multipart parsing #745

Improve memory use and performance of multipart parsing #745

Conversation

quinnj commented Aug 31, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mbauman commented Aug 31, 2021

Choose a reason for hiding this comment

quinnj commented Aug 31, 2021

codecov-commenter commented Aug 31, 2021 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

DilumAluthge Sep 2, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov-commenter commented Aug 31, 2021 •

edited

Loading

DilumAluthge Sep 2, 2021 •

edited

Loading