Skip to content

fix some subtle bugs with error handling and cleanup#243

Merged
quinnj merged 1 commit intoJuliaLang:masterfrom
vtjnash:jn/closewrite
Jul 18, 2022
Merged

fix some subtle bugs with error handling and cleanup#243
quinnj merged 1 commit intoJuliaLang:masterfrom
vtjnash:jn/closewrite

Conversation

@vtjnash
Copy link
Copy Markdown
Member

@vtjnash vtjnash commented Jul 15, 2022

TLS streams can be a bit odd, since they mostly ignore errors on the
underlying (unencrypted) stream. Clean up some of that error handling
(dealt with in wait_for_encrypted_data, so this actually ends up not
passing through mbedtls even though we still could feed it through
there), improve flow control reliability (avoids damaging the underlying
stream), implement new Base.closewrite API

@codecov
Copy link
Copy Markdown

codecov bot commented Jul 15, 2022

Codecov Report

Merging #243 (3d28e8e) into master (0c02e44) will increase coverage by 0.18%.
The diff coverage is 20.83%.

@@            Coverage Diff             @@
##           master     #243      +/-   ##
==========================================
+ Coverage   71.81%   72.00%   +0.18%     
==========================================
  Files          12       12              
  Lines         699      700       +1     
==========================================
+ Hits          502      504       +2     
+ Misses        197      196       -1     
Impacted Files Coverage Δ
src/ssl.jl 66.90% <20.83%> (-0.24%) ⬇️
src/ctr_drbg.jl 85.18% <0.00%> (+1.18%) ⬆️
src/entropy.jl 84.61% <0.00%> (+1.28%) ⬆️
src/x509_crt.jl 73.91% <0.00%> (+2.48%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0c02e44...3d28e8e. Read the comment docs.

@quinnj
Copy link
Copy Markdown
Member

quinnj commented Jul 16, 2022

Looks like closewrite is post-1.6 compatible

TLS streams can be a bit odd, since they mostly ignore errors on the
underlying (unencrypted) stream. Clean up some of that error handling
(dealt with in wait_for_encrypted_data, so this actually ends up not
passing through mbedtls even though we still could feed it through
there), improve flow control reliability (avoids damaging the underlying
stream), implement new `Base.closewrite` API
@vtjnash
Copy link
Copy Markdown
Member Author

vtjnash commented Jul 16, 2022

yes, but I don't bother calling closewrite on the underlying stream, so it is just about making the name right for those versions with a small shim

@quinnj quinnj merged commit 5c7735e into JuliaLang:master Jul 18, 2022
@vtjnash vtjnash deleted the jn/closewrite branch July 18, 2022 21:42
@thchr
Copy link
Copy Markdown

thchr commented Jul 28, 2022

I bisected the HTTP.jl error from JuliaWeb/HTTP.jl#896 to this commit.

@quinnj
Copy link
Copy Markdown
Member

quinnj commented Jul 28, 2022

Yeah, I think we better revert this change unless @vtjnash would have time to look into it sometime soon.

@vtjnash
Copy link
Copy Markdown
Member Author

vtjnash commented Jul 28, 2022

You will run into other more frequent deadlocks now if you attempt to revert this, since it does fix some problematic assumptions about stream behaviors. The HTTP says StackOverflowError, so it looks like there is a problem with the handlers there

Cint(MBEDTLS_ERR_NET_CONN_RESET)
if n == 0 ;@🤖 "f_recv $(isopen(bio) ? "WANT_READ" : "RECV_FAILED")"
return isreadable(bio) ? Cint(MBEDTLS_ERR_SSL_WANT_READ) :
Cint(MBEDTLS_ERR_NET_RECV_FAILED)
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you might need to feed through the actual error here, or indicate to mbedtls somehow if it reached eof on the underlying bio

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it support MBEDTLS_ERR_SSL_CONN_EOF? When I looked at the source, it wasn't clear to me how you indicated TCP FIN to mbedtls (before v1.3 of the TLS standard, this would have been an invalid termination, but was generally permitted in many clients and expected by some popular servers, so I assume mbedtls would support this)

@quinnj
Copy link
Copy Markdown
Member

quinnj commented Jul 28, 2022

The normal HTTP.request path doesn't have any mention of StackOverflow errors. If @vtjnash doesn't have time to investigate the issues that are arising from the changes he made, then I'll revert in the next 24 hours and tag a new release with the commits removed. I haven't seen or been shown specifically (only via vague mentions) code in HTTP.jl that is problematic, but if that can be specifically pointed out, I'm happy to dig in on the HTTP.jl side. I would obviously love to improve the robustness of the MbedTLS code, but we at least weren't having several opened issues pointing directly to the changed code prior to the changes here.

@vtjnash
Copy link
Copy Markdown
Member Author

vtjnash commented Jul 28, 2022

Could you write a test for what you think is wrong? This PR passes all tests, and additionally passes the stricter verification for stream-implementation correctness that is now on Julia master

@vtjnash
Copy link
Copy Markdown
Member Author

vtjnash commented Jul 29, 2022

Found it and added the missing handling + tested now #248

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants