Skip to content

Use openssl instead of tls in federator http2 client#3051

Merged
akshaymankar merged 21 commits intodevelopfrom
federation-openssl
Feb 28, 2023
Merged

Use openssl instead of tls in federator http2 client#3051
akshaymankar merged 21 commits intodevelopfrom
federation-openssl

Conversation

@akshaymankar
Copy link
Member

@akshaymankar akshaymankar commented Feb 1, 2023

Checklist

  • Add a new entry in an appropriate subdirectory of changelog.d
  • Read and follow the PR guidelines

@zebot zebot added the ok-to-test Approved for running tests in CI, overrides not-ok-to-test if both labels exist label Feb 1, 2023
@akshaymankar akshaymankar force-pushed the federation-openssl branch 3 times, most recently from 354d687 to 107fd90 Compare February 6, 2023 11:08
@akshaymankar akshaymankar marked this pull request as ready for review February 13, 2023 16:05
Copy link
Contributor

@pcapriotti pcapriotti left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

@akshaymankar akshaymankar force-pushed the federation-openssl branch 2 times, most recently from 4144d9d to b8ab49e Compare February 27, 2023 10:23
akshaymankar and others added 17 commits February 28, 2023 12:00
Make it clear that this only works with TLS 1.2 as of now
This will prevent reloading in case the files are being updated one by one.
It was broken in a previous commit so it was not testing with a hostname with
trailing dot at all.
Co-authored-by: Paolo Capriotti <paolo@capriotti.io>
This ensures that HTTP2 client doesn't close the connection before the response
body gets consumed.

In current implementation of the HTTP2 client there is a race between the part
which consumes the response and "background threads". These background threads
are sending and receiving data and they are not supposed to finish unless
connection gets abruptly terminated, however, due to the race they get a
`Async.cancel` when the response consumer function finishes executing.

Before this change, `Codensity` was supposed to ensure that the consumer doesn't
finish executing, but I am not sure why it didn't work, changing the code to use
CPS fixes this.
…Call"

This reverts commit febf71a2f00cb11aafb855e50c8d0c964af9c936.

Thanks to @pcapriotti for clarifying that the test was failing because the test
was exiting Codensity before making the assertion causing the test to fail.
@akshaymankar akshaymankar merged commit fd78663 into develop Feb 28, 2023
@akshaymankar akshaymankar deleted the federation-openssl branch February 28, 2023 14:15
smatting added a commit that referenced this pull request Mar 13, 2023
smatting added a commit that referenced this pull request Mar 13, 2023
akshaymankar added a commit that referenced this pull request Mar 21, 2023
* Revert "Revert "Use openssl instead of tls in federator http2 client (#3051)" (#3148)"

This reverts commit 615404b.

* Ensure that when http2 wants n bytes, we give it n bytes

`SSL.read ssl n` doesn't always return `n` bytes, so reading data multiple times
is necessary. Upstream PR has been made to warn future users:
haskell-cryptography/HsOpenSSL#81

* Add changelog

* Remove the IORef read buffer as openssl never returns extra bytes

* Read from openssl in tail recursion
lepsa pushed a commit to lepsa/wire-server that referenced this pull request Nov 28, 2023
* Use openssl instead of tls in federator http2 client

* changelog

* Strip trailing dot for hostname validation

* Move blessed ciphers close to where context is being built

Make it clear that this only works with TLS 1.2 as of now

* Check client certificate and private key to ensure they match

This will prevent reloading in case the files are being updated one by one.

* Add options to ssl context to workaround various bugs

https://www.openssl.org/docs/manmaster/man3/SSL_CTX_set_options.html#SSL_OP_ALL

* Remove leftover debugging code

* Ensure test for testing hostname with trailing dot is correct

It was broken in a previous commit so it was not testing with a hostname with
trailing dot at all.

* Remove commented out code for hs-tls

* Remove duplicated comment

* Slightly better types for CertifiateAndPrivateKeyDoNotMatch

* Share code to create ssl context between test and src

* Grammar

Co-authored-by: Paolo Capriotti <paolo@capriotti.io>

* federator: Pass response consumer continuation to discoverAndCall

This ensures that HTTP2 client doesn't close the connection before the response
body gets consumed.

In current implementation of the HTTP2 client there is a race between the part
which consumes the response and "background threads". These background threads
are sending and receiving data and they are not supposed to finish unless
connection gets abruptly terminated, however, due to the race they get a
`Async.cancel` when the response consumer function finishes executing.

Before this change, `Codensity` was supposed to ensure that the consumer doesn't
finish executing, but I am not sure why it didn't work, changing the code to use
CPS fixes this.

* Remove `-Wno-unused-imports`, perhaps added by mistake

* Federator Client: Simplify reading data from SSL

* Revert "federator: Pass response consumer continuation to discoverAndCall"

This reverts commit febf71a2f00cb11aafb855e50c8d0c964af9c936.

Thanks to @pcapriotti for clarifying that the test was failing because the test
was exiting Codensity before making the assertion causing the test to fail.

* federator-integration: Avoid exiting Codensity too soon

* federator: Run all code warpped in `withOpenSSL`

* federator-unit-tests: Ensure assertions happen without exiting Codensity

* Special handling for reading 0 bytes out of the TLS socket

---------

Co-authored-by: Paolo Capriotti <paolo@capriotti.io>
lepsa pushed a commit to lepsa/wire-server that referenced this pull request Nov 28, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ok-to-test Approved for running tests in CI, overrides not-ok-to-test if both labels exist

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

Comments