engine: return new handles from EngineHandle::initEngine

[jpsim]

Until now, EngineHandle has always had a single value of 1. In other words, the Envoy engine has always been a singleton with no way to run multiple independent engines.

In order to support multiple concurrent engines (#332), we first need to make EngineHandle::initEngine create a new engine and rather than have functions that use the engine do a singleton lookup, we can use the envoy_engine_t as an opaque handle type to get back the Envoy::Engine instance.

Several places were using a hardcoded value of 1, so we need to plumb through the engine handles.

With this change although you can now create multiple engine instances, it is not safe to do so because there is still some static state that will need to be untangled. Also note that like the singleton we have had until now, new engine instances that are created will leak, as there is no way to fully release the engine through the public API. This is intentional in order to reduce the risk associated with this change.

Mike and I made these changes while pairing. The bulk of the ideas here are really from him.

Description:
Risk Level: High.
Testing: Existing tests & Lyft apps validation.
Docs Changes: None.
Release Notes: Added.

Co-authored-by: Mike Schore mike.schore@gmail.com
Co-authored-by: Alyssa Wilk alyssar@chromium.org
Signed-off-by: JP Simard jp@jpsim.com

[jpsim]

Interestingly I don't see the asan leaks locally, only on CI:

$ ./bazelw test //test/common/... --config=macos --config=asan-dev
//test/common:engine_common_test                                (cached) PASSED in 44.2s
//test/common:engine_test                                       (cached) PASSED in 38.2s
//test/common:main_interface_test                               (cached) PASSED in 38.1s
//test/common/bridge:utility_test                               (cached) PASSED in 17.8s
//test/common/buffer:bridge_fragment_test                       (cached) PASSED in 15.1s
//test/common/common:lambda_logger_delegate_test                (cached) PASSED in 13.0s
//test/common/data:utility_test                                 (cached) PASSED in 13.7s
//test/common/extensions/filters/http/assertion:assertion_filter_test (cached) PASSED in 43.6s
//test/common/extensions/filters/http/platform_bridge:platform_bridge_filter_integration_test (cached) PASSED in 32.3s
//test/common/extensions/filters/http/platform_bridge:platform_bridge_filter_test (cached) PASSED in 42.9s
//test/common/extensions/retry/options/network_configuration:network_configuration_retry_options_test (cached) PASSED in 43.2s
//test/common/http:client_test                                  (cached) PASSED in 49.9s
//test/common/http:header_utility_test                          (cached) PASSED in 12.8s
//test/common/integration:client_integration_test               (cached) PASSED in 69.9s
//test/common/network:configurator_test                         (cached) PASSED in 19.2s
//test/common/network:src_addr_socket_option_impl_test          (cached) PASSED in 33.9s
//test/common/network:synthetic_address_impl_test               (cached) PASSED in 15.6s
//test/common/stats:utility_test                                (cached) PASSED in 41.2s
//test/common/stream_info:extra_stream_info_test                (cached) PASSED in 13.8s
//test/common/thread:lock_guard_test                            (cached) PASSED in 11.9s

[Augustyniak]

Should we update the description of the PR? It contains some WIP notes.

[Augustyniak]

Let's mention #332 in the description of the PR too.

[jpsim]

@Augustyniak I've updated the PR description to make more sense as a commit message when merging this.

[goaway]

nit: Not sure this strictly needs to be in release notes, since technically this is not a public type/interface.

[jpsim]

Yes, although it's possible to have relied on the fact that this was a predictable value from external consumers, as brittle as that may have been.

[goaway]

Nice work @jpsim!