Skip to content

core: integration with Envoy's API listener #616

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
junr03 merged 41 commits into from
Jan 30, 2020
Merged

core: integration with Envoy's API listener #616

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
4e4ace3
wip: headers, data, trailers working and unit tested
Dec 17, 2019
20d3e1e
wire up reset, and track stream state for cleanup
Dec 18, 2019
8418ee9
terminal states updated. Missing tests.
Dec 18, 2019
56914a0
cleanup
Dec 19, 2019
62ebd14
update types and config
Dec 23, 2019
fcb0870
wip integration with envoy and dynamic forward proxy
Dec 24, 2019
76aee26
use named listener
Dec 26, 2019
f069c82
api handle
Dec 27, 2019
f38e77c
fmt
Dec 27, 2019
cec541f
updated interfaces to keep up with envoy changes
Jan 7, 2020
9b8b5e2
first pass at closed state pushed down to the core
Jan 13, 2020
1ab5cd6
Merge branch 'master' into hcm-integration
Jan 13, 2020
cf69083
always zero-init
Jan 14, 2020
85d1acf
v3 types
Jan 14, 2020
3cd3a1a
Merge branch 'master' into hcm-integration
Jan 17, 2020
c9721d9
added fixme before merge
Jan 17, 2020
e131965
update after api listener PR merged upstream
Jan 21, 2020
f97b8b1
fmt
Jan 21, 2020
a98cfbf
update test
Jan 22, 2020
5e740ae
swift mock
Jan 22, 2020
e6d0d91
update cleanup and dispatchable
Jan 23, 2020
0f2ba0f
preferred network and cluster updates
Jan 24, 2020
0ef3ef3
buffering
Jan 24, 2020
65b7ba9
moved cancellation to synchronous code, and added test suite
Jan 25, 2020
58b60d1
fix
Jan 27, 2020
96ca3ca
clean up platform level cancel code
Jan 27, 2020
4ef47cb
fmt
Jan 27, 2020
ef68ba6
mutex on streams_
Jan 27, 2020
e0ac9cb
cross-thread unit test
Jan 28, 2020
644d338
missing BUILD file
Jan 28, 2020
024f15b
swift test
Jan 28, 2020
ddf302b
test output
Jan 28, 2020
413d813
full test suite for races
Jan 28, 2020
c718e49
fix memory leak
Jan 29, 2020
3ee492b
preferred network test
Jan 29, 2020
a3c2d8a
basic integration test
Jan 29, 2020
9fcbf08
fmt
Jan 29, 2020
191105a
comment clarity
Jan 29, 2020
f3d8ff2
dispatch lock for race between cancellation and non-terminal encodings
Jan 30, 2020
ed88f67
definition
Jan 30, 2020
8af70cf
increase timeout for perf build
Jan 30, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .github/workflows/core.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ jobs:
- name: 'Install dependencies'
run: ./ci/mac_ci_setup.sh
- name: 'Run tests'
run: bazel test //test/common/...
run: bazel test --test_output=all //test/common/...
tsan:
name: tsan
runs-on: ubuntu-18.04
Expand All @@ -34,7 +34,7 @@ jobs:
export PATH=/usr/lib/llvm-8/bin:$PATH
export CC=clang
export CXX=clang++
bazel test --config=clang-tsan //test/common/...
bazel test --config=clang-tsan --test_output=all //test/common/...
asan:
name: asan
runs-on: ubuntu-18.04
Expand All @@ -50,4 +50,4 @@ jobs:
export PATH=/usr/lib/llvm-8/bin:$PATH
export CC=clang
export CXX=clang++
bazel test --config=clang-asan //test/common/...
bazel test --config=clang-asan --test_output=all //test/common/...
6 changes: 3 additions & 3 deletions .github/workflows/perf.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ jobs:
sizecurrent:
name: size_current
runs-on: ubuntu-18.04
timeout-minutes: 45
timeout-minutes: 60
steps:
- uses: actions/checkout@v1
with:
Expand All @@ -26,7 +26,7 @@ jobs:
sizemaster:
name: size_master
runs-on: ubuntu-18.04
timeout-minutes: 45
timeout-minutes: 60
steps:
- uses: actions/checkout@v1
with:
Expand All @@ -45,7 +45,7 @@ jobs:
name: size_compare
needs: [sizecurrent, sizemaster]
runs-on: ubuntu-18.04
timeout-minutes: 45
timeout-minutes: 30
steps:
- uses: actions/checkout@v1
- name: 'Install dependencies'
Expand Down
93 changes: 65 additions & 28 deletions library/common/config_template.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,19 +3,51 @@
*/
const char* config_template = R"(
static_resources:
listeners:
- name: base_api_listener
address:
socket_address:
protocol: TCP
address: 0.0.0.0
port_value: 10000
api_listener:
api_listener:
"@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
stat_prefix: hcm
route_config:
name: api_router
virtual_hosts:
- name: api
domains:
- "*"
routes:
- match:
prefix: "/"
route:
cluster_header: x-envoy-mobile-cluster
http_filters:
- name: envoy.filters.http.dynamic_forward_proxy
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.http.dynamic_forward_proxy.v3.FilterConfig
dns_cache_config:
name: dynamic_forward_proxy_cache_config
dns_lookup_family: AUTO
dns_refresh_rate: {{ dns_refresh_rate_seconds }}s
- name: envoy.router
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
clusters:
- name: base # Note: the direct API depends on the existence of a cluster with this name.
- name: base
connect_timeout: {{ connect_timeout_seconds }}s
dns_refresh_rate: {{ dns_refresh_rate_seconds }}s
http2_protocol_options: {}
lb_policy: ROUND_ROBIN
load_assignment:
cluster_name: base
endpoints: &base_endpoints
- lb_endpoints:
- endpoint:
address:
socket_address: {address: {{ domain }}, port_value: 443}
lb_policy: CLUSTER_PROVIDED
cluster_type:
name: envoy.clusters.dynamic_forward_proxy
typed_config:
"@type": type.googleapis.com/envoy.extensions.clusters.dynamic_forward_proxy.v3.ClusterConfig
dns_cache_config:
name: dynamic_forward_proxy_cache_config
dns_lookup_family: AUTO
dns_refresh_rate: {{ dns_refresh_rate_seconds }}s
transport_socket: &base_transport_socket
name: envoy.transport_sockets.tls
typed_config:
Expand All @@ -28,33 +60,38 @@ const char* config_template = R"(
#include "certificates.inc"
R"(
sni: {{ domain }}
type: LOGICAL_DNS
upstream_connection_options: &upstream_opts
tcp_keepalive:
keepalive_interval: 10
keepalive_probes: 1
keepalive_time: 5
- name: base_wlan # Note: the direct API depends on the existence of a cluster with this name.
transport_socket: *base_transport_socket
upstream_connection_options: *upstream_opts
- name: base_wlan
connect_timeout: {{ connect_timeout_seconds }}s
dns_refresh_rate: {{ dns_refresh_rate_seconds }}s
http2_protocol_options: {}
lb_policy: ROUND_ROBIN
load_assignment:
cluster_name: base_wlan
endpoints: *base_endpoints
lb_policy: CLUSTER_PROVIDED
cluster_type:
name: envoy.clusters.dynamic_forward_proxy
typed_config:
"@type": type.googleapis.com/envoy.extensions.clusters.dynamic_forward_proxy.v3.ClusterConfig
dns_cache_config:
name: dynamic_forward_proxy_cache_config
dns_lookup_family: AUTO
dns_refresh_rate: {{ dns_refresh_rate_seconds }}s
transport_socket: *base_transport_socket
type: LOGICAL_DNS
upstream_connection_options: *upstream_opts
- name: base_wwan # Note: the direct API depends on the existence of a cluster with this name.
- name: base_wwan
connect_timeout: {{ connect_timeout_seconds }}s
dns_refresh_rate: {{ dns_refresh_rate_seconds }}s
http2_protocol_options: {}
lb_policy: ROUND_ROBIN
load_assignment:
cluster_name: base_wwan
endpoints: *base_endpoints
lb_policy: CLUSTER_PROVIDED
cluster_type:
name: envoy.clusters.dynamic_forward_proxy
typed_config:
"@type": type.googleapis.com/envoy.extensions.clusters.dynamic_forward_proxy.v3.ClusterConfig
dns_cache_config:
name: dynamic_forward_proxy_cache_config
dns_lookup_family: AUTO
dns_refresh_rate: {{ dns_refresh_rate_seconds }}s
transport_socket: *base_transport_socket
type: LOGICAL_DNS
upstream_connection_options: *upstream_opts
- name: stats
connect_timeout: {{ connect_timeout_seconds }}s
Expand Down
4 changes: 3 additions & 1 deletion library/common/engine.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,9 @@ envoy_status_t Engine::run(std::string config, std::string log_level) {
postinit_callback_handler_ = main_common_->server()->lifecycleNotifier().registerCallback(
Envoy::Server::ServerLifecycleNotifier::Stage::PostInit, [this]() -> void {
Server::Instance* server = TS_UNCHECKED_READ(main_common_)->server();
http_dispatcher_->ready(server->dispatcher(), server->clusterManager());
auto api_listener = server->listenerManager().apiListener()->get().http();
ASSERT(api_listener.has_value());
http_dispatcher_->ready(server->dispatcher(), api_listener.value());
Copy link
Contributor

@goaway goaway Jan 29, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this additional delay (beyond construction) still necessary? We're no longer waiting on an initial round of DNS resolution and the filter dynamic forwarding filter contains built in buffering logic while we do wait. If we're unsure, maybe insert a TODO to follow up.

Copy link
Contributor

@goaway goaway Jan 30, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still suggest adding a TODO and/or issue for this so we still have visibility to follow up.

});
} // mutex_

Expand Down
6 changes: 5 additions & 1 deletion library/common/http/BUILD
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,16 +14,20 @@ envoy_cc_library(
"//library/common/buffer:bridge_fragment_lib",
"//library/common/buffer:utility_lib",
"//library/common/http:header_utility_lib",
"//library/common/network:synthetic_address_lib",
"//library/common/thread:lock_guard_lib",
"//library/common/types:c_types_lib",
"@envoy//include/envoy/buffer:buffer_interface",
"@envoy//include/envoy/event:dispatcher_interface",
"@envoy//include/envoy/http:api_listener_interface",
"@envoy//include/envoy/http:async_client_interface",
"@envoy//include/envoy/http:header_map_interface",
"@envoy//include/envoy/upstream:cluster_manager_interface",
"@envoy//source/common/buffer:buffer_lib",
"@envoy//source/common/common:lock_guard_lib",
"@envoy//source/common/common:minimal_logger_lib",
"@envoy//source/common/common:thread_lib",
"@envoy//source/common/common:thread_synchronizer_lib",
"@envoy//source/common/http:codec_helper_lib",
"@envoy//source/common/http:headers_lib",
"@envoy//source/common/http:utility_lib",
],
Expand Down
Loading