feat: support HTTP CONNECT

[zirain]

xref: #3663

[zirain]

encap.yaml

# This configuration takes incoming HTTP requests on port 10000 and encapsulates it in a CONNECT
# request which is sent upstream port 10001.
# `curl -H 'Host: www.google.com' --resolve www.google.com:10000:127.0.0.1 https://www.google.com:10000`
bootstrap_extensions:
  - name: envoy.bootstrap.internal_listener
    typed_config:
      "@type": type.googleapis.com/envoy.extensions.bootstrap.internal_listener.v3.InternalListener
static_resources:
  listeners:
    - name: http
      address:
        socket_address:
          protocol: TCP
          address: 0.0.0.0
          port_value: 10000
      filter_chains:
        - filters:
            - name: envoy.filters.network.http_connection_manager
              typed_config:
                "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
                stat_prefix: ingress_http
                access_log:
                  - name: envoy.file_access_log
                    typed_config:
                      "@type": type.googleapis.com/envoy.extensions.access_loggers.file.v3.FileAccessLog
                      log_format:
                        text_format: |
                          [%START_TIME%] "%REQ(:METHOD)% %REQ(X-ENVOY-ORIGINAL-PATH?:PATH)% %PROTOCOL%" %RESPONSE_CODE% %RESPONSE_FLAGS% "%UPSTREAM_TRANSPORT_FAILURE_REASON%" %BYTES_RECEIVED% %BYTES_SENT% %DURATION% %RESP(X-ENVOY-UPSTREAM-SERVICE-TIME)% "%REQ(X-FORWARDED-FOR)%" "%REQ(USER-AGENT)%" "%REQ(X-REQUEST-ID)%" "%REQ(:AUTHORITY)%" "%UPSTREAM_HOST%" %UPSTREAM_CLUSTER% %UPSTREAM_LOCAL_ADDRESS% %DOWNSTREAM_LOCAL_ADDRESS% %DOWNSTREAM_REMOTE_ADDRESS% %REQUESTED_SERVER_NAME% %ROUTE_NAME%
                      path: /dev/stdout
                route_config:
                  name: local_route
                  virtual_hosts:
                    - name: local_service
                      domains: ["*"]
                      routes:
                        - match:
                            prefix: "/"
                          route:
                            cluster: encap_cluster
                http_filters:
                  - name: envoy.filters.http.router
                    typed_config:
                      "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
    - name: encap
      internal_listener: {}
      filter_chains:
        - filters:
            - name: tcp
              typed_config:
                "@type": type.googleapis.com/envoy.extensions.filters.network.tcp_proxy.v3.TcpProxy
                access_log:
                  - name: envoy.file_access_log
                    typed_config:
                      "@type": type.googleapis.com/envoy.extensions.access_loggers.file.v3.FileAccessLog
                      log_format:
                        text_format: |
                          [%START_TIME%] "%REQ(:METHOD)% %REQ(X-ENVOY-ORIGINAL-PATH?:PATH)% %PROTOCOL%" %RESPONSE_CODE% %RESPONSE_FLAGS% "%UPSTREAM_TRANSPORT_FAILURE_REASON%" %BYTES_RECEIVED% %BYTES_SENT% %DURATION% %RESP(X-ENVOY-UPSTREAM-SERVICE-TIME)% "%REQ(X-FORWARDED-FOR)%" "%REQ(USER-AGENT)%" "%REQ(X-REQUEST-ID)%" "%REQ(:AUTHORITY)%" "%UPSTREAM_HOST%" %UPSTREAM_CLUSTER% %UPSTREAM_LOCAL_ADDRESS% %DOWNSTREAM_LOCAL_ADDRESS% %DOWNSTREAM_REMOTE_ADDRESS% %REQUESTED_SERVER_NAME% %ROUTE_NAME%
                      path: /dev/stdout
                stat_prefix: tcp_stats
                cluster: cluster_0
                tunneling_config:
                  hostname: example.com
  clusters:
    - name: encap_cluster
      load_assignment:
        cluster_name: encap_cluster
        endpoints:
          - lb_endpoints:
              - endpoint:
                  address:
                    envoy_internal_address:
                      server_listener_name: encap
    - name: cluster_0
      type: STRICT_DNS
      # This ensures HTTP/2 CONNECT is used for establishing the tunnel.
      typed_extension_protocol_options:
        envoy.extensions.upstreams.http.v3.HttpProtocolOptions:
          "@type": type.googleapis.com/envoy.extensions.upstreams.http.v3.HttpProtocolOptions
          explicit_http_config:
            http2_protocol_options: {}
      load_assignment:
        cluster_name: cluster_0
        endpoints:
          - lb_endpoints:
              - endpoint:
                  address:
                    socket_address:
                      address: 172.18.255.200 # Replace with the actual IP of the proxy service
                      port_value: 80

connect.yaml

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: backend
spec:
  hostnames:
    - "example.com"
  parentRefs:
    - group: gateway.networking.k8s.io
      kind: Gateway
      name: eg
  rules:
    - backendRefs:
        - group: ""
          kind: Service
          name: backend
          port: 3000
      matches:
        - path:
            type: PathPrefix
            value: /
---
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: BackendTrafficPolicy
metadata:
  name: connect
spec:
  targetRef:
    group: gateway.networking.k8s.io
    kind: Gateway
    name: eg
  httpUpgrade:
    - type: CONNECT
---
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: ClientTrafficPolicy
metadata:
  name: connect
spec:
  httpUpgrade:
    - type: CONNECT
  targetRef:
    group: gateway.networking.k8s.io
    kind: Gateway
    name: eg
  http2:
    allowConnect: true
---

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 70.64%. Comparing base (9ec0374) to head (70108a8).
Report is 4 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #6256      +/-   ##
==========================================
+ Coverage   70.62%   70.64%   +0.02%     
==========================================
  Files         220      220              
  Lines       36829    36861      +32     
==========================================
+ Hits        26011    26042      +31     
- Misses       9285     9286       +1     
  Partials     1533     1533              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[zhaohuabing]

@zirain
I think #3663 is asking for EG as the HTTP CONNECT proxy, but what we're implementing in this PR is EG as the HTTP CONNECT client?

• EG as the HTTP CONNECT proxy: Client -- (HTTP CONNECT channel with Auth headers) ---> Envoy ---(Proxied TCP traffic)---> Backend Service
• EG as the HTTP CONNECT client: Envoy -- (HTTP CONNECT channel) ---> Backend Service(HTTP Proxy)

It should be possible to use the HTTP Verb "CONNECT" to start a tcp tunnel through envoy to an upstream target.
This would have the benefit that one can use the http filters during initialization of the tcp tunnel (i.e. oauth2 filter for authentication and ext_auth for authorization).

[zirain]

@zirain I think #3663 is asking for EG as the HTTP CONNECT proxy, but what we're implementing in this PR is EG as the HTTP CONNECT client?

• EG as the HTTP CONNECT proxy: Client -- (HTTP CONNECT channel with Auth headers) ---> Envoy ---(Proxied TCP traffic)---> Backend Service
• EG as the HTTP CONNECT client: Envoy -- (HTTP CONNECT channel) ---> Backend Service(HTTP Proxy)

It should be possible to use the HTTP Verb "CONNECT" to start a tcp tunnel through envoy to an upstream target.
This would have the benefit that one can use the http filters during initialization of the tcp tunnel (i.e. oauth2 filter for authentication and ext_auth for authorization).

encap show an demo encapsulates in a CONNECT request, and the proxy provisioned by EG terminates h2 CONNECT then send to the backend service.

[zhaohuabing]

To improve the UI, is it possible to get the upgrade type from BackendTrafficPolicy only?

[arkodg]

yeah +1 to BTP

apiVersion: gateway.envoyproxy.io/v1alpha1
kind: BackendTrafficPolicy
metadata:
  name: connect
spec:
  targetRef:
    group: gateway.networking.k8s.io
    kind: Gateway
    name: eg
  httpUpgrade:
    - type: CONNECT
      connect: ...

#5300 (comment)

[zirain]

Confirmed with maintainer, we can leave the UpgradeConfigs on HCM empty.

Will support HTTP2/UDP CONNECT in a separated PR.

[arkodg]

my preference it to make it more explicit with a subfield, bool flag like ForwardTCP or Terminate , so its clearer

https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/http/upgrades#connect-support

Envoy can handle CONNECT in one of two ways, either proxying the CONNECT headers through as if they were any other request, and letting the upstream terminate the CONNECT request, or by terminating the CONNECT request, and forwarding the payload as raw TCP data.

When CONNECT upgrade configuration is set up, the default behavior is to proxy the CONNECT request, treating it like any other request using the upgrade path.

If termination is desired, this can be accomplished by setting [connect_config](https://www.envoyproxy.io/docs/envoy/latest/api-v3/config/route/v3/route_components.proto#envoy-v3-api-field-config-route-v3-routeaction-upgradeconfig-connect-config)

[zirain]

I'm fine with both, which one is better?

[arkodg]

@envoyproxy/gateway-maintainers @envoyproxy/gateway-reviewers any preference?

[rudrakhp]

+1 to Terminate as a bool

[arkodg]

LGTM thanks !

[zirain]

/retest