Skip to content

Conversation

mariash
Copy link
Member

@mariash mariash commented Nov 22, 2021

Add another listener to Envoy that proxies port 61443 to 8080 inside of
container. It serves the SSL certificate that contains SAN with all
application internal routes.

#180173340

@mariash mariash requested a review from aminjam November 22, 2021 19:06
@aminjam
Copy link

aminjam commented Nov 30, 2021

@mariash After deploying the changes in this PR and cloudfoundry/rep#32, I was able to curl another app by using the internal domain. That said, there were two things that didn't look right to me:

  1. I see that newly generated /etc/cf-assets/envoy_config/envoy.yaml is missing validation_context under 8080. I don't think this is intentional.
validation_context_sds_secret_config:
   name: server-validation-context
   sds_config:
       path: /etc/cf-assets/envoy_config/sds-server-validation-context.yaml
  1. I only see the following ports running in the container. I expected to see 61443 as an addition to the current list of ports and not a replacement of the 61001. It looks like in the new configuration Admin interface has taken the port that was previously used for proxying to 8080.
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 0.0.0.0:2222            0.0.0.0:*               LISTEN      13/diego-sshd
tcp        0      0 0.0.0.0:8080            0.0.0.0:*               LISTEN      7/proxy
tcp        0      0 0.0.0.0:61443           0.0.0.0:*               LISTEN      -
tcp        0      0 127.0.0.1:61001         0.0.0.0:*               LISTEN      -
tcp        0      0 0.0.0.0:61002           0.0.0.0:*               LISTEN      -

Add another listener to Envoy that proxies port 61443 to 8080 inside of
container. It serves the SSL certificate that contains SAN with all
application internal routes.

[#180173340](https://www.pivotaltracker.com/story/show/180173340)
@mariash
Copy link
Member Author

mariash commented Dec 1, 2021

@aminjam thank you for review and finding these issues. There was a code in container creation that was creating a map of application ports to proxy ports and that was removing one of the port mappings that had the same application port. I changed it to use the slice and that fixed both of these issues. There is an extra listener with validation context and extra port.

tcp        0      0 0.0.0.0:61443           0.0.0.0:*               LISTEN      -
tcp        0      0 0.0.0.0:61001           0.0.0.0:*               LISTEN      -
tcp        0      0 0.0.0.0:61002           0.0.0.0:*               LISTEN      -
tcp        0      0 127.0.0.1:61003         0.0.0.0:*               LISTEN      -
tcp        0      0 0.0.0.0:2222            0.0.0.0:*               LISTEN      38/diego-sshd
tcp        0      0 0.0.0.0:8080            0.0.0.0:*               LISTEN      8/proxy

I ran tests against the code - cats with cf-networking and routing, rats to make sure PR is ok.

@aminjam
Copy link

aminjam commented Dec 2, 2021

The PR looks good. There are couple of (unintentional) features this features is enabling that might be interesting to think through:

  1. Different apps can have the same internal route. This means that C2C with TLS could potentially lead the user to 2 different apps. Here are the steps to reproduce:
cf push appA, appB, AppC
cf map-route appB apps.internal --hostname something
cf map-route appC apps.internal --hostname something
cf restart appB
cf restart appC
cf add-network-policy appA appB --port 61443 --protocol tcp
cf add-network-policy appA appC --port 61443 --protocol tcp
cf ssh appA -c "bash -c 'for i in {1..10}; do curl -s --http1.1 https://something.apps.internal:61443 && echo ""; done'"
  1. DNS is resolved randomly and there is no round-robin or any other algorithm to access apps with multiple instances. In other words, it's possible that DNS resolves to the same instance causing an imbalance.

@mariash
Copy link
Member Author

mariash commented Dec 2, 2021

@aminjam yes this is interesting that you can bind same route to different apps. It is like having an app with multiple instances. This would be the same behavior that is available now without TLS where users ignore certificate with https to port 61001 or use unproxied port mapping and send request to 8080 directly with ipsec enabled in the installation. Not sure if this is a security concern since you have to add network policy to the second app.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

2 participants