Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace Traefik with kamal-proxy #940

Merged
merged 43 commits into from
Sep 17, 2024
Merged

Replace Traefik with kamal-proxy #940

merged 43 commits into from
Sep 17, 2024

Conversation

djmb
Copy link
Collaborator

@djmb djmb commented Sep 16, 2024

Kamal now uses our custom proxy kamal-proxy. The proxy responds to a deploy command which tells it which container to send requests to. It then waits for the container to respond on its healthcheck URL and returns once that is successful.

Changes this brings:

  • The app, proxy and accessories will now run in a custom network called "kamal". This allows us to use the container id as a stable hostname that will survive container restarts.
  • No need for cordfiles, which were a hack used to stop Traefik from sending request to a container to make it unhealthy. Kamal-proxy just sends traffic where we tell it to.
  • The healthcheck config has been removed. Containers are healthchecked externally be the proxy waiting for the healthcheck URL to return 200s. For non proxy containers (jobs etc), there is a new readiness_timeout option, which is used to wait for them to be ready. You can still set a custom healthcheck using the options config which allows you to pass custom options to docker run.

To enable proper multi-app support:

  • All files except for audit logs and locks for an app will be stored in .kamal/apps/#{service}[-#{destination}], so they can be easily removed by removing that folder.

Upgrading:

  • To upgrade from Kamal 1.x you need to run kamal proxy upgrade to upgrade the proxy containers. This will remove Traefik and boot the proxy and app containers in the kamal docker network. There will be a similar downgrade command added to Kamal 1.9 to reverse the process.
  • You'll also need to reboot accessories to move them into the kamal network.
  • You'll need to rename the pre-traefik-reboot and post-traefik-reboot hooks to pre-proxy-reboot and post-proxy-reboot.

Config changes:

  • traefik & healthcheck both removed
  • set proxy: true not traefik: true for non primary role roles that use a proxy
  • options for kamal-proxy, go under the proxy setting:
    hosts - a list of hosts that should be forwarded to this app by the proxy. Allows multiple apps to use the same server and proxy, and have the requests routed by the host. Currently the proxy only supports one host
    ssl - requires hosts to be set if enabled, if enabled it will automatically set up HTTPS via Let's Encrypt. Should only be used with one server!
    app_port - the port on the application that we will proxy (default 80)
    deploy_timeout - how long kamal-proxy waits for the container to come up (default 30 seconds)
    response_timeout - how long kamal-proxy waits for a response (default 30 seconds)
    healthcheck - configure the healthcheck URL, interval and timeout
    buffering - configure request and response buffering, max request and response size and memory buffer size
    logging - log extra request and response headers
    forward_headers - whether to forward X-Forwarded-For and X-Forwarded-Proto, defaults to false

djmb and others added 30 commits September 16, 2024 16:44
The proxy can be enabled via the config:

```
proxy:
  enabled: true
  hosts:
    - 10.0.0.1
    - 10.0.0.2
```

This will enable the proxy and cause it to be run on the hosts listed
under `hosts`, after running `kamal proxy reboot`.

Enabling the proxy disables `kamal traefik` commands and replaces them
with `kamal proxy` ones. However only the marked hosts will run the
kamal-proxy container, the rest will run Traefik as before.
Avoid docker inspect:
1. Use the container ID as the host
2. Configure the port, default to 3000
We need to drop to be fixed so multiple applications put the config in
the same place.
This will allow us to share files with the proxy via the host.
This will make running kamal remove simpler, we can just clean up that
directory.
@nickhammond
Copy link
Contributor

This is exciting! I haven't looked through the whole PR but just had a couple of questions come to mind, quotes added for context.

The app, proxy and accessories will now run in a custom network called "kamal". This allows us to use the container id as a stable hostname that will survive container restarts.

How will this work if you're currently using networks? Or if you're hosting multiple apps on the same server? I'm guessing the custom options just get added along with the default kamal network. By default this should give us more of a docker compose experience which is really nice, being able to connect to db for a host accessory for instance. There's a decent amount of questions around networking with Kamal that this will help address.

hosts - a list of hosts that should be forwarded to this app by the proxy. Allows multiple apps to use the same server and proxy, and have the requests routed by the host. Currently the proxy only supports one host
ssl - requires hosts to be set if enabled, if enabled it will automatically set up HTTPS via Let's Encrypt. Should only be used with one server!

SSL is one of the more difficult things to get going with traefik, is Thruster no longer the replacement for traefik? If an app does need to support multiple hosts, how would we go about that? I have multiple apps at the moment that utilize more than one host. Should the ssl section say requires host and not hosts?

@djmb
Copy link
Collaborator Author

djmb commented Sep 17, 2024

The app, proxy and accessories will now run in a custom network called "kamal". This allows us to use the container id as a stable hostname that will survive container restarts.

How will this work if you're currently using networks? Or if you're hosting multiple apps on the same server? I'm guessing the custom options just get added along with the default kamal network. By default this should give us more of a docker compose experience which is really nice, being able to connect to db for a host accessory for instance. There's a decent amount of questions around networking with Kamal that this will help address.

Right now Kamal will attempt to create the kamal network, but if it already exists it will just use it. So you could create the network yourself in the docker-setup hook if you want any custom options for it.

It might make sense to make the name of the network configurable. E.g. if you are running multiple applications on one server and want them in separate networks, or if you have an existing network you'd like to use.

hosts - a list of hosts that should be forwarded to this app by the proxy. Allows multiple apps to use the same server and proxy, and have the requests routed by the host. Currently the proxy only supports one host
ssl - requires hosts to be set if enabled, if enabled it will automatically set up HTTPS via Let's Encrypt. Should only be used with one server!

SSL is one of the #112 to get going with traefik, is Thruster no longer the replacement for traefik? If an app does need to support multiple hosts, how would we go about that? I have multiple apps at the moment that utilize more than one host. Should the ssl section say requires host and not hosts?

kamal-proxy currently only supports one host per application, but we'll add support for multiple hosts soon - see basecamp/kamal-proxy#14. I've switched the key from hosts to host for now, and we'll allow either once the support is in place.

Thruster is designed to run in the same container as the app, so its not a Traefik replacement. It can do SSL termination itself, but Kamal gets in the way if you are using it, so kamal-proxy also has support.

@djmb djmb merged commit 434490b into main Sep 17, 2024
9 checks passed
@djmb djmb deleted the proxy branch September 17, 2024 08:47
@nickhammond
Copy link
Contributor

@djmb Appreciate the context! I forgot about Thruster actually running within the container, makes sense.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants