change local dev to use k8s

[hahn-kev]

No description provided.

[rmunn]

One challenge is that while Kubernetes can bind-mount a local directory into a volume, you must specify an absolute path to the directory to be mounted. You can't have it be relative to the config file, as we can with docker compose. Which means that we can't just bind-mount the GIt repo into the backend and/or frontend containers and run dotnet watch or pnpm run dev in them — because on one developer's machine, the Git repo may live in a path like C:\Users\Kevin\repos\lexbox and on someone else's machine the path may be /home/rmunn/code/lexbox. So there's no way to check a Kubernetes config file into Git that will be guaranteed to work on all developer machines.

[rmunn]

There may be a solution. It would involve running the backend and frontend locally with the dotnet watch and pnpm instructions so that you get live updates, and then when you're ready to see your code running in the Kubernetes containers, you check it into Git and do a push. Details below.

We start with backend (api) and frontend (ui) containers that have an init container defined. That init container will check the repo volume (which we will define using localDir) and, if it's empty, clone it from GitHub so that we have a pretty recent copy of the repo. (Your local repo may have more commits that you haven't pushed to GitHub yet, but that's fine). Then the init container finishes, and the api and ui containers start up, with the repo mounted on /repo.

There will be a task in our Taskfile.yml that does the following:

• Create (if one does not already exist) an SSH key with no passphrase, stored in $HOME/.ssh/id_lexbox_dev.rsa
• Add (if it does not already exist) an entry called k8sdevin your$HOME/.ssh/config` file
• Use ssh-keygen -L to get the public key of that SSH key
• Write the name of that SSH key into a .env file that Kubernetes will use to populate a Kubernetes secret
• Start up the Kubernetes container environment, which will use that secret to populate the .ssh/authorized_keys file so that you can ssh into the frontend and backend boxes

Then you set up a Git remote called k8sdev that will push to k8sdev:/repo. The k8sdev entry in your .ssh/config will be used to determine which key to use, and the authorized_keys file in the container will ensure that that key is allowed to push.

Then when you want to see your code running in the dev k8s boxes, you check your changes into a commit and run git push k8sdev. That pushes your commit to the repo running inside the k8s container, where dotnet watch and/or pnpm run dev pick up on the change and automatically update the backend or frontend with the changes you just pushed.

Additionally, we will create a Git post-checkout hook that looks to see if you have checked out a new branch, and ssh'es into the k8s containers to checkout that same branch in there (possibly pushing it if necessary). That way when you move from develop onto a feature/my-new-work branch, your local k8s dev containers will also switch to the same branch so that you can continue pushing.

That all sounds complicated, but it should be straightforward to write a task that will do all that for you. So all you'll need to do is run task devsetup and it will set up the SSH key, the Git remote, the Git hooks, and so on.

SUMMARY

There will be a task in our Taskfile.yml that sets up the dev environment. You will run the backend and frontend locally using dotnet watch and pnpm run dev, and then when you want to see your code running in local Kubernetes containers you will commit it and run git push k8sdev. Everything else will be handled more or less automatically for you.