-
Notifications
You must be signed in to change notification settings - Fork 644
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: use dynamodb instead of ssm for JIT config #4446
base: main
Are you sure you want to change the base?
feat: use dynamodb instead of ssm for JIT config #4446
Conversation
f3d5721
to
89cac18
Compare
Add extra capabilities to support ephemeral multiarch runners. In workflow jobs with large matrix, github can request large number of runners simultaneously. This can cause the ssm to be overwhelmed. This PR replaces the ssm with dynamodb to store the JIT config. Also fixed a couple things: - `runners_ssm_housekeeper` schema was not aligned across modules - syntax for `env` in `github_agent.ubuntu.pkr.hcl` Added a couple of improvements: - updated baseline provisioning script for prebuilt runners to make it closer to what github hosted runners do - startup-runner.sh now automatically discovers attached ebs volume and mounts it to /data
1711979
to
16a67b8
Compare
@id thanks for creating the PR. It is abit one so need to find some time to review. I see you also added a lambda for cleaning up github left overs. Would it be possible to split this in a sperate PR? Would help the review and testing part. Are you on our discord server? Maybe a short chat about some question is quicker when I go over the code. |
Not much time this week. Although I had a quick look. Before I dig in the details there are some concerns, I have.
|
@npalm thanks for checking this out. Yes, I'll split the PR into more sensible once. Hopefully will have time next week. About DynamoDB, I think we could use something like this in combination with using full isntance ARN for the keys instead of just instance id as it is now. I'll test this, and will report back.
|
If DynamoDB turns out to be working ok, we could as a minimum get rid of ssm housekeeper, since in DynamoDB we have built in TTL for items. |
Add extra capabilities to support ephemeral multiarch runners.
In workflow jobs with large matrix, github can request large number of runners simultaneously. This can cause the ssm to be overwhelmed.
This PR replaces the ssm with dynamodb to store the JIT config.
Also fixed a couple things:
runners_ssm_housekeeper
schema was not aligned across modulesenv
ingithub_agent.ubuntu.pkr.hcl
Added a couple of improvements:
startup-runner.sh
now automatically discovers attached ebs volume, mounts it to/data
, and points root directory for docker and containerd daemons to/data/docker
and/data/containerd
respectively.