Allow arg/env in userdata for host-containers

[ellistarn]

What I'd like:
I'd like to be able to more easily parameterize env and args when running containers (similar to how I might w/ docker/k8s) using bottlerocket user data. I see there is a similar feature here: #1244, but IIUC, it doesn't quite go far enough to cover my use case.

[settings.host-containers.my-container]
enabled = true
env.MY_ENV_VAR = "my env var's value"
args = ["list", "of", "strings"]

Any alternatives you've considered:
I might be able to do this with just

[settings.host-containers.admin]
enabled = true
user-data = "aGkgdGhlcmUKaG93IGFyZSB5b3UK"

but this is beyond my current knowledge of bottlerocket.

[ellistarn]

Any word here?

[tjkirch]

Any word here?

Thanks for the idea - it would definitely be nice for users, and we plan to work on it, but haven't started yet.

[ellistarn]

This might be somewhat heretical, but it would be amazing in my eyes if this supported a variant of the YAML representation of https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.19/#podspec-v1-core. Perhaps a subset, like env, arg, cmd.

[ellistarn]

This would make bottlerocket an extremely attractive OS for running Kubernetes control planes.

[gabegorelick]

This is really needed for bootstrap containers. Currently, it seems like you have to build custom images to do anything useful. Even with user-data support, I'm not aware of any public images that handle user-data at /.bottlerocket/host-containers/current/user-data. So you have to end up having to build a generic host image that execs whatever's in /.bottlerocket/host-containers/current/user-data (assuming your user data is a script).

Supporting custom cmd/args would mean avoiding having to maintain that base image since you can just bash /.bottlerocket/host-containers/current/user-data from whatever image you like.