Configuration option to support deployment with K8s readOnlyRootFilesystem security context

[Rozzii]

This proposal is aimed to introduce an optional new way to handle configuration generation at deployment time.
This proposal does not aim to change the default behavior of the ironic-image.

This new feature would be implemented as at least 1 new environment variable that could result in a different way of rendering the different configuration files. The configuration files would be moved to a path designated by the new configuration variable and no file writing operation would take place on the container's own file system.

Requirements for users:

• The path specified by the new environment variable has to be backed by a mounted volume .

Impact:

• Most likely files at these locations will be directed to the new custom location:
• /etc/keepalived/*
• /var/lib/misc/*
• /etc/dnsmasq.conf
• /tmp/ipa.conf
• /etc/ironic/*
• /bin/*
• /usr/local/lib/python*
• /var/lib/ironic/*
• /etc/apache2/*

Unknowns

• There is a possibility that more than one new environment variable needs to be introduced as there might be certain executables that wouldn't allow custom paths for config files (this needs to be checked and solved accordingly)

The impact has to be considered in the context of a K8s pod so because of the "run scripts" of Ironic the impact on a individual container level will be smaller but if all the effected paths of all the possible Ironic container deployments are congregated we will have the above described impact.

Non goals

• Changing the default behavior of configuration handling
• Deprecating any existing features
• Introducing new dependencies
• Making the new way of deployment the a recommended or defacto way of using ironic

Additional goals discovered during issue discussion

• Move some of the config files to sane default places.

[metal3-io-bot]

This issue is currently awaiting triage.
If Metal3.io contributors determine this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.
The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[tuminoid]

First thing is to make each file mountable, and then the second thing is to make it either default or create a kustomization to actually run ironic-image with read-only filesystem. This then required adding some emptyDir mounts so the config files are not persisted, as they're not persisted now either.

• /bin/*

This is going to be annoying as in /bin we have liveness and readiness probes, whose path is defined in BMO manifests.

• /usr/local/lib/python*

This is pretty weird one as well. Why are we writing here?

• /etc/dnsmasq.conf
• /tmp/ipa.conf

These two are the most important ones to move into subdirectories. The other config files already are.

[Rozzii]

First thing is to make each file mountable, and then the second thing is to make it either default or create a kustomization to actually run ironic-image with read-only filesystem. This then required adding some emptyDir mounts so the config files are not persisted, as they're not persisted now either.

• /bin/*

This is going to be annoying as in /bin we have liveness and readiness probes, whose path is defined in BMO manifests.

• /usr/local/lib/python*

This is pretty weird one as well. Why are we writing here?

• /etc/dnsmasq.conf
• /tmp/ipa.conf

These two are the most important ones to move into subdirectories. The other config files already are.

So dividing the response here to a few parts;

• I agree that it would be reasonable to move the defaults also to more reasonable places, it can be done as part of this work so I don't object.
• As we have discussed ofline I would like to do it with the least amount of mounts so I aim to put the feature behind "feature gate" so if no custom location is specified nothing would happen thus we would not break the BMO ironic manifests or IrSO, I don't want to prioritize making the default "sane" locations mountable but as a side effect I can do that too but from K8s user perspective the less mounts the user have to deal with the better the UX is.