Add limayaml param settings to provisioning script environment

[jandubois]

They will be prefixed with PARAM_, so param.FOO becomes PARAM_FOO.

This is useful because parameter substitution happens when a template is instantiated, so [ "{{.Param.ROOTFUL}}" = true ] becomes [ "true" = true ] in the cloud-init-output.log.

This mechanism also works better when the parameter contains quotes, which would break a simplistic FOO="{{.Param.FOO}}".

Test sample:

param:
  FOO: 'foo"bar'

provision:
- mode: system
  script: |
    echo "$PARAM_FOO"

Output:

LIMA 2024-08-30T09:13:11-07:00| Executing /mnt/lima-cidata/provision.system/00000000
foo"bar
LIMA 2024-08-30T09:13:11-07:00| Exiting with code 0

Example of real-life usage (from #2515) will be to replace

-     # Setting shell variable makes it easier to read cloud-init-output.log
-     readonly ROOTFUL="{{.Param.ROOTFUL}}"
-     if [ "$ROOTFUL" = true ]; then
+     if [ "$PARAM_ROOTFUL" = true ]; then

[jandubois]

Needs docs!

Also wondering if probes should have access too (they currently don't have access to LIMA_CIDATA_* variables, I think).

[jandubois]

I'm not proud of the hack to make the $PARAM_* variables available to the probes, but I couldn't figure out a cleaner way as long as we support arbitrary interpreters via the hashbang line.

Anyways, I think the PR is ready for review now.

[norio-nomura]

This change looks like something I considered before implementing #2498 but ultimately abandoned because it seemed too complex for me. I find it impressive and at the same time feel that it was indeed too challenging for me.

I haven't finished reading the probes hack yet, but I'll share what I understand so far.

[jandubois]

This change looks like something I considered before implementing #2498

This change just build on top of #2498, which is still needed so we can use {{.Param.Key}} in all the places that are not provisioning scripts or probes.

I haven't finished reading the probes hack yet, but I'll share what I understand so far.

I've moved the hack to a separate function and added rather verbose documentation about it. It feels kind of too long, but maybe it is useful.

[norio-nomura]

The implementation and comments for prefixExportParam() made it very clear. Thanks to that, I can suggest a simpler command.
Adding tests to test-misc.yaml is also very clear and good.

[norio-nomura]

LGTM!

[jandubois]

LGTM!

🙏 Thanks! All checks have passed, so I will merge now.

[nirs]

This is awesome improvement, I wanted to ask how to inject values from the instance yaml to the cloud-init scripts.

However the names makes this harder to understand. Maybe rename to:

env:
  MY_KEY: VALUE
provision:
  - mode: system
    script: |
      #!/bin/bash
      echo "MY_KEY"

No magic prefix, what you put in env added to the provision scripts.

This is also similar to docker/podman --env flag.

[norio-nomura]

Maybe rename to:

The .env field already exists. However, it had too broad of an impact to use solely for modifying the behavior of scripts within limayaml. That's why I wanted variables that could only be used in Lima, and after some consideration, I implemented .param.

[nirs]

Yes being able to limit the scope is nice. Did you consider something like this?

env:
  KEY: global
bootEnv:
  BOOT_KEY: boot
provision:
  - mode: system
    script: |
      echo $KEY       # -> gloabl
      echo $BOOT_KEY  # -> boot
probes:
  - mode: system
    script: |
      echo $KEY       # -> gloabl
      echo $BOOT_KEY  # -> boot

Since all provision steps are run by the same global boot.sh script, we can pass the bootEnv to the scripts by exporting the boot env vars from the script.

I did not check how probes are run, but since they are scripts there must be some shell running them and we can modify its environment.

[etho201]

Is it possible to define the parameters from an external file (for example, variables.yaml) in order to keep custom/sensitive parameters out of the template files?

[jandubois]

Is it possible to define the parameters from an external file (for example, variables.yaml) in order to keep custom/sensitive parameters out of the template files?

Not right now, but it will be once the implementation of the basedOn mechanism I described in #2520 (reply in thread) is complete. I'm working on it right now; the tricky part is merging YAML files while retaining comments and not just values.

You will be able to write your template like this:

basedOn:
- ~/.variables.yaml
- template://fedora

provision:
…

Or you could have your private template that contains just the variables, and then reference a public base template:

basedOn: [https://example.com/my-shared-template.yaml]

param:
  secret: password123

The settings from the base templates will be merged with your instance template when the instance is created, similar to $LIMA_HOME/_config/default.yaml is providing defaults when the instance starts.

That means values from the base templates act as defaults, but the settings in the main template take precedence.

It also means that the combined template will include your secrets, but it is stored in the Lima instance directory, which is only readable by your own user (permission 600).

[nirs]

Or you could have your private template that contains just the variables, and then reference a public base template:

basedOn: [https://example.com/my-shared-template.yaml]

param:
  secret: password123

This is not a good way to store secrets. We can add keychain integration so the owner of the secret can unlock the keychain or lima for accessing the secret when stating a cluster.

I looked at this for this issue:
nirs/oc-clusterset#1

[jandubois]

This is not a good way to store secrets.

I agree, but @etho201 asked about secrets that are already in a text file. It is similar to ~/.aws containing credentials.

We can add keychain integration so the owner of the secret can unlock the keychain or lima for accessing the secret when stating a cluster.

As long as the secret eventually ends up in ~/.lima/INSTANCE/lima.yaml it does not make a big difference; you still end up with the clear text on the disk. Even if not there, it will still be in the cloud-init data somehow, with the instance being able to access it.

So ideally you want something like ssh agent forwarding, but it all depends on how those secrets are going to be used during provisioning. I can't think of a generic solution.

[jandubois]

BTW, if you want to continue this discussion, please move it to the Discussions. Having it at the bottom of the thread of an already closed pull request means it will not be very visible.