Support secrets of type environment

[PigeonF]

Is your feature request related to a problem? Please describe.

The compose spec specifies that secrets can be read from the environment (https://github.com/compose-spec/compose-spec/blob/master/spec.md#secrets-top-level-element, list item environment), but as far as I can tell, devel currently only supports file

podman-compose/podman_compose.py

Line 560 in 91bc6eb

source_file = declared_secret.get("file", None)

and external/name

podman-compose/podman_compose.py

Line 596 in 91bc6eb

if declared_secret.get("external", False) or declared_secret.get("name", None):

Describe the solution you'd like

podman-compose should be able to load secrets from environment variables

Additional context

Related issues: #655 #440 #589

[71ms1]

Yesterday I made an ugly hack in an attempt to get it working :)

podman create --name=xxx_minio_1 ... --secret minio_root_password,type=env,target=MINIO_ROOT_PASSWORD -p 9090:9090 --restart no minio/minio:latest server /data --console-address :9090

More or less it is working quite fine for me now.

def get_secret_args(compose, cnt, secret):
    secret_name = secret if is_str(secret) else secret.get('source', None)
    if not secret_name or secret_name not in compose.declared_secrets.keys():
        raise ValueError(
            'ERROR: undeclared secret: "{}", service: "{}"'
            .format(secret, cnt['_service'])
        )
    declared_secret = compose.declared_secrets[secret_name]

    source_file = declared_secret.get('file', None)
    dest_file = ''
    secret_opts = ''

    secret_target = None if is_str(secret) else secret.get('target', None)
    secret_uid = None if is_str(secret) else secret.get('uid', None)
    secret_gid = None if is_str(secret) else secret.get('gid', None)
    secret_mode = None if is_str(secret) else secret.get('mode', None)
    secret_type = None if is_str(secret) else secret.get('type', None)

    if source_file:
        if not secret_target:
            dest_file = '/run/secrets/{}'.format(secret_name)
        elif not secret_target.startswith("/"):
            dest_file = '/run/secrets/{}'.format(secret_target if secret_target else secret_name)
        else:
            dest_file = secret_target
        volume_ref = [
            '--volume', '{}:{}:ro,rprivate,rbind'.format(source_file, dest_file)
        ]
        if secret_uid or secret_gid or secret_mode:
            print(
                'WARNING: Service "{}" uses secret "{}" with uid, gid, or mode.'
                    .format(cnt['_service'], secret_target if secret_target else secret_name)
                + ' These fields are not supported by this implementation of the Compose file'
            )
        return volume_ref
    # v3.5 and up added external flag, earlier the spec
    # only required a name to be specified.
    # docker-compose does not support external secrets outside of swarm mode.
    # However accessing these via podman is trivial
    # since these commands are directly translated to
    # podman-create commands, albiet we can only support a 1:1 mapping
    # at the moment
    if declared_secret.get('external', False) or declared_secret.get('name', None):
        secret_opts += ',uid={}'.format(secret_uid) if secret_uid else ''
        secret_opts += ',gid={}'.format(secret_gid) if secret_gid else ''
        secret_opts += ',mode={}'.format(secret_mode) if secret_mode else ''
        secret_opts += ',type={}'.format(secret_type) if secret_type else ''
        secret_opts += ',target={}'.format(secret_target) if secret_target and secret_type == 'env' else ''
        # The target option is only valid for type=env,
        # which in an ideal world would work
        # for type=mount as well.
        # having a custom name for the external secret
        # has the same problem as well
        ext_name = declared_secret.get('name', None)
        err_str = 'ERROR: Custom name/target reference "{}" for mounted external secret "{}" is not supported'
        if ext_name and ext_name != secret_name:
            raise ValueError(err_str.format(secret_name, ext_name))
        elif secret_target and (secret_type != 'env' and secret_target != secret_name):
            raise ValueError(err_str.format(secret_target, secret_name))
        elif secret_target and secret_type != 'env':
            print('WARNING: Service "{}" uses target: "{}" for secret: "{}".'
                    .format(cnt['_service'], secret_target, secret_name)
                  + ' That is un-supported and a no-op and is ignored.')
        return [ '--secret', '{}{}'.format(secret_name, secret_opts) ]

    raise ValueError('ERROR: unparseable secret: "{}", service: "{}"'
                        .format(secret_name, cnt['_service']))

[muayyad-alsadi]

I'll wait for henryreed to complete is PR

and then if he did not solve it, I'll accept yours

#574 (comment)

[71ms1]

Sure, no problem. As I said this is just an ugly hack to get it working. I will prepare a PR as soon as I am ready.

In the meantime I try to get around another problem to pass --arch=amd64 to podman craete as this makes some trouble on macs with podman machine.

Ok, this one was not too obvious but just pass it over the run cmd like:

--podman-run-args"--arch=amd64"

and it will end up in 'create'

podman create --arch=amd64 --name=...

[muayyad-alsadi]

we are considering passing podman specific arguments using x-podman
but I'm considering a more generic way

https://github.com/containers/podman-compose/blob/devel/tests/uidmaps/docker-compose.yml

[redgoldlace]

Hey there - I was looking to use environment-based secrets with podman-compose and came across this issue. Is there any chance of this functionality being supported soon?

[FrankyBoy]

@71ms1 sorry to ping on such an old thread but is there any reason for these checks with external secrets? I.e. I was just trying to map an external secret with source and target and (I think) ran afoul of this line: elif secret_target and (secret_type != 'env' and secret_target != secret_name). Is there any technical reason behind this or am I missing something?

Relevant parts of compose look like so:

services:
  myService:
    secrets:
      - source: ExternalSecretName
        target: LocalSecretName

secrets:
  ExternalSecretName:
    external: true

The compose spec docs also don't say why this would not be allowed.

[p12tic]

Fixed in #971.

[evili]

Hi,

I think that this issue should not be closed. PR #971 refers to target environment variables loaded with some secret value.

The top-level "secrets" element in docker-compose admits the "environment" kind of secret so that the secret can be loaded from a environment variable to whatever target secret you want.

For example:

---
name: environment_secret_example

secrets:
  postgres_password:
    environment: "POSTGRES_PASSWORD"

services:
  db:
    image: "docker.io/library/postgres:16"
    container_name: postgres
    secrets:
      - postgres_password
    environment:
      POSTGRES_PASSWORD_FILE: /run/secrets/postgres_password

In this case the runtime environment variable POSTGRES_PASSWORD is "converted" to the "postgres_password" secret, then used by the "db" container as a file secret. Finally, the "db" container reads the file with the secret value "/run/secrets/postgres_password" as indicated by the container environment variable POSTGRES_PASSWORD_FILE.