Add docker_compose_v2 module

[felixfontein]

SUMMARY

Implements a basic module wrapping the Docker CLI compose plugin.

I copied some of the output parsing code from #586. Unfortunately docker compose's status reporting is pretty primitive, see also docker/compose#10872.

Fixes #216.
Closes #586.

Reference for changes to compose over time: https://docs.docker.com/compose/release-notes/

ISSUE TYPE

• New Module Pull Request

COMPONENT NAME

docker_compose_v2

[felixfontein]

This is now ready for review!

[felixfontein]

@lel-amri I've added the above note in response to #586 (comment).

[felixfontein]

https://ansible-collections.github.io/community.docker/pr/739/docker_compose_v2_module.html#notes

[lel-amri]

Great !

[geerlingguy]

Sadly, I think this is about as good as you can get there. At least we can take heart in the fact that if Docker were to change the command's UX too drastically, it would break people's brains in addition to the Ansible code, so hopefully they keep it pretty stable.

[briantist]

Looks great to me! Nice to finally have a v2 option. Shame there's no stable/machine-friendly output from the tool.

[briantist]

what in this module triggers a return syntax error?

[felixfontein]

It's the type: raw IIRC. Some older versions might also barf on choices in return values.

[felixfontein]

@lel-amri thanks for your work on this in #586!
@lel-amri @briantist @geerlingguy thanks for reviewing!

[b-reich]

@felixfontein Many thanks for your PR. Is there still the possibility for a docker compose pull or docker compose build --pull?

[felixfontein]

@b-reich right now no. I'm currently working on a docker_compose_v2_pull module which models docker compose pull. I originally wanted to have it done on last weekend, but parsing the docker compose pull output turned out to be more complex than I hoped for. I hope I can find some time this weekend to finish it.

About docker compose build: I think it's also a good idea to have that as a separate module, and I can try taking a look at it. It would be great if you could create a new issue for this (a proper feature request) and sketch out one or multiple scenarios in which you (or others) would use docker compose build.

[b-reich]

@b-reich right now no. I'm currently working on a docker_compose_v2_pull module which models docker compose pull. I originally wanted to have it done on last weekend, but parsing the docker compose pull output turned out to be more complex than I hoped for. I hope I can find some time this weekend to finish it.

About docker compose build: I think it's also a good idea to have that as a separate module, and I can try taking a look at it. It would be great if you could create a new issue for this (a proper feature request) and sketch out one or multiple scenarios in which you (or others) would use docker compose build.

Ahh I understand. If there are things I can support, please ping me.
Is there any advantages over creating an extra module over the v1 behaviour as parameter?
The Issue #750

[felixfontein]

Ahh I understand. If there are things I can support, please ping me.

Thanks!

Is there any advantages over creating an extra module over the v1 behaviour as parameter?

Short answer: look at docker_image and compare its usage to the new docker_image_* modules.

Long answer: it reduces the complexity, both of the code and of the user interface (the argument spec). This benefits both the user (less bugs, documentation easier to understand since the amount of options per module is small, you don't have to wonder whether option force now applies to pulling, starting, stopping, building, removing, or multiple of these, ...) and contributors (the code is simpler, there are no potentially surprising interactions between different 'modes' of the same module that you didn't think of, ...). Also it makes handling check mode easier, since it's easy to document which operations support check mode and which don't (push doesn't, pull only partially, build and tag support it fully - so what should the documentation for docker_image say? you don't want to read a wall of text and compare it to every of your docker_image tasks to figure out which one work fine in check mode and which don't).

There are some cases where a combined module makes things easier to express (like for docker_image: you can build, tag, and push the tagged image in one single task!). I hope that there are no cases where separate modules make something impossible or something common very inefficient. For docker_compose_v2 and docker_compose_v2_pull (and of course for docker_image_*), I'm pretty sure that this is the case. For docker_compose_v2_build it's kind of a guess, that's why I ask for use-cases to hopefully notice if a separate module is a bad idea :)

[felixfontein]

(Just look at https://docs.ansible.com/ansible/latest/collections/community/docker/docker_image_module.html#parameters and note that originally the options grouped in build were top-level options as well. Of course in the beginning there were a lot less of them, and it didn't look so bad, but then someone requests a new option x for building, then someone a new option y, and eventually you end up with 20 options just specificly for building images. Then you have five more for tagging, four for pulling, three for pushing, and you end up with a large soup of parameters. Obviously the numbers I mentioned here are a bit exaggerated, but they're not that far from the truth, especially if you look at docker build --help and start looking for things docker_image doesn't yet support.)

[felixfontein]

PR for docker_compose_v2_pull module: #751

[pgassmann]

@felixfontein separate modules for compose, pull and build are ok for me.

Another option is to have basic pull and build flag options in the standard docker_compose_v2 module, and for more special cases, have a separate pull/build module.

This matches the compose command, the compose up comand has --build and --pull options, and there are separate compose pull and compose build commands with more options.

I just need an way to update services with locally built images for regular maintenance updates.

I now see, that the pull option is already implemented for the docker_compose_v2 module.

Separate Issue for the build feature: #750

[pgassmann]

the module currently does not have support for --diff.
But the return values contain a nice list of actions.

@felixfontein Would it be possible to display those in diff mode? or are there strict conventions for diff output?

I now implemented a workaround with a second task:

- name: docker compose up ({{ webserver_domain }} in {{ webserver_path }})
  community.docker.docker_compose_v2:
    project_src: '{{ webserver_path }}'
    state: present
    remove_orphans: yes
    pull: "{{ webserver_docker_pull }}"
    build: "{{ webserver_docker_pull }}"
  register: webserver__register_compose

- name: print actions by compose
  vars:
    actions: "{{ webserver__register_compose.actions }}"
  debug:
    var: actions
  when: webserver__register_compose.changed
  changed_when: webserver__register_compose.changed

This generates the following output. also works in check mode:

TASK [teamapps.general.webserver : docker compose up (www-dev.example.com) in /container/contao)] *******
changed: [website-dev1]

TASK [teamapps.general.webserver : print actions by compose] ***********
changed: [website-dev1] => 
  actions:
  - id: contao-sftp-1
    status: Starting
    what: container

[felixfontein]

@pgassmann I guess the question is what you expect from diff mode :) I agree that the actions are better than nothing, but it's not really what I personally would expect (I would expect it to tell me which configuration changes trigger the actions - but that's not so easy to retrieve). Let's maybe create an issue and collect ideas.

[goldyfruit]

@felixfontein what is the equivalent of the files argument with the new module?

Previously I was doing like this:

  community.docker.docker_compose:
    project_src: "{{ _ovos_installer_working_directory }}/{{ _ovos_installer_project_name }}/compose"
    project_name: "{{ _project_name }}"
    files: "{{ _composition_files }}"
    pull: true
    remove_orphans: "{{ ovos_installer_docker_compose_remove_orphans }}"
    remove_images: "{{ ovos_installer_docker_compose_remove_images }}"
    remove_volumes: "{{ ovos_installer_docker_compose_remove_volumes }}"

[felixfontein]

@goldyfruit there is no equivalent at the moment. I've never used that feature nor saw it being used, so I didn't want to implement it in the first version to avoid screwing it up ;) Please create an issue (Feature Request) for it.

[goldyfruit]

@goldyfruit there is no equivalent at the moment. I've never used that feature nor saw it being used, so I didn't want to implement it in the first version to avoid screwing it up ;) Please create an issue (Feature Request) for it.

Feature request created. 👍

[felixfontein]

If someone is interested, #776 adds support for a scale option to docker_compose_v2.

[pgassmann]

Ansible 9.2.0 was released yesterday, containing the new docker_compose_v2 modules: https://github.com/ansible-community/ansible-build-data/blob/main/9/CHANGELOG-v9.rst

BTW: Is there anywhere a better readable, rendered version of this Changelog for ansible-community?

[felixfontein]

BTW: Is there anywhere a better readable, rendered version of this Changelog for ansible-community?

@pgassmann not yet, I'm currently working of creating a MD version, I'll hopefully have time to get a first working version done on Friday. I'm currently running very low on spare time for any open source work, but from Friday on it will be better :)

(If you're interested: ansible-community/antsibull-changelog#139)

In the meantime you can complain to GitHub which screwed up the RST preview of quite many changelogs and other RST documents (https://github.com/orgs/community/discussions/86715).

[pgassmann]

@felixfontein don't apologize for not having much time for open source work. You are the most active and helpful contributor on the issues that affect me. It's telling, that you are the one who already implemented a a solution that fixes the issue in my question.
Thank you for all your efforts, I appreciate your support!

Shouldn't the changelog be part of the rendered website or a docs site like:
https://ansible-community-website.readthedocs.io/
or https://docs.ansible.com

[felixfontein]

The Ansible changelog traditionally always was on git only and was never part of the docsite (docs.ansible.com); I don't know the reasons for that. One possible reason might be that the docsite build takes quite some time, so it's usually only updated after the actual release happens, which can be confusing since for some time (a few hours at most) the changelog wouldn't mention the latest version.

In any case, here's a markdown version of the Ansible 9 changelog: https://github.com/felixfontein/ansible-build-data/blob/markdown/9/CHANGELOG-v9.md (PR: ansible-community/ansible-build-data#364). And here's the community.docker changelog as markdown: https://github.com/felixfontein/community.docker/blob/changelog/CHANGELOG.md (PR: #788).

[briantist]

I think it would be worthwhile to publish the changelog if possible. Individual collections can already do this fairly easily if they want to with a symlink and a small change, see:

• https://docs.ansible.com/ansible/devel/collections/community/hashi_vault/docsite/CHANGELOG.html
• https://github.com/ansible-collections/community.hashi_vault/blob/main/docs/docsite/extra-docs.yml#L5

But I can't speak to what this process would be for the community package as a whole, nor how delayed its publishing might be.

The collection-level published changelog could be added with a small PR for anyone who's interested in doing so!