Skip to content
This repository has been archived by the owner on Aug 2, 2023. It is now read-only.

ImportError: cannot import name 'AnsibleCollectionLoader' from 'ansible.utils.collection_loader' #41

Closed
darthlukan opened this issue Oct 12, 2020 · 17 comments

Comments

@darthlukan
Copy link

As the title suggests, we're running into an issue whereby ansible-lint is unable to execute because of an import error. I'm unsure if this is an issue in our workflow.yaml, ansible-lint-action, or ansible-lint itself.

We noticed the issue 5 days ago but didn't look into it. Here is the output of the step in our workflow:

Run ansible/ansible-lint-action@master
  with:
    targets: ansible/playbook.yaml
  ansible/participants-setup.yaml
  ansible/participants-deploy.yaml
  
    override-deps: ansible==2.9
/usr/bin/docker run --name e5c3525f869eade1344f8bb774ab1a3f111c3_cea4b0 --label 1e5c35 --workdir /github/workspace --rm -e INPUT_TARGETS -e INPUT_OVERRIDE-DEPS -e INPUT_ARGS -e TARGETS -e OVERRIDE -e HOME -e GITHUB_JOB -e GITHUB_REF -e GITHUB_SHA -e GITHUB_REPOSITORY -e GITHUB_REPOSITORY_OWNER -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RETENTION_DAYS -e GITHUB_ACTOR -e GITHUB_WORKFLOW -e GITHUB_HEAD_REF -e GITHUB_BASE_REF -e GITHUB_EVENT_NAME -e GITHUB_SERVER_URL -e GITHUB_API_URL -e GITHUB_GRAPHQL_URL -e GITHUB_WORKSPACE -e GITHUB_ACTION -e GITHUB_EVENT_PATH -e GITHUB_PATH -e GITHUB_ENV -e RUNNER_OS -e RUNNER_TOOL_CACHE -e RUNNER_TEMP -e RUNNER_WORKSPACE -e ACTIONS_RUNTIME_URL -e ACTIONS_RUNTIME_TOKEN -e ACTIONS_CACHE_URL -e GITHUB_ACTIONS=true -e CI=true -v "/var/run/docker.sock":"/var/run/docker.sock" -v "/home/runner/work/_temp/_github_home":"/github/home" -v "/home/runner/work/_temp/_github_workflow":"/github/workflow" -v "/home/runner/work/_temp/_runner_file_commands":"/github/file_commands" -v "/home/runner/work/gitops-workshop/gitops-workshop":"/github/workspace" 1e5c35:25f869eade1344f8bb774ab1a3f111c3  ""
+ args=("$@")
+ '[' /entrypoint.sh = /entrypoint.sh ']'
+ echo -E '\nRunning Ansible Lint...\n'
\nRunning Ansible Lint...\n
+ ansible::lint ''
+ : 'ansible/playbook.yaml
ansible/participants-setup.yaml
ansible/participants-deploy.yaml
'
+ : /github/workspace
+ pushd /github/workspace
+ override_python_packages
+ [[ -n ansible==2.9 ]]
+ pip install ansible==2.9
/github/workspace /github/workspace
WARNING: The directory '/github/home/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Collecting ansible==2.9
  Downloading ansible-2.9.0.tar.gz (14.1 MB)
Requirement already satisfied: jinja2 in /usr/local/lib/python3.8/site-packages (from ansible==2.9) (2.11.2)
Requirement already satisfied: PyYAML in /usr/local/lib/python3.8/site-packages (from ansible==2.9) (5.3.1)
Requirement already satisfied: cryptography in /usr/local/lib/python3.8/site-packages (from ansible==2.9) (3.1.1)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python3.8/site-packages (from jinja2->ansible==2.9) (1.1.1)
Requirement already satisfied: six>=1.4.1 in /usr/local/lib/python3.8/site-packages (from cryptography->ansible==2.9) (1.15.0)
Requirement already satisfied: cffi!=1.11.3,>=1.8 in /usr/local/lib/python3.8/site-packages (from cryptography->ansible==2.9) (1.14.3)
Requirement already satisfied: pycparser in /usr/local/lib/python3.8/site-packages (from cffi!=1.11.3,>=1.8->cryptography->ansible==2.9) (2.20)
Building wheels for collected packages: ansible
  Building wheel for ansible (setup.py): started
  Building wheel for ansible (setup.py): finished with status 'done'
  Created wheel for ansible: filename=ansible-2.9.0-py3-none-any.whl size=16130179 sha256=7ecff50a9b01a2966b8f7add0152a27951566ad9565b0a45f1bd45b3534bd0e8
  Stored in directory: /tmp/pip-ephem-wheel-cache-5sfflf9p/wheels/d8/87/fb/65b7fd9034ef5f109d906139de22ea10259f98cbf5c4f73616
Successfully built ansible
Installing collected packages: ansible
  Attempting uninstall: ansible
    Found existing installation: ansible 2.10.0
    Uninstalling ansible-2.10.0:
      Successfully uninstalled ansible-2.10.0
Successfully installed ansible-2.9.0
+ pip check
WARNING: The directory '/github/home/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
No broken requirements found.
+ echo 'Completed installing override dependencies...'
Completed installing override dependencies...
+ local opts
++ parse_args
++ local opts=
++ ((  0  ))
++ eval set -- ''
+++ set --
++ echo ''
++ return 0
+ opts=
+ shopt -s globstar
+ ansible-lint -v --force-color ansible/playbook.yaml ansible/participants-setup.yaml ansible/participants-deploy.yaml
Traceback (most recent call last):
  File "/usr/local/bin/ansible-lint", line 5, in <module>
    from ansiblelint.__main__ import main
  File "/usr/local/lib/python3.8/site-packages/ansiblelint/__init__.py", line 22, in <module>
    from ansiblelint.rules import AnsibleLintRule
  File "/usr/local/lib/python3.8/site-packages/ansiblelint/rules/__init__.py", line 12, in <module>
    import ansiblelint.utils
  File "/usr/local/lib/python3.8/site-packages/ansiblelint/utils.py", line 38, in <module>
    from ansible.parsing.mod_args import ModuleArgsParser
  File "/usr/local/lib/python3.8/site-packages/ansible/parsing/mod_args.py", line 26, in <module>
    from ansible.plugins.loader import module_loader, action_loader
  File "/usr/local/lib/python3.8/site-packages/ansible/plugins/loader.py", line 25, in <module>
    from ansible.utils.collection_loader import AnsibleCollectionLoader, AnsibleFlatMapLoader, AnsibleCollectionRef
ImportError: cannot import name 'AnsibleCollectionLoader' from 'ansible.utils.collection_loader' (/usr/local/lib/python3.8/site-packages/ansible/utils/collection_loader/__init__.py)

And here is the content of our .github/workflows/ansible-lint.yml:

name: Ansible Lint

on:
  push:
    branches: [ master ]
  pull_request:
    branches: [ master ]

jobs:
  build:
    runs-on: ubuntu-latest
    # Steps represent a sequence of tasks that will be executed as part of the job
    steps:
    # Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
    - uses: actions/checkout@v2
    # Run ansible-lint
    - name: ansible-lint
      uses: ansible/ansible-lint-action@master
      with:
        targets: |
          ansible/playbook.yaml
          ansible/participants-setup.yaml
          ansible/participants-deploy.yaml
        override-deps: ansible==2.9
        args: ""
        # args: |
        #   --exclude=ansible/files
        #   --exclude=ansible/vars
        #   --exclude=ansible/templates
        #   --exclude=ansible/requirements.yaml
@vitabaks
Copy link

I have the same problem if using override-deps: ansible==<other_ansible_version>
https://github.com/vitabaks/postgresql_cluster/runs/1264526883
https://github.com/vitabaks/postgresql_cluster/blob/master/.github/workflows/ansible-lint.yml

vitabaks added a commit to vitabaks/autobase that referenced this issue Oct 16, 2020
Ansible lint actions are currently broken for overrides of the Ansible version.  Disable until fixed.

ansible/ansible-lint-action#41
@SturmB
Copy link

SturmB commented Nov 10, 2020

I am still using Ansible v2.9 and can report that I am affected by this bug, too. I'm just using the defaults in the ansible-lint.yml file, too; no overriding going on of which I am aware.

I just went ahead and deleted the action until I have time to learn v2.10 and upgrade to it.

@shawngmc
Copy link

shawngmc commented Dec 29, 2020

I'm running into the same issue, so I've done a bit of testing.

  • Do not specify an ansible version: Works, uses 2.10.4 from initial setup
  • Do not specify an ansible version with ubuntu-20.04: Works, uses 2.10.4 from initial setup
  • Specify ansible 2.10.4: Works, uses 2.10.4 from initial setup, override says it's satisfied
  • Specify ansible 2.10.3: Works, uses 2.10.3
  • Specify ansible 2.10.2: Works, uses 2.10.2
  • Specify ansible 2.10.1: Works, uses 2.10.1
  • Specify ansible 2.10.0: Works, uses 2.10.0
  • Specify ansible 2.10: Works, uses 2.10.0
  • Specify ansible 2.9.6: Fails with ImportError: cannot import name 'AnsibleCollectionLoader' from 'ansible.utils.collection_loader', uses 2.9.6
  • Specify ansible 2.9: Fails with ImportError: cannot import name 'AnsibleCollectionLoader' from 'ansible.utils.collection_loader', uses 2.9.0
  • Specify ansible 2.9.6 with ubuntu-20.04: Fails with ImportError: cannot import name 'AnsibleCollectionLoader' from 'ansible.utils.collection_loader', uses 2.9.6
  • Specify ansible 2.9.6 and ansible-lint 4.3.6: Fails with ImportError: cannot import name 'AnsibleCollectionLoader' from 'ansible.utils.collection_loader', uses ansible 2.9.6 and ansible-lint 4.3.6
  • Specify ansible 2.9.6 on a fork of ansible-lint-action that installs ansible 2.9.6: Works, uses ansible 2.9.6
  • Specify ansible 2.10.4 on a fork of ansible-lint-action that installs ansible 2.9.6: Quiet failure. Appears to 'work', but the ansible 2.10.4 install fails due to an error shown below. Still runs the ansible-lint, but using ansible 2.9.6.

Conclusions:

  • Specifying 2.10.x works, so the override-deps isn't completely broken.
  • I didn't expect ubuntu-20.04 to fix it, but at least the behavior hasn't changed. This is important due to Ubuntu-latest workflows will use Ubuntu-20.04 actions/runner-images#1816, but since everything is done in a python:3.8slim container, this is insulated.
  • Downgrading ansible-lint doesn't fix it - which implies that it's not just some minor reference from ansible-lint to ansible.
  • A fork of the action specifying 2.9.6 in the Dockerfile FIXES the issue, but then breaks 2.10.x, so that's not a great solution.

Idea:
I'm wondering if a better solution might be to not install ansible or ansible-lint in the Dockerfile, and only install them at run time. This sounds silly, but it would be the least-breaking change; if there is no override dep specified, add one in for 'the latest'. This would also speed up the build a good chunk for non-default versions, since you aren't installing and uninstalling the default.

That said, though I'm very experienced with Docker and not bad with Python, I'm an absolute ansible neophyte. I just started using ansible yesterday. I can try to make this change (I might also sneak in a python version upgrade, which would improve performance), but it would probably be better if someone more versed in ansible does this.

Upgrade from 2.9.x to 2.10.x error:

Collecting ansible==2.10.4
  Downloading ansible-2.10.4.tar.gz (28.6 MB)
    ERROR: Command errored out with exit status 1:
     command: /usr/local/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-_8tmb6pm/ansible_b6913abd93f741afa2a75f7673431b85/setup.py'"'"'; __file__='"'"'/tmp/pip-install-_8tmb6pm/ansible_b6913abd93f741afa2a75f7673431b85/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-8ib71dlr
         cwd: /tmp/pip-install-_8tmb6pm/ansible_b6913abd93f741afa2a75f7673431b85/
    Complete output (31 lines):
    
    
                ### ERROR ###
    
                Upgrading directly from ansible-2.9 or less to ansible-2.10 or greater with pip is
                known to cause problems.  Please uninstall the old version found at:
    
                /usr/local/lib/python3.8/site-packages/ansible/__init__.py
    
                and install the new version:
    
                    pip uninstall ansible
                    pip install ansible
    
                If you have a broken installation, perhaps because ansible-base was installed before
                ansible was upgraded, try this to resolve it:
    
                    pip install --force-reinstall ansible ansible-base
    
                If ansible is installed in a different location than you will be installing it now
                (for example, if the old version is installed by a system package manager to
                /usr/lib/python3.8/site-packages/ansible but you are installing the new version into
                ~/.local/lib/python3.8/site-packages/ansible with `pip install --user ansible`)
                or you want to install anyways and cleanup any breakage afterwards, then you may set
                the ANSIBLE_SKIP_CONFLICT_CHECK environment variable to ignore this check:
    
                    ANSIBLE_SKIP_CONFLICT_CHECK=1 pip install --user ansible
    
                ### END ERROR ###
    
    
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

@iranzo
Copy link

iranzo commented Feb 19, 2021

Facing same issue :/

@Mystic8b
Copy link

The problem is 4 months old, can someone have a temporary solution?

@iranzo
Copy link

iranzo commented Feb 19, 2021

I ended up forking the repo and merging there the patch for it that's still pending at https://github.com/iranzo/ansible-lint-action/ and added also tagging...

https://github.com/iranzo/ansible-lint-action/releases/tag/v4.1.1 is the one containing that change if you want to give it a try

@Mystic8b
Copy link

@iranzo Thank you so much! I will definitely try in a couple of days.

@JMLX42
Copy link

JMLX42 commented Jun 12, 2021

Any update in this?

@Mystic8b
Copy link

Any update in this?

I'm using solution from mr @iranzo and everything works for me. However, I can't find an explanation for why the problem has not been resolved yet and there is no answer from the developers ...

@aarighi
Copy link

aarighi commented Aug 23, 2021

Same problem here, with ansible 2.9.25

@Mystic8b
Copy link

Mystic8b commented Aug 28, 2021

ImportError: cannot import name 'AnsibleCollectionLoader' from 'ansible.utils.collection_loader' (/usr/local/lib/python3.8/site-packages/ansible/utils/collection_loader/__init__.py)

It looks like this project is dead. Maybe there are some alternatives?

@thestevenbell
Copy link

This issue came up when running playbooks requiring differing versions of ansible and ansible-core. When ansible-core is installed first and then later ansible 2.9 is installed without doing a full unistall of ansible-core, attempting to install ansible-galaxy dependencies fails. A work around is to run ansible in a virtual environment (venv or fresh docker image) for each playbook or to add logic to uninstall ansible and ansible-core and then reinstall the specific version needed for the playbook.

sudo python3 -m pip uninstall ansible ansible-core -y
sudo python3 -m pip --proxy "$HTTPS_PROXY" install ansible==2.9.26

@haneyf
Copy link

haneyf commented Oct 18, 2021

thestevenbell -- thanks! that was exactly the thing I was hitting.

jistr added a commit to jistr/os-migrate that referenced this issue Mar 23, 2022
There is an issue with ansible-lint - it fails with "ImportError:
cannot import name 'AnsibleCollectionLoader' from
'ansible.utils.collection_loader'" when Ansible is installed on the
host and we're running lint in a venv.

We're working around it by uninstalling Ansible in the `test-lint`
make target. We're checking that we run in a toolbox container before
doing the uninstall.

ansible/ansible-lint-action#41 (comment)

Co-Authored-By: Adrian Brown <[email protected]>
jistr pushed a commit to os-migrate/os-migrate that referenced this issue Mar 25, 2022
This is to work around an issue with ansible-lint:

ansible/ansible-lint-action#41
cloudguruab added a commit to os-migrate/os-migrate that referenced this issue Mar 27, 2022
* New: Customize availbility zone for conversion hosts

It is now possible to customize conversion host availability zones via
these new variables:

os_migrate_src_conversion_availability_zone
os_migrate_dst_conversion_availability_zone

* Dev: Don't run lints with Ansible 2.9

This is to work around an issue with ansible-lint:

ansible/ansible-lint-action#41

* Dev: Refactor argument ordering to satisfy pylint

We shouldn't change the ordering of arguments of create_or_update when
subclassing Resource, we should just add a new parameter at the end.

* dev: allow vagrant on headless servers

* dev: update vagrant to fedora35

horizon was broken on fedora35, dont need so it is disabled.

* dev: documentation update for variables in e2e setup

Co-authored-by: Jiri Stransky <[email protected]>
@Jmennius
Copy link

Jmennius commented Apr 19, 2022

Bumped into this while trying to use ansible-vault or ansible-lint.
I have Ansible 2.9 locked in dependency manager (because of mitogen) but ansible-lint was not locked to a specific version. When ansible-lint was updated to v6* (which is not compatible with ansible 2.9) - it pulled in ansible-core which broke everything. Solved my locking ansible-lint to v5 (and making sure dependency manager removes unneeded packages).

@roelvanduijnhoven
Copy link

roelvanduijnhoven commented Jun 22, 2022

For me it was solved by uninstalling ansible-base, and afterwards installing the target version. I did not have ansible-core installed. So: for anybody looking for somebody to copy paste in their terminal; this is what I would recommend:

sudo python3 -m pip uninstall ansible ansible-base ansible-core

Followed by whatever version you want to install. In my case:

sudo python3 -m pip install ansible==2.8

@jottr
Copy link

jottr commented Jun 22, 2022

Dear maintainers, please merge the patch proposed earlier in the thread or tell us why you can't merge it.

Thanks!

Ping @ssbarnea

@ssbarnea
Copy link
Member

We have no plans to support more than latest version of ansible-lint and ansible-core with this action.

The newer version of this action does not allow user to install a custom version of ansible or linter because we are using a pre-compiled container image with the tools. We have no container image building here.

I will close this thread because it is too old and loaded with information that no longer applies.

@ansible ansible locked as off-topic and limited conversation to collaborators Jun 24, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.