Compatibiliy azure-cli and the azure collection

[dtometzki]

SUMMARY

The ansible-collection and the azure-cli in on eviroment are incombatible. There are conflicts in many azure-mgmt* packages.
Is it a way to install both in one enviroment

ISSUE TYPE

• Bug

COMPONENT NAME

azure-mgmt* release depencies

ANSIBLE VERSION

ansible [core 2.11.5]
config file = None
configured module search path = ['/home/damian/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /home/damian/.local/lib/python3.10/site-packages/ansible
ansible collection location = /home/damian/.ansible/collections:/usr/share/ansible/collections
executable location = /home/damian/.local/bin/ansible
python version = 3.10.0 (default, Oct 4 2021, 00:00:00) [GCC 11.2.1 20210728 (Red Hat 11.2.1-1)]
jinja version = 3.0.2
libyaml = True

##### COLLECTION VERSION
<!--- Paste verbatim output from "ansible-galaxy collection list <namespace>.<collection>"  between the quotes
for example: ansible-galaxy collection list community.general
-->
```paste below

CONFIGURATION

# /usr/local/lib/python3.10/site-packages/ansible_collections
Collection        Version
----------------- -------
community.general 3.7.0  

OS / ENVIRONMENT

Fedora 35
python 3.10

STEPS TO REPRODUCE

Install first with:
pip install azure-cli
pip install -r requirements-azure.txt

Then try if az version works ?

EXPECTED RESULTS

ACTUAL RESULTS

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
pygithub 1.55 requires pyjwt>=2.0, but you have pyjwt 1.7.1 which is incompatible.
azure-identity 1.6.1 requires msal<2.0.0,>=1.7.0, but you have msal 1.0.0 which is incompatible.
azure-identity 1.6.1 requires msal-extensions~=0.3.0, but you have msal-extensions 0.1.3 which is incompatible.
azure-cli 2.28.1 requires azure-cli-core==2.28.1, but you have azure-cli-core 2.11.1 which is incompatible.
azure-cli 2.28.1 requires azure-graphrbac~=0.60.0, but you have azure-graphrbac 0.61.1 which is incompatible.
azure-cli 2.28.1 requires azure-keyvault~=1.1.0, but you have azure-keyvault 1.0.0a1 which is incompatible.
azure-cli 2.28.1 requires azure-mgmt-authorization~=0.61.0, but you have azure-mgmt-authorization 0.51.1 which is incompatible.
azure-cli 2.28.1 requires azure-mgmt-batch~=16.0.0, but you have azure-mgmt-batch 5.0.1 which is incompatible.
azure-cli 2.28.1 requires azure-mgmt-cdn==11.0.0, but you have azure-mgmt-cdn 3.0.0 which is incompatible.
azure-cli 2.28.1 requires azure-mgmt-compute~=22.1.0, but you have azure-mgmt-compute 10.0.0 which is incompatible.
azure-cli 2.28.1 requires azure-mgmt-containerinstance~=8.0.0, but you have azure-mgmt-containerinstance 1.4.0 which is incompatible.
azure-cli 2.28.1 requires azure-mgmt-containerregistry==8.1.0, but you have azure-mgmt-containerregistry 2.0.0 which is incompatible.
azure-cli 2.28.1 requires azure-mgmt-containerservice~=16.1.0, but you have azure-mgmt-containerservice 9.1.0 which is incompatible.
azure-cli 2.28.1 requires azure-mgmt-cosmosdb~=6.4.0, but you have azure-mgmt-cosmosdb 0.5.2 which is incompatible.
azure-cli 2.28.1 requires azure-mgmt-devtestlabs~=4.0, but you have azure-mgmt-devtestlabs 3.0.0 which is incompatible.
azure-cli 2.28.1 requires azure-mgmt-dns~=8.0.0, but you have azure-mgmt-dns 2.1.0 which is incompatible.
azure-cli 2.28.1 requires azure-mgmt-eventhub~=4.1.0, but you have azure-mgmt-eventhub 2.0.0 which is incompatible.
azure-cli 2.28.1 requires azure-mgmt-hdinsight~=8.0.0, but you have azure-mgmt-hdinsight 0.1.0 which is incompatible.
azure-cli 2.28.1 requires azure-mgmt-iothub==2.1.0, but you have azure-mgmt-iothub 0.7.0 which is incompatible.
azure-cli 2.28.1 requires azure-mgmt-keyvault==9.1.0, but you have azure-mgmt-keyvault 1.1.0 which is incompatible.
azure-cli 2.28.1 requires azure-mgmt-loganalytics~=11.0.0, but you have azure-mgmt-loganalytics 1.0.0 which is incompatible.
azure-cli 2.28.1 requires azure-mgmt-marketplaceordering==1.1.0, but you have azure-mgmt-marketplaceordering 0.1.0 which is incompatible.
azure-cli 2.28.1 requires azure-mgmt-monitor~=2.0.0, but you have azure-mgmt-monitor 0.5.2 which is incompatible.
azure-cli 2.28.1 requires azure-mgmt-network~=19.0.0, but you have azure-mgmt-network 12.0.0 which is incompatible.
azure-cli 2.28.1 requires azure-mgmt-privatedns~=1.0.0, but you have azure-mgmt-privatedns 0.1.0 which is incompatible.
azure-cli 2.28.1 requires azure-mgmt-rdbms~=9.1.0b1, but you have azure-mgmt-rdbms 1.9.0 which is incompatible.
azure-cli 2.28.1 requires azure-mgmt-recoveryservices~=2.0.0, but you have azure-mgmt-recoveryservices 0.4.0 which is incompatible.
azure-cli 2.28.1 requires azure-mgmt-recoveryservicesbackup~=0.14.0, but you have azure-mgmt-recoveryservicesbackup 0.6.0 which is incompatible.
azure-cli 2.28.1 requires azure-mgmt-redis~=13.0.0, but you have azure-mgmt-redis 5.0.0 which is incompatible.
azure-cli 2.28.1 requires azure-mgmt-resource==19.0.0, but you have azure-mgmt-resource 10.2.0 which is incompatible.
azure-cli 2.28.1 requires azure-mgmt-search~=8.0, but you have azure-mgmt-search 3.0.0 which is incompatible.
azure-cli 2.28.1 requires azure-mgmt-servicebus~=6.0.0, but you have azure-mgmt-servicebus 0.5.3 which is incompatible.
azure-cli 2.28.1 requires azure-mgmt-sql~=3.0.1, but you have azure-mgmt-sql 0.10.0 which is incompatible.
azure-cli 2.28.1 requires azure-mgmt-storage~=18.0.0, but you have azure-mgmt-storage 11.1.0 which is incompatible.
azure-cli 2.28.1 requires azure-mgmt-trafficmanager~=0.51.0, but you have azure-mgmt-trafficmanager 0.50.0 which is incompatible.
azure-cli 2.28.1 requires azure-mgmt-web~=2.0.0, but you have azure-mgmt-web 0.41.0 which is incompatible.

[Fred-sun]

@dtometzki Because you are using a newer version of Azure CLI Core, the other packages require the latest version as well. If you want to use it now, install the version specified in requirements-azure. In addition, we are planning to upgrade all dependent versions. It is already in our schedule! Thank you very much!

[dtometzki]

Hello @Fred-sun,

many thanks azure-cli 2.11.1 is an very old version. But do have any time schedules when the update is ready ?
Many Thanks
Damian

[Fred-sun]

Hello @Fred-sun,

many thanks azure-cli 2.11.1 is an very old version. But do have any time schedules when the update is ready ? Many Thanks Damian

@dtometzki We have planned to upgrade, the specific completion date is yet to be determined. Thank you very much!

[l3ender]

Duplicate of #477.

[dtometzki]

Yes perhaps duplicate but without solution yet :-)

[l3ender]

azure-cli-core was updated to 2.34.0 in #775; does that resolve this issue? It was released in v1.12.0.

[krapgras]

As this is unfortunately not yet fixed and most likely always be a challenge with the different development speeds i will share my fix.

Because i have been struggling on this as well and everyone talks about virtenv but nobody really shows how to use it.

What I did is 3 step, I created 2 tasks handling the azure-cli (creation + destruction) and then your main task(s) where you can use the Azure cli commands.

Role "install_azure-cli_venv"

---
# tasks file for roles/install_azure-cli_venv
- name: "Install azure-cli in {{ working_directory }}/venv"
  ansible.builtin.pip:
    name: azure-cli
    virtualenv: "/tmp/{{ inventory_hostname }}/venv"
    virtualenv_python: python3
    state: latest

Role 'main task'

---
# tasks file for roles/Run_AZ_Command
- name: Run AZ Command
  ansible.builtin.shell: |
    set -e
    source venv/bin/activate
      az command goes here                
  args:
    executable: /bin/bash
    chdir: "/tmp/{{ inventory_hostname }}"

Role 'remove_azure-cli_venv'

---
# tasks file for roles/remove_azure-cli_venv
- name: Remove python virtual env
  ansible.builtin.file:
    path: "/tmp/{{ inventory_hostname }}/venv"
    state: absent

I hope this helps someone struggling with the same :)
I also agree that this is every time some extra space/data download but it's not the worst as you will only get the basic python3 setup and you will only download azure-cli with deps.

[Fred-sun]

@dtometzki I escalated all my dependencies without resolving the conflict between them. For the time being, you can install both in separate virtual environments on the same server, thank you!

[Fred-sun]

@dtometzki Also, We will close this issue and continue this issue in #477 (Duplicate)! Thanks!

[Fred-sun]

@dtometzki Since there will be conflict between azure-cli and requirements-azure.txt when installed together, it is recommended to directly install the specified version of azure-cli to effectively avoid this problem, thank you!

Like this:
1. sudo pip3 install ansible
2. ansible-galaxy collection install azure.azcollection
3. pip3 install azure-cli==2.34.0
4. pip3 install -r ~/.ansible/collections/ansible_collections/azure/azcollection/requirements-azure.txt

Or: 
1. sudo pip3 install ansible
2. ansible-galaxy collection install azure.azcollection
3. pip3 install -r ~/.ansible/collections/ansible_collections/azure/azcollection/requirements-azure.txt
4. pip3 install azure-cli==2.34.0