Az ml extension fails with ERROR: The extension ml already exists

[srinisudharsan]

Describe the bug

The same command was working until yesterday.
Az cli version 2.52.0
az ml version 2.30.0

Command run:
az ml online-endpoint show --subscription _{--resource-group --workspace --name --only-show-errors}

cannot import name '_T' from 'marshmallow.fields' (/opt/az/azcliextensions/ml/marshmallow/fields.py)

STDERR: ERROR: An error occurred whilst updating.
ERROR: The extension ml already exists.
ERROR: The command failed with an unexpected error. Here is the traceback:
ERROR: [Errno 17] File exists: '/opt/az/azcliextensions/ml'
Traceback (most recent call last):
File "/opt/az/lib/python3.10/site-packages/azure/cli/core/extension/operations.py", line 429, in update_extension
_add_whl_ext(cli_ctx=cmd_cli_ctx, source=download_url, ext_sha256=ext_sha256,
File "/opt/az/lib/python3.10/site-packages/azure/cli/core/extension/operations.py", line 113, in _add_whl_ext
raise CLIError('The extension {} already exists.'.format(extension_name))
knack.util.CLIError: The extension ml already exists.

Related command

az ml online-endpoint show --subscription _{--resource-group --workspace --name --only-show-errors}

Errors

cannot import name '_T' from 'marshmallow.fields' (/opt/az/azcliextensions/ml/marshmallow/fields.py)

STDERR: ERROR: An error occurred whilst updating.
ERROR: The extension ml already exists.
ERROR: The command failed with an unexpected error. Here is the traceback:
ERROR: [Errno 17] File exists: '/opt/az/azcliextensions/ml'
Traceback (most recent call last):
File "/opt/az/lib/python3.10/site-packages/azure/cli/core/extension/operations.py", line 429, in update_extension
_add_whl_ext(cli_ctx=cmd_cli_ctx, source=download_url, ext_sha256=ext_sha256,
File "/opt/az/lib/python3.10/site-packages/azure/cli/core/extension/operations.py", line 113, in _add_whl_ext
raise CLIError('The extension {} already exists.'.format(extension_name))
knack.util.CLIError: The extension ml already exists.

Issue script & Debug output

Command run:
az ml online-endpoint show --subscription _{--resource-group --workspace --name --only-show-errors}

cannot import name '_T' from 'marshmallow.fields' (/opt/az/azcliextensions/ml/marshmallow/fields.py)

STDERR: ERROR: An error occurred whilst updating.
ERROR: The extension ml already exists.
ERROR: The command failed with an unexpected error. Here is the traceback:
ERROR: [Errno 17] File exists: '/opt/az/azcliextensions/ml'
Traceback (most recent call last):
File "/opt/az/lib/python3.10/site-packages/azure/cli/core/extension/operations.py", line 429, in update_extension
_add_whl_ext(cli_ctx=cmd_cli_ctx, source=download_url, ext_sha256=ext_sha256,
File "/opt/az/lib/python3.10/site-packages/azure/cli/core/extension/operations.py", line 113, in _add_whl_ext
raise CLIError('The extension {} already exists.'.format(extension_name))
knack.util.CLIError: The extension ml already exists.

Expected behavior

Show the az ml endpoint

Environment Summary

Az cli version 2.52.0
az ml version 2.30.0
OS ubuntu 22.04

Additional context

No response

[yonzhan]

Thank you for opening this issue, we will look into it.

[microsoft-github-policy-service]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @azureml-github.

[srinisudharsan]

May be related to
microsoft/promptflow#3902

[singankit]

Thanks @srinisudharsan for raising the issue. The issue is already being tracked and fix is being worked on. Will update the bug once fix is released.

[srinisudharsan]

@singankit could you please provide the issue where it is being tracked

[frank-dong-ms]

@singankit this issue is blocking our production pipeline, is there any workaround for this issue? Like can we pin azure cli version or ml extension version anything like that?

[singankit]

@frank-dong-ms downgrading marshmallow will certainly help. Following command can be used as workaround:

pip install "marshmallow<3.24.0"

[frank-dong-ms]

@singankit thanks, does that mean error "knack.util.CLIError: The extension ml already exists." is also caused by marshmallow upgrade?

[singankit]

@frank-dong-ms Actual error is from marshmallow which is causing ml extension failure at load time.

[MetelStairs]

Any update on this? It is currently halting our production pipelines, forcing our pipeline to pip install marshmallow==3.23.2 does not fix the issue

[davidblom603]

This is also blocking our production deployment pipeline. Any updates on how to resolve this? We are using GitHub Actions to deploy to Azure Machine Learning with the ubuntu-latest runner. The suggestion to run pip install marshmallow==3.23.2 doesn't seem to help.

[EmreCem]

@MetelStairs @davidblom603 Make sure you are targeting the correct path when running the pip install command. You need to specify the extensions directory. For example for ubuntu-latest by default, you would need to run:
pip install marshmallow==3.23.2 --target /opt/az/azcliextensions/ml --upgrade
after installing the ml extension.

[davidblom603]

@EmreCem Thanks! After adding --upgrade, the deployment was successful.

az extension add -n ml
pip install marshmallow==3.23.2 --target /opt/az/azcliextensions/ml --upgrade

[jsilvcast]

@EmreCem Thank you! It works for me too.

@MetelStairs @davidblom603 Make sure you are targeting the correct path when running the pip install command. You need to specify the extensions directory. For example for ubuntu-latest by default, you would need to run: pip install marshmallow==3.23.2 --target /opt/az/azcliextensions/ml --upgrade after installing the ml extension.

[EmreCem]

@singankit Are there any updates on this issue? I see that a fix was already released for the Python SDK yesterday: Azure/azure-sdk-for-python#39053
When can we expect a fix for the CLI extension?