Skip to content

Conversation

@evelyn-ys
Copy link
Member

@evelyn-ys evelyn-ys commented May 28, 2025

Related command

Description

Fix #31547

ARM endpoint API $ARM_HOST/metadata/endpoints may return different formats for different API versions and different clouds. For example:

  • https://management.azure.com/metadata/endpoints?api-version=2015-01-01 returns a json object without suffixes
{"galleryEndpoint":"https://gallery.azure.com/","graphEndpoint":"https://graph.windows.net/","portalEndpoint":"https://portal.azure.com/","authentication":{"loginEndpoint":"https://login.microsoftonline.com/","audiences":["https://management.core.windows.net/","https://management.azure.com/"]}}
  • https://management.azure.com/metadata/endpoints?api-version=2019-10-01 returns a json array with suffixes
[{"portal":"https://portal.azure.com","authentication":{"loginEndpoint":"https://login.microsoftonline.com/","audiences":["https://management.core.windows.net/","https://management.azure.com/"],"tenant":"common","identityProvider":"AAD"},"media":"https://rest.media.azure.net","graphAudience":"https://graph.windows.net/","graph":"https://graph.windows.net/","name":"AzureCloud","suffixes":{"azureDataLakeStoreFileSystem":"azuredatalakestore.net","acrLoginServer":"azurecr.io","sqlServerHostname":"database.windows.net","azureDataLakeAnalyticsCatalogAndJob":"azuredatalakeanalytics.net","keyVaultDns":"vault.azure.net","storage":"core.windows.net","azureFrontDoorEndpointSuffix":"azurefd.net"},"batch":"https://batch.core.windows.net/","resourceManager":"https://management.azure.com/","vmImageAliasDoc":"https://raw.githubusercontent.com/Azure/azure-rest-api-specs/master/arm-compute/quickstart-templates/aliases.json","activeDirectoryDataLake":"https://datalake.azure.net/","sqlManagement":"https://management.core.windows.net:8443/","gallery":"https://gallery.azure.com/"},{"portal":"https://portal.azure.cn","authentication":{"loginEndpoint":"https://login.chinacloudapi.cn","audiences":["https://management.core.chinacloudapi.cn","https://management.chinacloudapi.cn"],"tenant":"common","identityProvider":"AAD"},"media":"https://rest.media.chinacloudapi.cn","graphAudience":"https://graph.chinacloudapi.cn","graph":"https://graph.chinacloudapi.cn","name":"AzureChinaCloud","suffixes":{"acrLoginServer":"azurecr.cn","sqlServerHostname":"database.chinacloudapi.cn","keyVaultDns":"vault.azure.cn","storage":"core.chinacloudapi.cn","azureFrontDoorEndpointSuffix":""},"batch":"https://batch.chinacloudapi.cn","resourceManager":"https://management.chinacloudapi.cn","vmImageAliasDoc":"https://raw.githubusercontent.com/Azure/azure-rest-api-specs/master/arm-compute/quickstart-templates/aliases.json","sqlManagement":"https://management.core.chinacloudapi.cn:8443","gallery":"https://gallery.chinacloudapi.cn"},{"portal":"https://portal.azure.us","authentication":{"loginEndpoint":"https://login.microsoftonline.us","audiences":["https://management.core.usgovcloudapi.net","https://management.usgovcloudapi.net"],"tenant":"common","identityProvider":"AAD"},"media":"https://rest.media.usgovcloudapi.net","graphAudience":"https://graph.windows.net","graph":"https://graph.windows.net","name":"AzureUSGovernment","suffixes":{"acrLoginServer":"azurecr.us","sqlServerHostname":"database.usgovcloudapi.net","keyVaultDns":"vault.usgovcloudapi.net","storage":"core.usgovcloudapi.net","azureFrontDoorEndpointSuffix":""},"batch":"https://batch.core.usgovcloudapi.net","resourceManager":"https://management.usgovcloudapi.net","vmImageAliasDoc":"https://raw.githubusercontent.com/Azure/azure-rest-api-specs/master/arm-compute/quickstart-templates/aliases.json","sqlManagement":"https://management.core.usgovcloudapi.net:8443","gallery":"https://gallery.usgovcloudapi.net"}]
  • https://management.azure.com/metadata/endpoints?api-version=2022-09-01 returns a json object with suffixes
{"portal":"https://portal.azure.com","authentication":{"loginEndpoint":"https://login.microsoftonline.com","audiences":["https://management.core.windows.net/","https://management.azure.com/"],"tenant":"common","identityProvider":"AAD"},"media":"https://rest.media.azure.net","graphAudience":"https://graph.windows.net/","graph":"https://graph.windows.net/","name":"AzureCloud","suffixes":{"azureDataLakeStoreFileSystem":"azuredatalakestore.net","acrLoginServer":"azurecr.io","sqlServerHostname":"database.windows.net","azureDataLakeAnalyticsCatalogAndJob":"azuredatalakeanalytics.net","keyVaultDns":"vault.azure.net","storage":"core.windows.net","azureFrontDoorEndpointSuffix":"azurefd.net","storageSyncEndpointSuffix":"afs.azure.net","mhsmDns":"managedhsm.azure.net","mysqlServerEndpoint":"mysql.database.azure.com","postgresqlServerEndpoint":"postgres.database.azure.com","mariadbServerEndpoint":"mariadb.database.azure.com","synapseAnalytics":"dev.azuresynapse.net","attestationEndpoint":"attest.azure.net"},"batch":"https://batch.core.windows.net/","resourceManager":"https://management.azure.com/","vmImageAliasDoc":"https://raw.githubusercontent.com/Azure/azure-rest-api-specs/master/arm-compute/quickstart-templates/aliases.json","activeDirectoryDataLake":"https://datalake.azure.net/","sqlManagement":"https://management.core.windows.net:8443/","microsoftGraphResourceId":"https://graph.microsoft.com/","appInsightsResourceId":"https://api.applicationinsights.io","appInsightsTelemetryChannelResourceId":"https://dc.applicationinsights.azure.com/v2/track","attestationResourceId":"https://attest.azure.net","synapseAnalyticsResourceId":"https://dev.azuresynapse.net","logAnalyticsResourceId":"https://api.loganalytics.io","ossrDbmsResourceId":"https://ossrdbms-aad.database.windows.net"}
  • However, https://management.redmond.ext-s31r1304.masd.stbtest.microsoft.com/metadata/endpoints?api-version=2022-09-01 returns a json array with suffixes
[{"portal":"https://portal.redmond.ext-s31r1304.masd.stbtest.microsoft.com/","authentication":{"loginEndpoint":"https://adfs.redmond.ext-s31r1304.masd.stbtest.microsoft.com/adfs","audiences":["https://management.adfs.s31r1304.masd.stbtest.microsoft.com/e8b7057b-9773-4e8f-ba0e-57c12612fb80"]},"graphAudience":"https://graph.redmond.ext-s31r1304.masd.stbtest.microsoft.com/","graph":"https://graph.redmond.ext-s31r1304.masd.stbtest.microsoft.com/","name":"AzureStack-User-e8b7057b-9773-4e8f-ba0e-57c12612fb80","suffixes":{"keyVaultDns":"vault.redmond.ext-s31r1304.masd.stbtest.microsoft.com","storage":"redmond.ext-s31r1304.masd.stbtest.microsoft.com"},"gallery":"https://providers.s31r1304.masd.stbtest.microsoft.com:30016/"}]

What makes things worse, ARM doesn't have spec definition for this API. This PR aims for enhancing az cloud register to be able to handle both Json Object and Json Array response formats with/without suffixes

Testing Guide

History Notes

[Component Name 1] BREAKING CHANGE: az command a: Make some customer-facing breaking change
[Component Name 2] az command b: Add some customer-facing feature


This checklist is used to make sure that common guidelines for a pull request are followed.

Copilot AI review requested due to automatic review settings May 28, 2025 09:53
@azure-client-tools-bot-prd
Copy link

azure-client-tools-bot-prd bot commented May 28, 2025

️✔️AzureCLI-FullTest
️✔️acr
️✔️latest
️✔️3.12
️✔️3.9
️✔️acs
️✔️latest
️✔️3.12
️✔️3.9
️✔️advisor
️✔️latest
️✔️3.12
️✔️3.9
️✔️ams
️✔️latest
️✔️3.12
️✔️3.9
️✔️apim
️✔️latest
️✔️3.12
️✔️3.9
️✔️appconfig
️✔️latest
️✔️3.12
️✔️3.9
️✔️appservice
️✔️latest
️✔️3.12
️✔️3.9
️✔️aro
️✔️latest
️✔️3.12
️✔️3.9
️✔️backup
️✔️latest
️✔️3.12
️✔️3.9
️✔️batch
️✔️latest
️✔️3.12
️✔️3.9
️✔️batchai
️✔️latest
️✔️3.12
️✔️3.9
️✔️billing
️✔️latest
️✔️3.12
️✔️3.9
️✔️botservice
️✔️latest
️✔️3.12
️✔️3.9
️✔️cdn
️✔️latest
️✔️3.12
️✔️3.9
️✔️cloud
️✔️latest
️✔️3.12
️✔️3.9
️✔️cognitiveservices
️✔️latest
️✔️3.12
️✔️3.9
️✔️compute_recommender
️✔️latest
️✔️3.12
️✔️3.9
️✔️computefleet
️✔️latest
️✔️3.12
️✔️3.9
️✔️config
️✔️latest
️✔️3.12
️✔️3.9
️✔️configure
️✔️latest
️✔️3.12
️✔️3.9
️✔️consumption
️✔️latest
️✔️3.12
️✔️3.9
️✔️container
️✔️latest
️✔️3.12
️✔️3.9
️✔️containerapp
️✔️latest
️✔️3.12
️✔️3.9
️✔️core
️✔️latest
️✔️3.12
️✔️3.9
️✔️cosmosdb
️✔️latest
️✔️3.12
️✔️3.9
️✔️databoxedge
️✔️latest
️✔️3.12
️✔️3.9
️✔️dls
️✔️latest
️✔️3.12
️✔️3.9
️✔️dms
️✔️latest
️✔️3.12
️✔️3.9
️✔️eventgrid
️✔️latest
️✔️3.12
️✔️3.9
️✔️eventhubs
️✔️latest
️✔️3.12
️✔️3.9
️✔️feedback
️✔️latest
️✔️3.12
️✔️3.9
️✔️find
️✔️latest
️✔️3.12
️✔️3.9
️✔️hdinsight
️✔️latest
️✔️3.12
️✔️3.9
️✔️identity
️✔️latest
️✔️3.12
️✔️3.9
️✔️iot
️✔️latest
️✔️3.12
️✔️3.9
️✔️keyvault
️✔️latest
️✔️3.12
️✔️3.9
️✔️lab
️✔️latest
️✔️3.12
️✔️3.9
️✔️managedservices
️✔️latest
️✔️3.12
️✔️3.9
️✔️maps
️✔️latest
️✔️3.12
️✔️3.9
️✔️marketplaceordering
️✔️latest
️✔️3.12
️✔️3.9
️✔️monitor
️✔️latest
️✔️3.12
️✔️3.9
️✔️mysql
️✔️latest
️✔️3.12
️✔️3.9
️✔️netappfiles
️✔️latest
️✔️3.12
️✔️3.9
️✔️network
️✔️latest
️✔️3.12
️✔️3.9
️✔️policyinsights
️✔️latest
️✔️3.12
️✔️3.9
️✔️privatedns
️✔️latest
️✔️3.12
️✔️3.9
️✔️profile
️✔️latest
️✔️3.12
️✔️3.9
️✔️rdbms
️✔️latest
️✔️3.12
️✔️3.9
️✔️redis
️✔️latest
️✔️3.12
️✔️3.9
️✔️relay
️✔️latest
️✔️3.12
️✔️3.9
️✔️resource
️✔️latest
️✔️3.12
️✔️3.9
️✔️role
️✔️latest
️✔️3.12
️✔️3.9
️✔️search
️✔️latest
️✔️3.12
️✔️3.9
️✔️security
️✔️latest
️✔️3.12
️✔️3.9
️✔️servicebus
️✔️latest
️✔️3.12
️✔️3.9
️✔️serviceconnector
️✔️latest
️✔️3.12
️✔️3.9
️✔️servicefabric
️✔️latest
️✔️3.12
️✔️3.9
️✔️signalr
️✔️latest
️✔️3.12
️✔️3.9
️✔️sql
️✔️latest
️✔️3.12
️✔️3.9
️✔️sqlvm
️✔️latest
️✔️3.12
️✔️3.9
️✔️storage
️✔️latest
️✔️3.12
️✔️3.9
️✔️synapse
️✔️latest
️✔️3.12
️✔️3.9
️✔️telemetry
️✔️latest
️✔️3.12
️✔️3.9
️✔️util
️✔️latest
️✔️3.12
️✔️3.9
️✔️vm
️✔️latest
️✔️3.12
️✔️3.9

@azure-client-tools-bot-prd
Copy link

azure-client-tools-bot-prd bot commented May 28, 2025

️✔️AzureCLI-BreakingChangeTest
️✔️Non Breaking Changes

@yonzhan
Copy link
Collaborator

yonzhan commented May 28, 2025

Thank you for your contribution! We will review the pull request and get back to you soon.

@github-actions
Copy link

The git hooks are available for azure-cli and azure-cli-extensions repos. They could help you run required checks before creating the PR.

Please sync the latest code with latest dev branch (for azure-cli) or main branch (for azure-cli-extensions).
After that please run the following commands to enable git hooks:

pip install azdev --upgrade
azdev setup -c <your azure-cli repo path> -r <your azure-cli-extensions repo path>

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR enhances the az cloud register command to support multiple JSON response formats from the ARM metadata endpoint.

  • Added handling for JSON response types: both dict and list formats
  • Improved robustness for suffix retrieval by checking for the presence of the 'suffixes' key

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
src/azure-cli/azure/cli/command_modules/cloud/custom.py Handles JSON object or array responses from the metadata endpoint
src/azure-cli-core/azure/cli/core/cloud.py Refines suffix handling by checking existence of the 'suffixes' key

@evelyn-ys evelyn-ys requested a review from kairu-ms May 30, 2025 01:10
@evelyn-ys evelyn-ys self-assigned this Jun 3, 2025
kairu-ms
kairu-ms previously approved these changes Jun 10, 2025
Copy link
Member

@jiasli jiasli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it is the client's responsibility to handle inconsistent API responses. If the API version is the same (2022-09-01), the response SHOULD be consistent.

It is a bad practice to use undocumented API or handle non-compliant API response.

Since there is no spec for this API, the service can change it at will without any responsibility. It is impossible to enumerate all possibilities for an it, so it can break Azure CLI all of a sudden, even causing security or privacy issues (if targeting the wrong endpoint).

We learnt several lessons from using undocumented API or handling non-compliant API response in the past:

def _get_processed_arm_endpoint(name, add_dot=False, fallback_value=None):
if is_suffix:
return (_add_starting_dot(arm_dict['suffixes'][name]) if add_dot else arm_dict['suffixes'][name]) if name in arm_dict['suffixes'] else fallback_value
return (_add_starting_dot(arm_dict['suffixes'][name]) if add_dot else arm_dict['suffixes'][name]) if 'suffixes' in arm_dict and name in arm_dict['suffixes'] else fallback_value
Copy link
Member

@jiasli jiasli Jun 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If arm_dict does contain suffixes but arm_dict['suffixes'] is None, name in arm_dict['suffixes'] will fail:

arm_dict = {'suffixes': None}
name = 'test'
print('suffixes' in arm_dict and name in arm_dict['suffixes'])

Output:

Traceback (most recent call last):
  File "D:\cli\testproj\main.py", line 3, in <module>
    print('suffixes' in arm_dict and name in arm_dict['suffixes'])
                                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: argument of type 'NoneType' is not iterable

It is hard to enumerate all possibilities for an undocumented API.

Copy link
Member Author

@evelyn-ys evelyn-ys Jun 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice catch! Updated to use if name in arm_dict.get('suffixes', {}) now

@evelyn-ys evelyn-ys requested a review from jiasli June 10, 2025 13:51
@evelyn-ys evelyn-ys marked this pull request as draft June 12, 2025 02:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

AAD Auto-Assign Auto assign by bot Portal az portal

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Error when registering Azure Stack Hub

5 participants