Skip to content

Conversation

@haroldrandom
Copy link
Contributor

@haroldrandom haroldrandom commented Jun 30, 2020

Add benchmark command to check every commands that could apear in Azure CLi and Azure CLI Extensions.

Run benchmark on the given commands, and display them generally. Run them with "python -m azure.cli {COMMAND}" in a seperately process.
It could run in live mode if you have logged in and the command could be executed properly in Shell.

Close Azure/azure-cli-extensions#64.

Changes will be on boarded to Azure/azure-cli Azure/azure-cli#14156

Test Guide:

Successful commands:

  • azdev perf benchmark "version" "nework vnet" "group list"
  • azdev perf benchmark "az storage account -h"

Non-existing commands:
It will run it as they are existing, and display it as normal. But the result is useless. like azdev perf benchmark qhjhwkehqkjhwe

@haroldrandom haroldrandom added the Perf azdev perf * label Jun 30, 2020
@haroldrandom haroldrandom self-assigned this Jun 30, 2020
@haroldrandom haroldrandom changed the title benchmark command Add benchmark command Jul 1, 2020
@haroldrandom haroldrandom requested review from jiasli and yonzhan July 1, 2020 06:43
@yonzhan
Copy link
Collaborator

yonzhan commented Jul 1, 2020

benchmark command

@jiasli
Copy link
Member

jiasli commented Jul 1, 2020

Nice work!! 🎉

Let's simply it to

azdev perf benchmark "az version"

Just like

az find "az vm create"

and choose the command to run in the CI later.

For --help command, we can explicitly include --help in the command, such as

azdev perf benchmark "az rest --help"

@haroldrandom
Copy link
Contributor Author

@jiasli Refine as you suggest and update PR description.

There is one could be discussable - currently, I don't help to judge whether a given command is valid or not, I will run it anyway.

@jiasli
Copy link
Member

jiasli commented Jul 2, 2020

I am actually think maybe we can use the dev version of azdev in the Azure CLI pipeline, instead of the one released on PyPI. This will make the development more agile.

@haroldrandom
Copy link
Contributor Author

haroldrandom commented Jul 2, 2020

I am actually think maybe we can use the dev version of azdev in the Azure CLI pipeline, instead of the one released on PyPI. This will make the development more agile.

I will make it read on Azure/azure-cli ASAP

@haroldrandom haroldrandom merged commit d5f7458 into Azure:master Jul 2, 2020
@haroldrandom haroldrandom deleted the benchmark-mvp branch July 2, 2020 04:49
RenSilvaAU pushed a commit to RenSilvaAU/azure-cli-dev-tools that referenced this pull request Aug 7, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Perf azdev perf *

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CI job for checking perf. of extensions just like in CLI repo

3 participants