Skip to content

Conversation

@hjhhh3000vitae
Copy link
Contributor

@hjhhh3000vitae hjhhh3000vitae commented Aug 2, 2019

Using azure-mgmt-resource_3.1.0

This checklist is used to make sure that common guidelines for a pull request are followed.

  • The PR has modified HISTORY.rst describing any customer-facing, functional changes. Note that this does not include changes only to help content. (see Modifying change log).

  • I adhere to the Command Guidelines.

@hjhhh3000vitae hjhhh3000vitae marked this pull request as ready for review August 2, 2019 00:19
@mmyyrroonn
Copy link
Contributor

@hjhhh3000vitae Hello. You should run azdev test <module_name> to see the failure test. Then run azdev test <module_name> --live --lf to record the test again in this PR to pass the CI.

@hjhhh3000vitae
Copy link
Contributor Author

A code change to fix incompatibility to new api version is made in this PR according to Azure/azure-rest-api-specs#6393

@hjhhh3000vitae
Copy link
Contributor Author

@hjhhh3000vitae Hello. You should run azdev test <module_name> to see the failure test. Then run azdev test <module_name> --live --lf to record the test again in this PR to pass the CI.

Hello @myronfanqiu ! Thanks for the instruction. I tried azdev test resource and it told me all 84 tests succeeded, but for azdev test resource --live --lf, it said 38 out of 84 tests failed and looks like it deleted recordings of these failed tests. Do you know if this behavior is expected?

@mmyyrroonn
Copy link
Contributor

@hjhhh3000vitae Hello. You should run azdev test <module_name> to see the failure test. Then run azdev test <module_name> --live --lf to record the test again in this PR to pass the CI.

Hello @myronfanqiu ! Thanks for the instruction. I tried azdev test resource and it told me all 84 tests succeeded, but for azdev test resource --live --lf, it said 38 out of 84 tests failed and looks like it deleted recordings of these failed tests. Do you know if this behavior is expected?

It's weird if you passed all the tests by running azdev test resource since it failed in the CI system. If you run azdev test resource --live --lf, all the recording files will be deleted and re-recorded. Basically, you should pass the test by running azdev test resource --live --lf since the recording files are just a simulated cloud.

@hjhhh3000vitae
Copy link
Contributor Author

@hjhhh3000vitae Hello. You should run azdev test <module_name> to see the failure test. Then run azdev test <module_name> --live --lf to record the test again in this PR to pass the CI.

Hello @myronfanqiu ! Thanks for the instruction. I tried azdev test resource and it told me all 84 tests succeeded, but for azdev test resource --live --lf, it said 38 out of 84 tests failed and looks like it deleted recordings of these failed tests. Do you know if this behavior is expected?

It's weird if you passed all the tests by running azdev test resource since it failed in the CI system. If you run azdev test resource --live --lf, all the recording files will be deleted and re-recorded. Basically, you should pass the test by running azdev test resource --live --lf since the recording files are just a simulated cloud.

@myronfanqiu Thanks. I checked the tests locally and fixed some of them.

For now, on the local side, I tried running tests using azdev test resource --live against dev branch and still some of the tests are failing so I am assuming it is configuration issue?

On the online side, however, I checked the log of the tests. It looks like none of the failing/error tests is from the resource module, which I am working on in this PR. Most of them are from AppService. Is that anyway I can kick-off a online testing pipeline against 'dev' branch to do a comparison here? Thank you.

I am not sure why the online checks are still failing now as looking from the log, all the failing tests are not from resource module, which I am working on in this branch but from AppService mostly.

@mmyyrroonn
Copy link
Contributor

mmyyrroonn commented Aug 15, 2019

@hjhhh3000vitae Hello. You should run azdev test <module_name> to see the failure test. Then run azdev test <module_name> --live --lf to record the test again in this PR to pass the CI.

Hello @myronfanqiu ! Thanks for the instruction. I tried azdev test resource and it told me all 84 tests succeeded, but for azdev test resource --live --lf, it said 38 out of 84 tests failed and looks like it deleted recordings of these failed tests. Do you know if this behavior is expected?

It's weird if you passed all the tests by running azdev test resource since it failed in the CI system. If you run azdev test resource --live --lf, all the recording files will be deleted and re-recorded. Basically, you should pass the test by running azdev test resource --live --lf since the recording files are just a simulated cloud.

@myronfanqiu Thanks. I checked the tests locally and fixed some of them.

For now, on the local side, I tried running tests using azdev test resource --live against dev branch and still some of the tests are failing so I am assuming it is configuration issue?

On the online side, however, I checked the log of the tests. It looks like none of the failing/error tests is from the resource module, which I am working on in this PR. Most of them are from AppService. Is that anyway I can kick-off a online testing pipeline against 'dev' branch to do a comparison here? Thank you.

I am not sure why the online checks are still failing now as looking from the log, all the failing tests are not from resource module, which I am working on in this branch but from AppService mostly.

Hello, you should run azdev test resource against your development branch and push these recording yml together within this PR.

I tested the appservice locally and all tests passed. Could you please run azdev test appservice to see the results?

If you can passed the test locally, we should check our CI system. If you cannot pass them locally, we need to figure out what happened to the appservice. In either case, we need your help.

@mmyyrroonn
Copy link
Contributor

@hjhhh3000vitae Hello. Did these command modules use the resource SDK?

@hjhhh3000vitae
Copy link
Contributor Author

@hjhhh3000vitae Hello. Did these command modules use the resource SDK?

I can confirm they are using ResourceManagementClient from resource SDK.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because the !!python/unicode in response body, response body cannot be parsed correctly. Why do you have such string in your recording file? When I delete !!python/unicode manually, it works. In addition, please have a look at other !!python/unicode in the following part.

He Huang added 12 commits August 21, 2019 18:07
The following tests cannot pass on live mode:

test_network_app_gateway_waf_policy
test_hdinsight_application
test_hdinsight_monitor
test_batchai_usages
test_batchai_job_level_mounting_scenario
test_batchai_config_less_cluster_with_file_systems
test_batchai_cluster_with_setup_command
test_batchai_cluster_with_file_systems
test_batchai_auto_scale_scenario
test_batchai_cluster_with_auto_storage
test_network_lb_zone
test_batchai_manual_scale_scenario
@hjhhh3000vitae
Copy link
Contributor Author

@myronfanqiu @Juliehzl @yugangw-msft Tests now all passed. Could you please review this PR.

@mmyyrroonn mmyyrroonn assigned acs-bot and Juliehzl and unassigned acs-bot Sep 2, 2019
@yugangw-msft yugangw-msft merged commit 8aa2eb5 into Azure:dev Sep 2, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants