Skip to content

Conversation

@haroldrandom
Copy link
Contributor

@haroldrandom haroldrandom commented Apr 17, 2020

Description
When running these 2 tests in serial:

  1. test_role.py::RoleAssignmentScenarioTest::test_role_assignment_handle_conflicted_assignments
  2. test_role.py::RoleAssignmentScenarioTest::test_role_assignment_mgmt_grp
    The second one will fail on Linux but succeed on Windows.

Why failed on Linux?
At the beginning, the 1st test would chdir to a temporary dir.

base_dir = os.curdir
try:
temp_dir = self.create_temp_dir()
os.chdir(temp_dir)

and at the end the of test, it would try to switch back but actually it didn't because the basr_dir is a relative path. So, the original directory is lost and the test procedure would stay at the temporary directory.

finally:
self.cmd('configure --default group="" --scope local')
os.chdir(os.path.basename(base_dir))
self.cmd('ad user delete --upn-or-object-id {upn}')

However, during the creation of tempory directory, it will register a callback to clean this folder away after the test is done.

    # IntegrationTestBase from base.py (azure_devtools)
    def create_temp_dir(self):
        """
        Create a temporary directory for testing. The test harness will delete the directory during tearing down.
        """
        temp_dir = tempfile.mkdtemp()
        self.addCleanup(lambda: shutil.rmtree(temp_dir, ignore_errors=True))

        return temp_dir

So, when the second test starts, it's in a non-existing directory, and the intialization of CLI instannce would fail due to LocalContext object require reading a file from current dir:

======================================================================================================================================== ERRORS ========================================================================================================================================
______________________________________________________________________________________________________ ERROR at setup of RoleAssignmentScenarioTest.test_role_assignment_mgmt_grp ______________________________________________________________________________________________________

self = <[AttributeError("'RoleAssignmentScenarioTest' object has no attribute '_testMethodName'",) raised in repr()] RoleAssignmentScenarioTest object at 0x7fa0ac645f60>, method_name = 'test_role_assignment_mgmt_grp', config_file = None, recording_name = None
recording_processors = None, replay_processors = None, recording_patches = None, replay_patches = None

    def __init__(self, method_name, config_file=None, recording_name=None,
                 recording_processors=None, replay_processors=None, recording_patches=None, replay_patches=None):
>       self.cli_ctx = get_dummy_cli()

/home/harold/Microsoft/azure-cli/src/azure-cli-testsdk/azure/cli/testsdk/base.py:82: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/home/harold/Microsoft/azure-cli/src/azure-cli-testsdk/azure/cli/testsdk/reverse_dependency.py:14: in get_dummy_cli
    return getattr(mod, 'DummyCli')(*args, **kwargs)
/home/harold/Microsoft/azure-cli/src/azure-cli-core/azure/cli/core/mock.py:34: in __init__
    invocation_cls=AzCliCommandInvoker)
/home/harold/Microsoft/azure-cli/src/azure-cli-core/azure/cli/core/__init__.py:73: in __init__
    dir_name=os.path.basename(self.config.config_dir), file_name='local_context'
/home/harold/Microsoft/azure-cli/src/azure-cli-core/azure/cli/core/local_context.py:27: in __init__
    self._load_local_context_file()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <azure.cli.core.local_context.AzCLILocalContext object at 0x7fa0ad3ad198>

    def _load_local_context_file(self):
>       current_dir = os.getcwd()
E       FileNotFoundError: [Errno 2] No such file or directory

/home/harold/Microsoft/azure-cli/src/azure-cli-core/azure/cli/core/local_context.py:30: FileNotFoundError

Why succeeded on Windows
The removal temporary dir is ignoring the permission error

shutil.rmtree(temp_dir, ignore_errors=True)

If set ignore=False, you will see:
image

So, the deletion wouldn't succeed and the second test would stuck in that temporary folder and succeed just lucky.

Testing Guide
Before fix, run via

pytest -x -v -p no:warnings --log-level=WARN \
src/azure-cli/azure/cli/command_modules/role/tests/latest/test_role.py::RoleAssignmentScenarioTest::test_role_assignment_handle_conflicted_assignments \
src/azure-cli/azure/cli/command_modules/role/tests/latest/test_role.py::RoleAssignmentScenarioTest::test_role_assignment_mgmt_grp

History Notes


This checklist is used to make sure that common guidelines for a pull request are followed.

@haroldrandom haroldrandom added this to the S168 milestone Apr 17, 2020
@haroldrandom haroldrandom self-assigned this Apr 17, 2020
@haroldrandom haroldrandom marked this pull request as ready for review April 17, 2020 04:48
@haroldrandom haroldrandom requested a review from jiasli as a code owner April 17, 2020 04:48
@haroldrandom haroldrandom changed the title {Role} fix test fail when running in serial due to buggy way to swtich back {Role} fix test fail when running in serial due to buggy way to swtich tempory folder back Apr 17, 2020
@haroldrandom haroldrandom merged commit 3bc98f1 into Azure:dev Apr 17, 2020
@haroldrandom haroldrandom deleted the fix-role-test branch April 17, 2020 04:54
@jiasli
Copy link
Member

jiasli commented Apr 17, 2020

FYI, the success on Windows is similar to microsoft/knack#177. Have to admit Windows is smarter on this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants