Skip to content

Conversation

@jiasli
Copy link
Member

@jiasli jiasli commented Jun 18, 2020

Description

Remove unnecessary cli_ctx reference, which makes pytest --capture=no fail:

RecursionError: maximum recursion depth exceeded

Root cause

When pytest captures stdout,

  1. pytest uses io.FileIO to patch sys.stdout
  2. When initializing CLI, CLI.out_file points to io.FileIO object
    out_file=sys.stdout,
  3. tests.util.redirect_io uses io.StringIO to patch sys.stdout, but this doesn't affect the existing CLI.out_file

Since CLI.out_file doesn't equal sys.__stdout__ (the original stdout), it won't be replaced with a colorama.ansitowin32.StreamWrapper:

knack/knack/cli.py

Lines 195 to 197 in 0d0308b

if self.out_file == sys.__stdout__:
# point out_file to the new sys.stdout which is overwritten by colorama
self.out_file = sys.stdout

As PreviewItem (same for ExperimentalItem and Deprecated) keeps a reference to cli_ctx, during invocation

command_kwargs = copy.deepcopy(self.group_kwargs)

  1. PreviewItem is deepcopied ->
  2. cli_ctx is deepcopied ->
  3. CLI.out_file is deepcopied

Because io.FileIO can't be pickled, copy.deepcopy fails and reaches

knack/knack/util.py

Lines 78 to 82 in 5f0bcc2

except TypeError:
if k == 'cli_ctx':
setattr(result, k, self.cli_ctx)
else:
raise

This makes cli_ctx not being deepcopied.

However, when pytest is called with pytest --capture=no, it doesn't patch sys.stdout,

  1. When initializing CLI, CLI.out_file equals sys.__stdout__
  2. tests.util.redirect_io uses io.StringIO to patch sys.stdout, but this doesn't affect the existing CLI.out_file

Since CLI.out_file equals sys.__stdout__ (the original stdout), it will be replaced with a colorama.ansitowin32.StreamWrapper and be deepcopied.

Because of a circular reference in AnsiToWin32

https://github.com/tartley/colorama/blob/3d8d48a95de10be25b161c914f274ec6c41d3129/colorama/ansitowin32.py#L28-L29

    def __getattr__(self, name):
        return getattr(self.__wrapped, name)

and io.StringIO can be pickled, when self.__wrapped is not set, deepcopy fails with exception RecursionError: maximum recursion depth exceeded.

To repro, use this simple script:

import colorama
import copy
import sys
import io

sys.stdout = io.StringIO()
colorama.init()
copy.deepcopy(sys.stdout)

Or

class StreamWrapper(object):
    def __init__(self, wrapped, converter):
        # self.__wrapped = wrapped
        self.__convertor = converter

    def __getattr__(self, name):
        return getattr(self.__wrapped, name)


sr = StreamWrapper("a", "b")
hasattr(sr, '__setstate__')

@jiasli jiasli marked this pull request as ready for review December 2, 2020 08:36
@jiasli jiasli self-assigned this Dec 9, 2020
@jiasli jiasli merged commit 6354f3d into microsoft:master Dec 16, 2020
@jiasli jiasli deleted the cli_ctx branch December 16, 2020 09:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants