Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Silent crash when exec produces a syntax error involving Unicode #520

Closed
pytestbot opened this issue May 27, 2014 · 4 comments
Closed

Silent crash when exec produces a syntax error involving Unicode #520

pytestbot opened this issue May 27, 2014 · 4 comments
Labels
good first issue easy issue that is friendly to new contributor type: bug problem that needs to be addressed

Comments

@pytestbot
Copy link
Contributor

Originally reported by: Kodi Arfer (BitBucket: Kodiologist, GitHub: Kodiologist)

Input:

#!python
def test_foo():
    exec u'foo u"\u03b1"'

Output:

===================== test session starts ======================
platform linux2 -- Python 2.7.6 -- py-1.4.20 -- pytest-2.5.2
collected 1 items 

test_something.py

(Except the output has no trailing newline, just a trailing space after test_something.py.)

Using --assert=plain makes no difference.
@pytestbot pytestbot added the type: bug problem that needs to be addressed label Jun 15, 2015
@RonnyPfannschmidt RonnyPfannschmidt added the good first issue easy issue that is friendly to new contributor label Jul 25, 2015
@Bjwebb
Copy link
Contributor

Bjwebb commented Sep 21, 2015

This seems to work as expected under Python 3 (with the addition of parentheses for exec).

Input:

#!python
def test_foo():
    exec(u'foo u"\u03b1"')

Output:

======================================= test session starts =======================================
platform linux -- Python 3.4.3, pytest-2.8.0, py-1.4.30, pluggy-0.3.1
rootdir: /home/bjwebb/code/pytesttest/3, inifile: 
collected 1 items 

test_something.py F

============================================ FAILURES =============================================
____________________________________________ test_foo _____________________________________________

    def test_foo():
>       exec(u'foo u"\u03b1"')
E         File "<string>", line 1
E           foo u"α"
E                  ^
E       SyntaxError: invalid syntax

test_something.py:3: SyntaxError
==================================== 1 failed in 0.01 seconds =====================================

Under Python 2, I get the crash, but it's not quiet as I get INTERNALERROR output.
Input:

#!python
def test_foo():
    exec u'foo u"\u03b1"'

Output:

======================================= test session starts =======================================
platform linux2 -- Python 2.7.10, pytest-2.8.0, py-1.4.30, pluggy-0.3.1
rootdir: /home/bjwebb/code/pytesttest/2, inifile: 
collected 1 items 

test_something.py 
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/home/bjwebb/code/pytest/_pytest/main.py", line 90, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File "/home/bjwebb/code/pytest/_pytest/main.py", line 121, in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File "/home/bjwebb/code/pytest/_pytest/vendored_packages/pluggy.py", line 724, in __call__
INTERNALERROR>     return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
INTERNALERROR>   File "/home/bjwebb/code/pytest/_pytest/vendored_packages/pluggy.py", line 338, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/home/bjwebb/code/pytest/_pytest/vendored_packages/pluggy.py", line 333, in <lambda>
INTERNALERROR>     _MultiCall(methods, kwargs, hook.spec_opts).execute()
INTERNALERROR>   File "/home/bjwebb/code/pytest/_pytest/vendored_packages/pluggy.py", line 596, in execute
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/home/bjwebb/code/pytest/_pytest/main.py", line 146, in pytest_runtestloop
INTERNALERROR>     item.config.hook.pytest_runtest_protocol(item=item, nextitem=nextitem)
INTERNALERROR>   File "/home/bjwebb/code/pytest/_pytest/vendored_packages/pluggy.py", line 724, in __call__
INTERNALERROR>     return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
INTERNALERROR>   File "/home/bjwebb/code/pytest/_pytest/vendored_packages/pluggy.py", line 338, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/home/bjwebb/code/pytest/_pytest/vendored_packages/pluggy.py", line 333, in <lambda>
INTERNALERROR>     _MultiCall(methods, kwargs, hook.spec_opts).execute()
INTERNALERROR>   File "/home/bjwebb/code/pytest/_pytest/vendored_packages/pluggy.py", line 595, in execute
INTERNALERROR>     return _wrapped_call(hook_impl.function(*args), self.execute)
INTERNALERROR>   File "/home/bjwebb/code/pytest/_pytest/vendored_packages/pluggy.py", line 253, in _wrapped_call
INTERNALERROR>     return call_outcome.get_result()
INTERNALERROR>   File "/home/bjwebb/code/pytest/_pytest/vendored_packages/pluggy.py", line 279, in get_result
INTERNALERROR>     _reraise(*ex)  # noqa
INTERNALERROR>   File "/home/bjwebb/code/pytest/_pytest/vendored_packages/pluggy.py", line 264, in __init__
INTERNALERROR>     self.result = func()
INTERNALERROR>   File "/home/bjwebb/code/pytest/_pytest/vendored_packages/pluggy.py", line 596, in execute
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/home/bjwebb/code/pytest/_pytest/runner.py", line 65, in pytest_runtest_protocol
INTERNALERROR>     runtestprotocol(item, nextitem=nextitem)
INTERNALERROR>   File "/home/bjwebb/code/pytest/_pytest/runner.py", line 75, in runtestprotocol
INTERNALERROR>     reports.append(call_and_report(item, "call", log))
INTERNALERROR>   File "/home/bjwebb/code/pytest/_pytest/runner.py", line 121, in call_and_report
INTERNALERROR>     report = hook.pytest_runtest_makereport(item=item, call=call)
INTERNALERROR>   File "/home/bjwebb/code/pytest/_pytest/vendored_packages/pluggy.py", line 724, in __call__
INTERNALERROR>     return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
INTERNALERROR>   File "/home/bjwebb/code/pytest/_pytest/vendored_packages/pluggy.py", line 338, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/home/bjwebb/code/pytest/_pytest/vendored_packages/pluggy.py", line 333, in <lambda>
INTERNALERROR>     _MultiCall(methods, kwargs, hook.spec_opts).execute()
INTERNALERROR>   File "/home/bjwebb/code/pytest/_pytest/vendored_packages/pluggy.py", line 595, in execute
INTERNALERROR>     return _wrapped_call(hook_impl.function(*args), self.execute)
INTERNALERROR>   File "/home/bjwebb/code/pytest/_pytest/vendored_packages/pluggy.py", line 249, in _wrapped_call
INTERNALERROR>     wrap_controller.send(call_outcome)
INTERNALERROR>   File "/home/bjwebb/code/pytest/_pytest/skipping.py", line 170, in pytest_runtest_makereport
INTERNALERROR>     rep = outcome.get_result()
INTERNALERROR>   File "/home/bjwebb/code/pytest/_pytest/vendored_packages/pluggy.py", line 279, in get_result
INTERNALERROR>     _reraise(*ex)  # noqa
INTERNALERROR>   File "/home/bjwebb/code/pytest/_pytest/vendored_packages/pluggy.py", line 264, in __init__
INTERNALERROR>     self.result = func()
INTERNALERROR>   File "/home/bjwebb/code/pytest/_pytest/vendored_packages/pluggy.py", line 596, in execute
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/home/bjwebb/code/pytest/_pytest/runner.py", line 224, in pytest_runtest_makereport
INTERNALERROR>     longrepr = item.repr_failure(excinfo)
INTERNALERROR>   File "/home/bjwebb/code/pytest/_pytest/python.py", line 722, in repr_failure
INTERNALERROR>     return self._repr_failure_py(excinfo, style=style)
INTERNALERROR>   File "/home/bjwebb/code/pytest/_pytest/python.py", line 715, in _repr_failure_py
INTERNALERROR>     style=style)
INTERNALERROR>   File "/home/bjwebb/code/pytest/_pytest/main.py", line 410, in _repr_failure_py
INTERNALERROR>     style=style, tbfilter=tbfilter)
INTERNALERROR>   File "/home/bjwebb/code/pytest/.ve2/lib/python2.7/site-packages/py/_code/code.py", line 412, in getrepr
INTERNALERROR>     return fmt.repr_excinfo(self)
INTERNALERROR>   File "/home/bjwebb/code/pytest/.ve2/lib/python2.7/site-packages/py/_code/code.py", line 590, in repr_excinfo
INTERNALERROR>     reprtraceback = self.repr_traceback(excinfo)
INTERNALERROR>   File "/home/bjwebb/code/pytest/.ve2/lib/python2.7/site-packages/py/_code/code.py", line 582, in repr_traceback
INTERNALERROR>     reprentry = self.repr_traceback_entry(entry, einfo)
INTERNALERROR>   File "/home/bjwebb/code/pytest/.ve2/lib/python2.7/site-packages/py/_code/code.py", line 543, in repr_traceback_entry
INTERNALERROR>     s = self.get_source(source, line_index, excinfo, short=short)
INTERNALERROR>   File "/home/bjwebb/code/pytest/.ve2/lib/python2.7/site-packages/py/_code/code.py", line 489, in get_source
INTERNALERROR>     lines.extend(self.get_exconly(excinfo, indent=indent, markall=True))
INTERNALERROR>   File "/home/bjwebb/code/pytest/.ve2/lib/python2.7/site-packages/py/_code/code.py", line 496, in get_exconly
INTERNALERROR>     exlines = excinfo.exconly(tryshort=True).split('\n')
INTERNALERROR>   File "/home/bjwebb/code/pytest/.ve2/lib/python2.7/site-packages/py/_code/code.py", line 376, in exconly
INTERNALERROR>     text = ''.join(lines)
INTERNALERROR> UnicodeDecodeError: 'ascii' codec can't decode byte 0xce in position 10: ordinal not in range(128)

========================================  in 0.01 seconds =========================================

@Bjwebb
Copy link
Contributor

Bjwebb commented Sep 21, 2015

The problem seems to be in py.code:

Input:

import py

try:
    exec u'''foo u"\u03b1"'''
except Exception as e:
    info = py.code.ExceptionInfo()
    print info.exconly()

Output:

Traceback (most recent call last):
  File "something.py", line 7, in <module>
    print info.exconly()
  File "/home/bjwebb/code/py/py/_code/code.py", line 376, in exconly
    text = ''.join(lines)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xce in position 10: ordinal not in range(128)

The issue seems to be that py.code has its own copy of format_exception_only that tries to return unicode (which fails when code.py tries to join them).
Input:

['  File "<string>", line 1\n', '    foo u"\xce\xb1"\n', '            ^\n', u'SyntaxError: invalid syntax\n']
[<type 'str'>, <type 'str'>, <type 'str'>, <type 'unicode'>]

Output:

['  File "<string>", line 1\n', '    foo u"\xce\xb1"\n', '            ^\n', u'SyntaxError: invalid syntax\n']
[<type 'str'>, <type 'str'>, <type 'str'>, <type 'unicode'>]

However, in this case we get a mixture of unicode and utf-8 encoded strings. This is because badline is a utf-8 encoded string - https://bitbucket.org/pytest-dev/py/src/b9cdab378c58c3c07dce13ab8bb58463ae6a0bba/py/_code/_py2traceback.py?at=default&fileviewer=file-view-default#_py2traceback.py-43

@Bjwebb
Copy link
Contributor

Bjwebb commented Sep 21, 2015

Exec is not the only way to get this error. Importing a non-ascii file with a syntax error has the same result:

Input:

import py

try:
    import bad
except Exception as e:
    info = py.code.ExceptionInfo()
    print info.exconly()

bad.py

# coding: utf-8
foo 'é'

Output

Traceback (most recent call last):
  File "something.py", line 7, in <module>
    print info.exconly()
  File "/home/bjwebb/code/py/py/_code/code.py", line 376, in exconly
    text = ''.join(lines)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 9: ordinal not in range(128)

@nicoddemus
Copy link
Member

Fixed in #578

@pytestbot pytestbot mentioned this issue Mar 29, 2018
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
good first issue easy issue that is friendly to new contributor type: bug problem that needs to be addressed
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@Bjwebb @RonnyPfannschmidt @pfctdayelise @nicoddemus @pytestbot