Failing to timeout regex?

[ecurtin]

Howdy! Thanks for your work on this projec!

I'm honestly not sure if I've found a bug or if I'm misunderstanding the usage of the decorator. I'm trying to use func_set_timeout() to timeout some catastrophically backtracking regexes. (They scale of O(2^n), nasty little things) However, in my tests the decorator works just fine on a sleep function but doesn't catch the regex. In fact the process has to be killed from another shell.

Execution environment:

• Centos 7 container
• Python 3.7.5
• Container running via Docker Desktop for macOS

import re
import time

import pytest

from func_timeout import func_set_timeout, FunctionTimedOut

@func_set_timeout(5)
def _sleep_method_func_timeout():
    time.sleep(10)

@func_set_timeout(5)
def _catastrophic_backtracking_regex():
    pattern = r"(\w+)\*([\w\s]+)*/$"
    re_compiled = re.compile(pattern)
    results = re_compiled.search('COPRO*HORIZON 2000                 HOR')
    print(results.groups())

# this passes
def test_func_timeout_decorator_sleep_func():
    with pytest.raises(FunctionTimedOut):
        _sleep_method_func_timeout()

# this does not pass and has to be killed externally
def test_func_timeout_decorater_regex():
    with pytest.raises(FunctionTimedOut):
        _catastrophic_backtracking_regex()

Terminal output:

[app@a456222c57ed src]$ pytest -v tests/test_timeout_handling.py 
================================================================= test session starts ==================================================================
platform linux -- Python 3.7.3, pytest-5.2.4, py-1.8.0, pluggy-0.13.0 -- /usr/local/pyenv/versions/3.7.3/bin/python3.7
cachedir: .pytest_cache
rootdir: /opt/app/src, inifile: pytest.ini
plugins: cov-2.8.1
collected 2 items                                                                                                                                      

tests/test_timeout_handling.py::test_func_timeout_decorator_sleep_func PASSED                                                                    [ 50%]
tests/test_timeout_handling.py::test_func_timeout_decorater_regex Killed

Is this related to #10? I don't think it is since that's really specific to Python2 and this is 3.7.5. Have I missed something in the README? Did I find a bug? Is my execution environment just a wacky snowflake?

Thanks again for your work!

[jalvarezit]

I have encountered this behaviour under the following circumstances:

• Ubuntu version: 21.10
• python version: 3.9.7
• package version: 4.3.5

Code example:

import re
from func_timeout import func_timeout, FunctionTimedOut
from timeit import default_timer as timer

if __name__ == '__main__':
    my_regex = '^(a|a?)+b$'
    text = 'aaaaaaaaaaaaaaaaaaaaaaaa'
    max_timeout = 1
    start = timer()
    try:
        func_timeout(max_timeout,re.search, (my_regex,text) )
    except FunctionTimedOut:
        print('Time exceeded')
    end = timer()
    print(end-start)

[kata198]

Func_timeout only has a chance to stop function between python instructions. If the call is stuck in libpython C code, it must return to the python stack before it can be terminated.
You can launch your regexes using the StoppableThread class provided, and raise an error to the user or whatever, but you cannot stop between the C functions, only between lines of python.
Perhaps you can split your data into sections such that it returns to python, or test on a small subset of the string?

[ecurtin]

Thank you for the clear, concise explanation @kata198 !