Determine argspec of static methods with defaults in Cython files

[simon-king-jena]

Fix this:

sage: cython("""
....: class Bla:
....:     @staticmethod
....:     def join(categories, bint as_list = False, tuple ignore_axioms=(), tuple axioms=()):
....:         pass
....: """)
sage: from sage.misc.sageinspect import sage_getargspec
sage: sage_getargspec(Bla.join)
  File "<string>", line unknown
SyntaxError: Unexpected EOF while parsing argument list

This does not happen if join is a bound or unbound (not static) method.

Since this will affect building the docs (see #16296), I'd say the component is "documentation".

CC: @nthiery

Component: documentation

Keywords: staticmethod cython argspec

Author: Simon King

Branch/Commit: 143b567

Reviewer: Travis Scrimshaw

Issue created by migration from https://trac.sagemath.org/ticket/16309

[simon-king-jena]

comment:1

That's really silly, as the names and defaults are easy to get:

sage: Bla.join.func_code.co_varnames
('categories', 'as_list', 'ignore_axioms', 'axioms')
sage: Bla.join.func_defaults
(False, (), ())

[nbruin]

comment:2

People have complained about inspect.isfunction being to picky before and a monkey patch has been proposed: http://cython-devel.gmang.org/2013-August/003804.html

We wouldn't need to hack that badly. We can just go with ducktyping: if hasattr(obj,'func_code') we're probably looking at a function (line 1292 of sageinspect.py).

It's been noted that testing for <type 'cython_function_or_method'> cannot be done via isinstance, since every cython module makes a separate version of that type (which is considered a bug but unlikely to be fixed (soon)). By the time you're checking for the name of a type you're probably better off looking for a telltale attribute.

[simon-king-jena]

comment:3

Replying to @nbruin:

People have complained about inspect.isfunction being to picky before and a monkey patch has been proposed: http://cython-devel.gmang.org/2013-August/003804.html

Indeed it is a bit odd that the inspect module does type checks rather than duck typing.

We wouldn't need to hack that badly. We can just go with ducktyping: if hasattr(obj,'func_code') we're probably looking at a function (line 1292 of sageinspect.py).

That's an important information, that I will use in #16296 for the @abstract_method decorator.

Anyway, I suppose we need to somehow copy what is done in the inspect module, replacing the type check by duck typing.

[simon-king-jena]

comment:4

It gets worse.

sage: cython("""
....: class Bla:
....:     @staticmethod
....:     def join(categories, bint as_list = False, tuple ignore_axioms=(), tuple axioms=()):
....:         pass
....: """)
sage: from sage.misc.sageinspect import sage_getsource
sage: sage_getsource(Bla)
Traceback (most recent call last):
...
/home/king/Sage/git/sage/local/lib/python2.7/site-packages/sage/misc/sageinspect.pyc in sage_getsourcelines(obj, is_binary)
   1836                     if B is not None and B is not obj:
   1837                         return sage_getsourcelines(B)
-> 1838                 if obj.__class__ != type:
   1839                     return sage_getsourcelines(obj.__class__)
   1840                 raise

AttributeError: class Bla has no attribute '__class__'

However, the source of Bla.join is almost correctly determined:

sage: print sage_getsource(Bla.join)
    def join(categories, bint as_list = False, tuple ignore_axioms=(), tuple axioms=()):
        pass

This should include the decorator!

[nbruin]

comment:5

Replying to @simon-king-jena:

However, the source of Bla.join is almost correctly determined:

sage: print sage_getsource(Bla.join)
    def join(categories, bint as_list = False, tuple ignore_axioms=(), tuple axioms=()):
        pass

This should include the decorator!

Are you sure? Getsource does exactly what __doc__ indicates. It would be @staticmethod's responsibility to modify that. However, I think cython processes @staticmethod specially, so you'd need to file a bug to cython for that. I do agree it would be nice to see that a method is static, but presently you'd need to write that in the docstring yourself.

You are correct in noting that sage_getsource for @staticmethod in a .py file does include the decorator. That suggests that cython's @staticmethod processing should perhaps adjust Bla.join.func_code.co_firstlineno and/or Bla.join.__doc__, where it writes file and line number info too.

[simon-king-jena]

comment:6

Replying to @nbruin:

This should include the decorator!

Are you sure? Getsource does exactly what __doc__ indicates. It would be @staticmethod's responsibility to modify that.

OK, good point. I think it would be odd to fix that on our side.

[simon-king-jena]

comment:7

Inserting the following

    if hasattr(obj, 'func_code'):
        try:
            args, varargs, varkw = inspect.getargs(obj.func_code)
            return inspect.ArgSpec(args, varargs, varkw, obj.func_defaults)
        except (TypeError, AttributeError):
            pass

then the problem is solved for static methods of Python classes defined in Cython code, however it is not solved for static methods of cdef classes and cpdef methods of cdef classes.

[simon-king-jena]

comment:8

Next problem:

sage: cython("""
....: class Foo:
....:     @staticmethod
....:     def join(categories, bint as_list = False, tuple ignore_axioms=(), tuple axioms=()): pass
....: cdef class Bar:
....:     @staticmethod
....:     def join(categories, bint as_list = False, tuple ignore_axioms=(), tuple axioms=()): pass
....:     cpdef meet(categories, bint as_list = False, tuple ignore_axioms=(), tuple axioms=()): pass
....: """)
sage: from sage.misc.sageinspect import sage_getargspec, sage_getsource, sage_getsourcelines
sage: sage_getfile(Foo)
'/home/king/.sage/temp/linux-etl7.site/3169/spyx/_home_king__sage_temp_linux_etl7_site_3169_tmp_qz2nuW_spyx/_home_king__sage_temp_linux_etl7_site_3169_tmp_qz2nuW_spyx_0.so'

Shouldn't the file end with .pyx?

[simon-king-jena]

comment:9

Problem, that is probably the reason why sage_getargspecs has problems:

sage: source = '    def join(categories, bint as_list = False, tuple ignore_axioms=(), tuple axioms=()): pass\n\n'
sage: from sage.misc.sageinspect import _sage_getargspec_cython
sage: _sage_getargspec_cython(source)
  File "<string>", line unknown
SyntaxError: Unexpected EOF while parsing argument list

and this (as I just found) boils down to this:

sage: from sage.misc.sageinspect import _split_syntactical_unit
sage: _split_syntactical_unit('()): pass')
('())', ': pass')

The expected result is

('()', '): pass')

[simon-king-jena]

Branch: u/SimonKing/ticket/16309