Create a plugin to handle singledispatch

[gbritton1]

For me, this code:

from functools import singledispatch
@singledispatch
def cmp(other, match):
    return match in (..., other)

@cmp.register(type)
def _(other, match):
    return isinstance(match, other)`

results in the mypy error:

severity: 'Error'
message: 'error:'overload' decorator expected'

on the line @cmp.register(type)

mypy version:

PS C:\Program Files (x86)\Python36-32> .\scripts\mypy.bat --version
mypy 0.471
PS C:\Program Files (x86)\Python36-32>

What should I do to resolve the error?

[ambv]

I cannot reproduce your error. I installed mypy 0.471, copied your example code and ran it against mypy, no errors. Can you provide an exact minimal example where running against mypy 0.471 produces the output you're seeing?

I tried to no avail with various combinations of options:

mypy --warn-incomplete-stub --warn-redundant-casts --warn-no-return --warn-unused-ignores --show-error-context --strict-optional --follow-imports=normal --python=3.6 --fast-parser

[gbritton1]

I attached a file with the exact code
testmypyerror.zip

also, here's the console log from the mypy run:

D:\Python>"C:\Program Files (x86)\Python36-32\Scripts\mypy.bat" d:\python\testmypyerror.py
d:\python\testmypyerror.py:8: error: 'overload' decorator expected
d:\python\testmypyerror.py:12: error: 'overload' decorator expected

D:\Python>

[gbritton1]

FWIW:
D:\Python>"C:\Program Files (x86)\Python36-32\Scripts\mypy.bat" --version
mypy 0.471

[miedzinski]

Mypy will produce this error if you have more than one named function _. In code sample attached to PR there is only one _, so mypy is fine with this. @gbritton1 is this correct?

[gbritton1]

In the code sample, there are actually two functions named _. But those names are not ever used. Since the functions are decorated using the singledispatch decorators, those names are merely placeholders.

mypy is not fine with this! That's why I filed this PR. I get the errors I posted above. Note, if I rename the second function to e.g, "_42" the error goes away.

Perhaps this is outside the scope of what mypy can do? I mean, how is mypy to know that the decorator actually results in no function named "_" to be created?

OTOH the error messages are misleading, at least to me.

FWIW I simply followed the docs at https://docs.python.org/3/library/functools.html which give the example:

@singledispatch
def fun(arg, verbose=False):
    if verbose:
        print("Let me just say,", end=" ")
    print(arg)
@fun.register(int)
def _(arg, verbose=False):
    if verbose:
        print("Strength in numbers, eh?", end=" ")
    print(arg)
@fun.register(list)
def _(arg, verbose=False):
    if verbose:
        print("Enumerate this:")
    for i, elem in enumerate(arg):
        print(i, elem)

which also yields the same mypy error messages

[miedzinski]

Perhaps this is outside the scope of what mypy can do? I mean, how is mypy to know that the decorator actually results in no function named "_" to be created?

_ actually is created (and redefined if you declare it more than once) - try using it. singledispatch registers decorated function, but doesn't somehow hide it from scope. It is still there. I agree that the error message could be better though.

This is related to #1332. Until this gets "fixed", you can define functions with names like _int, _list. Depending on your use case you will probably need @overload as well.

[gbritton1]

ah I see, now, you are quite correct. FWIW singledispatch could throw away the "_" function, I think that is, if I add

del _

then

print(fun(42))

it all works just fine.

Perhaps i should file a bug report against singledispatch. My gut tells me it should delete "_"

In fact, following your suggestion (_int, _list, etc), the namespace gets polluted with these functions

[miedzinski]

You shouldn't add del statements and certainly singledispatch won't do neither. Names prefixed with _ aren't exported anyway. Just use different names. I wouldn't be surprised if mypy despite allowing redefining _ variables would still prohibit such functions without @overload.

[gbritton1]

I added del to prove the concept. but what is your reasoning for not insisting singledispatch clean up after itself? IOW what is the value of the decorated functions with their original names? Especially when you follow the docs and use the same name for each one, so that at the end only the last one is actually defined?

I realize this is now OT. Apologies! Just trying to learn.

[miedzinski]

Because one may want to use both @singledispatch function and the exact implementation. It wouldn't be possible if singledispatch silently deleted your functions. If you really insist on "cleaning up", then del these functions yourself, but I don't understand why you would want that. Prefix them with _ and they won't be exported.

And yes, mypy issue tracker isn't the best place to discuss stuff like that, so let's end this. :)

[gvanrossum]

OK, so the full example (copied from the zip) is:

from functools import singledispatch
from typing import Sequence

@singledispatch
def cmp(other, match):
    return other in (..., match)

@cmp.register(type)
def _(other, match):
    return isinstance(match, other)

@cmp.register(Sequence)
def _(other, match):
    return all(cmp(m, o) for m, o in zip(match, other))

The reason mypy complains about this is the duplicate use of _. The reason is that to mypy, there is nothing special about @cmp.register it's just some decorator. I can reproduce the same behavior simpler:

def deco(f): return f
@deco
def foo(a): return 0
@deco
def foo(a): return 1

TBH I think the error message could be clearer. The error is much clearer when the redefined function is undecorated, e.g. this gives b.py:2: error: Name 'foo' already defined:

def foo(a): return 0
def foo(a): return 1

But either way mypy is just complaining about a redefinition, I'm not sure how we should address this. Should we special-case single-dispatch? Or should we special-case _?

[ilevkivskyi]

I think _ should be special-cased. It is often used as a "throw away" variable.

[JukkaL]

Special casing _ in assignment statements has an existing issue: #465