Method ambiguity detection: false positive?

[pygy]

foo(b::Bar...) = 1
# foo (generic function with 1 methods)

foo(a::Any, b::Bar) = 2
# foo (generic function with 2 methods)

foo(b::Bar, a::Any) = 3
# Warning: New definition
#     foo(Bar,Any)
# is ambiguous with:
#     foo(Any,Bar)
# To fix, define
#     foo(Bar,Bar)
# before the new definition.
# foo (generic function with 3 methods)

Semantically, foo(Bar,Bar)` is covered by the first definition, and the above example behaves as expected.

Is the warning really warranted?

[jiahao]

This actually seems like the perfect example of why an ambiguity warning is needed, since the dispatch is not greedy and thus has to make an arbitrary choice of which method to dispatch.

[lindahua]

Both method 2 and 3 fit the call foo(Bar, Bar). So it is said to be ambiguous.

[pygy]

Yes, actually, it makes sense.

[pygy]

Wait, no.

1, 2 and 3 all fit foo(Bar,Bar), yet you only get the error for the third definition. The call is actually dispatched to 1 (in this case, at least).

@JeffBezanson what gives?

[elextr]

@JeffBezanson can confirm, but IIUC ... parameter lists are considered a worse fit than a function with the correct number of parameters, so 2 clearly wins over 1, but 2 and 3 both have two parameters and they both match foo(Bar,Bar) equally well.

[pygy]

But here, 1 is winning. foo(Bar(), Bar()) yields 1.

Another strange one:

|(b::Bar, u::Union(Function, Int)) = 4
# Warning: New definition
#     |(Bar,Union(Int64,Function)) at ...
# is ambiguous with:
#     |(Any,Function) at deprecated.jl:19.
# To fix, define
#     |(Bar,Function)
# before the new definition.

[mbauman]

That second example makes sense… you're clashing with the old deprecated pipe operator (now |>). Bar is more specific than Any in arg1, but Function is more specific than the Union in arg2. That is indeed ambiguous.

But I agree that it seems like (b::Bar...) should live somewhere between (a::Bar, b::Bar) and (a::Bar, b::Any).

[elextr]

On re-considering, case 1 has only one argument, a tuple of Bar, case 2 and 3 require two arguments, so case 1 cannot disambiguate them.

And also cases 2 and 3 have an Any, so they are less specific than 1, which is an exact match to (Bar,Bar), and so 1 gets dispatched.

[pygy]

@mbauman: Isn't

|(b::Bar, u::Union(Function, Int)) = 4

equivalent to

|(b::Bar, u::Function) = 4
|(b::Bar, u::Int) = 4

? The Union may be less specific, the the pair (Bar, Union(Function,Int)) is more specific than (Any, Function).

@elextr:

So, basically, 1, which can't disambiguate 2 and 3 for arity reasons, disambiguates them anyway because it is still more specific, from a type standpoint.

Does not compute :-/

If 1 prevails, deterministically, it should be considered to disambiguate 2 and 3, and the warning should not be displayed.

[elextr]

Case 1 should fail to disambiguate both for arity and type reasons, the type of b is a tuple (of unknown length) of Bar. So at definition time it does not match either case 2 or 3.

Note that the tuple length is not defined until the actual call, and the majik that has ... compile a tuple of the actual call args only for the case 1 before choosing which to dispatch.

Both +(Bar()) and +(Bar(),Bar(),Bar()) are legal calls and dispatch to case 1, but have no relation to cases 2 or 3.

The Bar() + Bar() call matches (Bar,Bar) created by the ... against case 1, which is an exact match. It also matches Bar to Any and Bar to Bar for case 2 and Bar to Bar and Bar to Any for case 3, both of which are less specific matches than an exact match at case 1.

But all that is at call time, its not information available at definition time when the warning happens.

Thats dynamic dispatch though ;)

[pygy]

I don't understand why the existence of a variadic function can't be taken into account at compile time...
The argument list will be turned into a tuple for inner consumption, but the compiler could know that 1, as a variadic function, will come before 2 and 3 when called at runtime.

What prevents it from being deduced from the source code?

Aside, apologies, for the confusion in the code examples, I went from foo() to +() back to foo() in the original post.

[elextr]

Well, everything is possible but this would require some special handling, as I said case 1 has an arity of 1 and a type of tuple. So type deduction won't work, it needs special casing that the last parameter is variadic, and check that some expansion of the variadic could be a better match for the ambiguous case and so hide it.

Its up to the experts, but to me it seems like a lot of work in the compiler to suppress a warning that actually tells you what to do to prevent it, define +(Bar,Bar) first.

Defining that also means you have explicitly thought about it and made it do something sensible, not merely forgotten that the variadic will be called in the Bar() + Bar() case.

[pygy]

So that's the (current) of affairs, I thought that you were trying to explain a fundamental limitation, and, of course, I couldn't make sense of it.

Performance/exhaustiveness compromises are sometimes necessary, no problem with that.

In my case, I'm relying on reduce to handle +(Bar...), and it properly takes care of all cases, including the unary +.

I've added +(Bar, Bar), and |(Bar, Function) special cases, to avoid the warnings.

I'm leaving this open, for now, for future reference.

[ivarne]

This looks a lot like #5536

[JeffBezanson]

I do think this warning is unnecessary. Definitions 2 and 3 are ambiguous with each other, but it doesn't matter because neither of them would be called for Bar,Bar anyway.

Types are more important than argument count for specificity. A definition f(x::Bar...) is clearly defining f for type Bar over all other types, and the fact that it handles any number of arguments is just a bonus. This point is debatable but I find this interpretation the most useful in practice.

[timholy]

Closed by #16125 or #16220

[pygy]

Indeed, many thanks :-)