overload call / () operators?

[stevengj]

It might be nice if we could overload call (i.e. ()), so that one could provide a callable object and still have access to other data within it.

For example, I'm running into this with my PyCall module. Many Python objects are callable as functions, but I don't want to automatically convert all of them into anonymous functions because you still want to have access to other members/attributes of the objects. Currently, you have to do pycall(object, ...), but it would be much nicer if I could overload () so that object(...) was converted into a pycall as users expect.

[johnmyleswhite]

+1

I kept wanting to do this to make it easier to package a function with its gradient as a single object.

[pao]

See also (but not dup of) #1974.

[StefanKarpinski]

Jeff and I were just talking about making f(args...) syntax for apply(f,args...) (i.e. making function calling overloadable) and then jamming all dispatch into overloading apply. It's more subtle than that, however, because this hides a syntactic circularity/ambiguity: apply(f,args...) can't mean the same thing as f(args...) or you haven't gotten anywhere at all. This is related to types being callable, the possibility of being able to dispatch/specialize on function values (which we don't seem to have an issue for), and the idea dispatching on operator classes (can't find the discussion on that either). I feel that there's a possible unification here, but it's a very deep problem, and is going to require some serious consideration before we know what, if anything, to do about it.

[StefanKarpinski]

By "can't mean the same thing" I mean even though they look syntactically similar, they can't have the same meaning or you're defining apply(f,args...) in terms of itself. apply would most likely have to be a pseudofunction like ccall or new inside of type blocks.

[stevengj]

@StefanKarpinski, I'm not sure I understand the circularity. Make f(args...) turn into apply(f, args...), and make apply(f::Function, args...) go to jl_callback_call or some similar low-level function. Why is this a deep problem?

[pao]

Because apply(f::Function, args...) will dispatch to apply(f::Function, args...) with f = apply if isa(apply, Function).

[stevengj]

But you just need a single special case in the parser to eliminate this.

[StefanKarpinski]

Right, but that special case is precisely to avoid the circularity. Anyway, I'm being concrete enough for this to be very helpful. I'll try to come up with something more concrete soon.

[StefanKarpinski]

I'm not being concrete enough, rather.

[jwmerrill]

+1

The various polynomial packages define polyval or something like it for evaluating polynomials at a point. It would be nice if they could just overload function call.

ApproxFun ended up overloading getindex instead, as a partly justified workaround. There was some interesting discussion on the mailing list at the time.

Today I'm working on the Fast Gauss Transform for evaluating the convolution of a set of points with a gaussian kernel (similar to the existing KernelDensity package, but with a different algorithm). So far I have an api that looks like

t = fastgausstransform(points, values, bandwidth)
evaluate(t, 2.0)

but I have some anxiety about collisions if I export a function called evaluate, and I wish I could instead have

t = fastgausstransform(points, values, bandwidth)
t(2.0)

I realize I could return an anonymous function instead, but I would like to support more operations on the transforms, like differentiating or adding them.

[quinnj]

I would imagine this would also lend to a nicer solution here, so you know, it should probably be a high priority and stuff...

[toivoh]

Regarding syntax, I guess we would need syntax for defining functions and their superfunctions ( / supertypes) and not just syntax for methods as we have now.

Would we differentiate between abstract and concrete functions, or should all functions be subtypable? (I guess it comes down to what's needed for efficiency?)

[rfourquet]

Wouldn't it be useful to have call overloading handle optional ! for mutating operations? e.g. via a parameter, or a call! function.
Here is a hypothetical exemple where this would help to keep a consistent API: let's say rand(rng, dims) becomes rng(dims), with rng=MersenneTwister(), via call(rng::AbstractRNG, dims). Then it could be nice to have rand!(rng, array) become rng!(array), via call!(rng::AbstractRNG, array), and then be able to call rng(dims) or rng!(array) with the same rng instance (and without having to define an alias rng! =rng).

[stevengj]

I'm a little dubious about that. Up to now, ! hasn't really been part of the language, just a convention. Worse, this means that whether Julia uses call or call! would depend on the name of the variable (rng vs. rng!) being invoked, rather than on the value. (Because you still have to be able to handle variables named rng! unless you want to completely change the language to disallow ! in identifiers.)

[stevengj]

See the working branch #8008.

[rfourquet]

I think this idea is not compatible with ! in identifiers, as it would be here part of the syntax of call overload: var(...) would translate to call(var, ...) while var!(...) would translate call!(var, ...).

[stevengj]

@rfourquet, if you have a function named foo!, how would you call it? In your proposal, foo!(...) would turn into call!(foo, ...), which would fail if foo doesn't exist (and probably wouldn't be what you want otherwise). This is the difficulty with allowing ! to both be a part of an arbitrary identifier and to be semantically meaningful (distinguishing between call and call!).

[JeffBezanson]

It might make sense to have something about mutating functions built into the language, but using a magic ! is not a good way to do it. ! should just stay part of identifier names.

[StefanKarpinski]

Rust's ownership model is quite interesting but seems far to finicky to use as is in Julia. I wonder if we could borrow some of the ideas and make a system that's less challenging for the proverbial "non-professional programmer" but keeps enough of the benefits to be worth it.

[JeffBezanson]

Implemented by #8712.