at-dot macro for adding dots to function calls #20321

stevengj · 2017-01-30T06:33:22Z

The PR adds a new macro @. (and parser support to convert this to ~~@__DOTS__~~ ~~@__DOT__~~ @__dot__) that adds dots to every function call, operator, and assignment in an expression. For example,

@. x += 3x^2 - sqrt(5x)

is equivalent to x .+= 3 .* x.^2 .- sqrt.(5 .* x).

You can opt-out of the dots for specific function calls by splicing with $, e.g. @. sqrt($sort(abs(x))) is equivalent to sqrt.(sort(abs.(x))). (This also works for function objects spliced directly into expressions by other means, e.g. it allows @. to work in conjunction with @views, and it should similarly allow some interoperability with other macros.)

cc @StefanKarpinski, since @. was (I think?) his suggestion.

nalimilan · 2017-01-30T09:12:01Z

The feature is really cool, but I wonder whether a more descriptive name wouldn't be in order. @broadcast?

tkelman · 2017-01-30T12:33:01Z

doc/src/stdlib/arrays.md


 ```@docs
 Base.broadcast
 Base.Broadcast.broadcast!
+Base.__DOTS__


we want to document the macro, not the helper function, right?

Right, whoops.

tkelman · 2017-01-30T12:38:30Z

base/exports.jl

@@ -1382,6 +1382,7 @@ export
    @polly,

    @assert,
+    @__DOTS__,


why export this if only lowering should see it under this name?

stevengj · 2017-01-30T15:57:22Z

I'm of two minds about the name.

If it were an ordinary macro name, I would prefer @dots (to emphasize that it turns things into fusing dot calls, not just broadcast calls). This has the advantage of not requiring any parser sugar.
@. is shorter and thus more lightweight and tempting to insert into code. And the specialness of the macro name is maybe reflective of the specialness of the dot-call sugar?

@StefanKarpinski, you suggested the @. syntax, do you have any further thoughts on why you preferred that over an ordinary macro name?

stevengj · 2017-01-30T17:30:28Z

(The other thing to keep in mind with @dots vs. @broadcast is that, going forward, I expect that only a minority of Julia users will call broadcast directly. More people will be familiar with the dot syntax — indeed, more people already are for operators, thanks to Matlab.)

StefanKarpinski · 2017-01-30T21:58:22Z

My thinking was that @. will be the easiest to remember: "I have all these dots all over the place, gee, isn't there a way to avoid those... oh yeah just write @. at the beginning. That was easy!" Otherwise people need to remember what name replaces all the dots. Keep in mind that many users will use f.() syntax and .+ operators without ever knowing or caring that they're implemented as higher order calls to broadcast. To them it will just be mysterious why one uses @broadcast to put dots on everything. There's also the advantage that @. can't possibly collide with any macro name.

StefanKarpinski · 2017-01-30T22:01:43Z

I see that you just said effectively the same thing, @stevengj :)

StefanKarpinski · 2017-01-30T22:02:45Z

Wouldn't it be cleaner to just allow the parser to recognize @. as a valid macro and internalize it as calling the $(Symbol("@.")) expander function?

stevengj · 2017-01-31T00:14:02Z

@StefanKarpinski, there is a lot of code that assumes a certain form for identifiers. e.g. the documentation system broke when I tried to attach a docstring to @. ...I could get it to work, but I was worried that I'd have to insert special-case code all over the place. The macro seems a lot easier to work with if it is just sugar for a normal Julia identifier.

@tkelman, the reason I export @__DOTS__ and have @. expand to the unqualified @__DOTS__ (rather than to Base.@__DOTS__) is that this way you can override @. inside a module by defining your own __DOTS__ macro, which seemed to add flexibility. (That's also why the __DOTS__ name is documented in the @. docstring.)

StefanKarpinski · 2017-01-31T03:28:20Z

Makes sense. It's definitely a nit, but should this be called @__dot__ instead? Singular because it corresponds to the syntax @. which has only a single dot. This happens to make it correspond, but default, to a behavior which puts many dots on things, but that might not be what other people define it to do. In generally, I've found that syntactic features with special names, like getindex and setindex! end up being less confusing if the special names correspond to the syntax rather than the default meaning of that syntax. The lowercase part is because we've usually used special names like __init__ have been lowercase. Otoh, we have @__FILE__ and @__DIR__, so that could be argued either way – I just thought we should be cognizant of the choice.

stevengj · 2017-01-31T04:33:24Z

I'm fine with @__dot__ or @__DOT__. I made it uppercase to correspond to the other "special" macros, but I don't have any strong preference.

…intro somewhat

tkelman · 2017-01-31T05:17:53Z

Lowercase would be preferable to me as it behaves more like a function modifier, and less like something from the C preprocessor which many of the other dunder-caps macros behave like.

StefanKarpinski

Bravo. That's a lovely PR.

StefanKarpinski · 2017-01-31T17:33:49Z

base/broadcast.jl

+            Expr(x.head, dotargs...)
+        end
+    end
+end


I should remember in the future that this is a perfect example of why you cannot do serious metaprogramming with strings and why you need data structures for it – a complex, recursive syntax transformation.

Sacha0 · 2017-01-31T18:12:35Z

doc/src/stdlib/arrays.md

-All mathematical operations and functions are supported for arrays
+See also the [dot syntax for vectorizing functions](@ref man-vectorized);
+for example, `f.(args...)` implicitly calls `broadcast(f, args...)`.
+Rather than relying on "vectorized" methods of function like `sin`


Should "of function like sin" be "of functions like sin"?

Sacha0 · 2017-01-31T18:15:54Z

test/broadcast.jl

@@ -244,6 +247,28 @@ let x = [1:4;]
    @test sin.(f17300kw.(x, y=1)) == sin.(f17300kw.(x; y=1)) == sin.(x .+ 1)
 end

+# splice escaping of @.
+let x = [4, -9, 1, -16]
+    @test [2, 3, 4, 5] == @.(1 + sqrt($sort(abs(x))))


Inconsequential, but missing a space between the @. and following expression?

If I'm not mistaken, I think in this case the macro is just being called with the parenthesized form, in which case there shouldn't be a space after @..

Ah, of course! Thanks for straightening me out :).

Sacha0 · 2017-01-31T18:20:20Z

test/broadcast.jl

    @test y === x == [7,6,5,4]
    x[1:2] .= 1
    @test y === x == [1,1,5,4]
-    x[1:2] .+= [2,3]
+    @. x[1:2] .+= [2,3]


Did you leave the dots in place here and just below to exercise @. when dots already exist, or did you mean to remove the dots here and just below? (A comment or two on these tests explaining your intention might help those who come after?)

Sacha0 · 2017-01-31T18:21:16Z

test/subarray.jl

@@ -479,7 +479,7 @@ Y = 4:-1:1

 @test isa(@view(X[1:3]), SubArray)

-@test X[1:end] == @view X[1:end]
+@test X[1:end] == @. @view X[1:end]


Was your intention here to exercise the interaction of @. with @view? If so, perhaps add a comment to that effect?

Or even better, put it in a testset. 🙂

Sacha0

This looks great! (Review regrettably brief.)

stevengj · 2017-02-02T03:22:28Z

AppVeyor failure seems like an unrelated network glitch: Cannot clone ColorTypes from https://github.com/JuliaGraphics/ColorTypes.jl.git. Failed to receive response: The server returned an invalid or unrecognized response

stevengj · 2017-02-02T03:23:59Z

Should be good to merge?

tkelman · 2017-02-02T04:14:08Z

nevermind, dunno what I was looking at

GeoffChurch · 2019-12-11T19:24:29Z

Just out of curiosity, was there a reason for the opt-out $ coming before the function name? I thought it would have inherited the broadcast dot placement, i.e.

@. sqrt(sort$(abs(x)))

instead of

@. sqrt($sort(abs(x)))

stevengj · 2019-12-11T22:14:19Z

@GeoffChurch, $symbol is the standard way of interpolating things directly into expressions in Julia.

GeoffChurch · 2019-12-11T23:40:56Z

Oh that makes sense, thank you

at-dot macro for adding dots to function calls

ec8adc8

stevengj added the broadcast Applying a function over a collection label Jan 30, 2017

stevengj requested a review from StefanKarpinski January 30, 2017 06:33

NEWS for at-dots

efa588b

tkelman reviewed Jan 30, 2017

View reviewed changes

fix doc ref to __DOTS__

5522239

stevengj added 2 commits January 30, 2017 20:40

doc tweak: 'Broadcasting' was not a unique title, expand the section …

658a371

…intro somewhat

__DOTS__ -> __DOT__

c693ac1

stevengj added 3 commits January 30, 2017 21:46

__DOT__ -> __dot__

88b9d4e

don't add redundant dots in at-dot

370920e

do add dots to .. operator in at-dot

29bb432

This was referenced Jan 31, 2017

support dots for unary operators #20249

Merged

update for new vectorization roadmap lindahua/Devectorize.jl#52

Open

StefanKarpinski approved these changes Jan 31, 2017

View reviewed changes

Sacha0 reviewed Jan 31, 2017

View reviewed changes

Sacha0 approved these changes Jan 31, 2017

View reviewed changes

stevengj added 3 commits January 31, 2017 14:19

typo

e81d3ec

comments on at-dot tests

2286a66

test tweaks

b0770f6

StefanKarpinski merged commit 2e0290a into JuliaLang:master Feb 2, 2017

stevengj deleted the at-dot branch February 2, 2017 13:11

stevengj mentioned this pull request Feb 3, 2017

add at-dot, at-view, and at-views macros; improve at-compat for .= & .+= JuliaLang/Compat.jl#316

Merged

IlyaOrson mentioned this pull request Jun 15, 2017

Allow non-vectorized model functions? JuliaNLSolvers/LsqFit.jl#12

Closed

tlienart mentioned this pull request Mar 27, 2020

Chasing dead links JuliaLang/www.julialang.org#690

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

at-dot macro for adding dots to function calls #20321

at-dot macro for adding dots to function calls #20321

stevengj commented Jan 30, 2017 •

edited

Loading

nalimilan commented Jan 30, 2017

tkelman Jan 30, 2017

stevengj Jan 31, 2017

tkelman Jan 30, 2017

stevengj commented Jan 30, 2017 •

edited

Loading

stevengj commented Jan 30, 2017

StefanKarpinski commented Jan 30, 2017

StefanKarpinski commented Jan 30, 2017

StefanKarpinski commented Jan 30, 2017

stevengj commented Jan 31, 2017 •

edited

Loading

StefanKarpinski commented Jan 31, 2017

stevengj commented Jan 31, 2017

tkelman commented Jan 31, 2017

StefanKarpinski left a comment

StefanKarpinski Jan 31, 2017

Sacha0 Jan 31, 2017

Sacha0 Jan 31, 2017

ararslan Jan 31, 2017

Sacha0 Jan 31, 2017

Sacha0 Jan 31, 2017

Sacha0 Jan 31, 2017

ararslan Jan 31, 2017

Sacha0 left a comment

stevengj commented Feb 2, 2017

stevengj commented Feb 2, 2017

tkelman commented Feb 2, 2017

GeoffChurch commented Dec 11, 2019

stevengj commented Dec 11, 2019

GeoffChurch commented Dec 11, 2019

at-dot macro for adding dots to function calls #20321

at-dot macro for adding dots to function calls #20321

Conversation

stevengj commented Jan 30, 2017 • edited Loading

nalimilan commented Jan 30, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

stevengj commented Jan 30, 2017 • edited Loading

stevengj commented Jan 30, 2017

StefanKarpinski commented Jan 30, 2017

StefanKarpinski commented Jan 30, 2017

StefanKarpinski commented Jan 30, 2017

stevengj commented Jan 31, 2017 • edited Loading

StefanKarpinski commented Jan 31, 2017

stevengj commented Jan 31, 2017

tkelman commented Jan 31, 2017

StefanKarpinski left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Sacha0 left a comment

Choose a reason for hiding this comment

stevengj commented Feb 2, 2017

stevengj commented Feb 2, 2017

tkelman commented Feb 2, 2017

GeoffChurch commented Dec 11, 2019

stevengj commented Dec 11, 2019

GeoffChurch commented Dec 11, 2019

stevengj commented Jan 30, 2017 •

edited

Loading

stevengj commented Jan 30, 2017 •

edited

Loading

stevengj commented Jan 31, 2017 •

edited

Loading