[JuliaSyntax] Allow macros enclosed in parens to treat newlines as whitespace

[MasonProtter]

Closes #52842

Supersedes JuliaLang/JuliaSyntax.jl#475. cc @c42f and @JeffBezanson who I discussed this with previously.

Macros currently have an restriction that I always found very annoying: If you want newlines in a macro call currently, you need to switch from e.g.

@info "description" x=f(...) y=g(...) z=h(...)

to

@info("description", 
      x=f(...), 
      y=g(...),
      z=h(...))

which requires ugly commas separating the arguments, and a call-like syntax for @info. I would prefer to at least have the option to write this as

(@info "description" 
   x=f(...) 
   y=g(...)
   z=h(...))

which is currently a syntax error, but now works with this PR:

julia> (@info "description"
           x = sin(1.5)
           y = 2
           z = 3)
┌ Info: description
│   x = 0.9974949866040544
│   y = 2
└   z = 3

I see this feature as directly analogous to how we allow infix function calls to span multiple lines if they're in parens:

julia> (1 + 
          2)
3

julia> (1 + sin(π)
        - 2)
-1.0

---

It took me a couple years, but I finally figured out how to make this work in a backwards compatible way without breaking cases like

julia> parsestmt(Expr, """
       (@inline function bar()
            @baz x
            y 
       end)
       """)
:(#= line 1 =# @inline function bar()
          #= line 1 =#
          #= line 2 =#
          #= line 2 =# @baz x
          #= line 3 =#
          y
      end)

and this doozy which the mocking tests generated

julia> parsestmt(Expr, """
       Ex(@a, function   @g
                 @la   
             end)
       """)
:(Ex(#= line 1 =# @a(), function #= line 1 =# @g()
          #= line 1 =#
          #= line 2 =#
          #= line 2 =# @la
      end))

but in this revision of the PR these now correctly parse, and I'm seeing all the tests pass locally at least.

---

Easter egg

A silly side-effect of this change is that you can now write lispy s-expression syntax in julia with no end in sight!:

eval(Expr(:macro, Expr(:call, :macro, :sig, :body),
          Expr(:block,
               Expr(:call,
                    :esc,
                    Expr(:quote,
                         Expr(:macro,
                              Expr(:call,
                                   Expr(:$, Expr(:ref, Expr(:., :sig, QuoteNode(:args)), 1)),
                                   Expr(:$, Expr(:...,
                                                 Expr(:ref, Expr(:., :sig, QuoteNode(:args)), Expr(:call, :(:), 2, :end)))),),
                              Expr(:block, Expr(:$,:body))))))))

using Base: isexpr

(@macro call(args...)
 esc(Expr(:call, args...)))

(@macro def(x, y)
 esc(Expr(:(=), x, y)))

(@macro var"if"(cond, yes, no=nothing)
 esc(Expr(:if, cond, yes, no)))

(@macro block(args...)
 esc(Expr(:block, args...)))

(@macro var"let"(bindings, body...)
 (@block
  argv = (@if isexpr(bindings, :tuple)
          bindings.args
          [bindings])
  esc(Expr(:let, argv..., Expr(:block, body...)))))

(@macro fn(sig, body...)
 esc(Expr(:(=), sig, Expr(:block, body...))))

(@macro var"for"(spec, body...)
 (@block
  (@def newspec (@if (spec.args[1] == :∈)
                 Expr(:(=), spec.args[2:end]...)
                 spec))
  esc(Expr(:for, newspec, Expr(:block, body...)))))


(@fn f(x::T) where {T}
   (@let (@def y x)
      T(y) + 1))

[mbaz]

This change would make long @plot commands in Gaston.jl more comfortable to use. Currently, one must write multiline commands like this:

@plot(settings,
      {title = Q"Two sinusoids", key = "columns 1", key = "box outside right top"},
      x, y,
      plotline, {title = "'sin'"})

whereas the proposed change would allow:

(@plot settings
       {title = Q"Two sinusoids", key = "columns 1", key = "box outside right top"}
       x y
       plotline {title = "'sin'"})

It's not a big difference but I think the additional convenience is worth it.

[c42f]

Hey, I think the general approach makes sense. Compatibility is tricky as always.

parse_brackets() is also used for parse_vect(), so for x[a,b] and x{a,b} and [a,b] syntax and there's some extra test cases to add and edge cases to check for sanity. Detecting indexing vs concatenation syntax within [] and {} has always been a bit fragile. Eg, after this patch the following might be an inconsistency:

# old,new: parsed as vcat?
[@foo x
y]

# old: syntax error.  new: parsed as `vect`?
[x,
@foo y
z]

It may be best to only allow macro_whitespace_newline=true inside round brackets.

[MasonProtter]

Ah good point @c42f. I've fixed that and added a test-case for it.

[MasonProtter]

friendly ping @mlechu / @c42f