Adding new brackets to Julia - ideas wanted

[tonyhffong]

Related issue: #7128
A proof-of-concept PR: #8892 (more technical discussions inside)

Currently, we have (), [], {}. We could introduce more bracket types to Julia so oft-used functionalities are more convenient to use, such as non-concatenating array/matrix construction, perhaps even with some "leftovers" that modules can use to enable terser and more readable code (the latter is much motivated by my selfish reasons).

The PR proposes three new bracket types: ⟨⟩,⟦⟧,⦃⦄ (which can be entered by latex symbols \langle,\rangle, \lBrack, \rBrack, \lBrace, \rBrace). ⟦⟧ and ⦃⦄ also have ASCII equivalents [| |] and {| |}, respectively.

They are mapped according to whether there's something in front of the brackets. For example, (some_expression)⟦args...⟧ is mapped to @Brack_call( (some_expression), args... ), and ⟦args...⟧ is mapped to @Brack_enclose( args... ). The macro definition is in base and is currently defaulted to Brack_call and Brack_enclose generic functions in these examples. These can be stagedfunction, obviously, and it is part of the discussion below. Type signatures would provide the customization on actual behavior.

The key question is what kinds of functionalities should these new brackets be used for? Note that:

• The parsing rule inside these brackets can be different from one another. And they have to be decided for all users. For examples,
  • if we want to use ⟦⟧ for matrix construction, there are decisions on treatment on space, commas, semicolons, or perhaps even newlines, and how to make higher dimensional construction "easier", etc.
  • if we want g{| a, b, c |} to mean some kind of "lifted" function call, we may want to parse the list as argument list.
• Similar to the interior parsing rule, whether or not these bracket constructs would lower to a staged function, and what this staged function actually do are also a universal decision.

With respect to the PR, let's use this issue to keep track of design/usage ideas (my lisp code ugliness 😄, unicode issues etc. can be done in the PR ).

[stevengj]

I'd prefer to leave them free for packages to use.

[mauro3]

Letting packages define a meaning for them might also be confusing, in particular if they differ much. I remember being very confused by a python package using >> for file I/O (in python this is usually bitshift).

Maybe ⟦⟧ could be "reserved" for an indexing like operation. For instance, a graph datatype could use g[...] to index into vertices and g⟦...⟧ into edges.

[stevengj]

This issue here is that funny-shaped brackets don't really have a canonical meaning either in mathematics or in programming languages, so it doesn't really make sense to me to "bless" a meaning in Base unless we have a burning need for a new bracket-based syntax.

The fact that [...] is getindex in Julia will no doubt already encourage any package using ⟦...⟧ to use it for an indexing-like operation anyway; I don't see that we need to do more with this. (Unless there is a mathematical domain in which ⟦...⟧ has some other well-established meaning, in which case: more power to them.)

[tonyhffong]

Decisions made in parsing strategies tend to favor particular usages. For example, if the parser only recognizes commas to separate exprs' inside, it tends to favor a collection/ indexing type of usage. If the parser allow two kinds of separators (says, commas and semi-colons), it enables matrix usage. Maybe we want more kinds of separators. Maybe certain reserved words inside will change its meaning, like for in comprehensions. That is all going into parser. It would be difficult to influence that in base or in packages. (julia macros may wrestle back some control in base, but macros are not type sensitive so we cannot dispatch by package, only by the AST structure and only in base.)

Another note on multiple dispatch, since the brackets would be mapped to standard calls, e.g. g⟦args...⟧ -> Brack_call( :g, args...). Packages "claim" their usage by providing at least 1 package specific type in the args such as Brack_call( g::Symbol, args::MyPackageType...) and Brack_call( g::Symbol, ::Type{PackageType}, x::Any...). Calls using only common types Brack_call( g::Symbol, x::Int... ) would get nowhere unless the community wants it to mean something (say array with no implicit concatenation, or, if we decide vanilla [] becomes non-concatenating, the opposite).

I think a common usage would look like this (updated):

⟦PackageTokenType, args ... ⟧
# fancy_lookup⟦PackageTokenType, args ... ⟧ # deemed less desirable in the subsequent discussion
variable⟦ args ... ⟧

where PackageTokenType is an empty type uniquely defined by the package so it's pretty easy to realize that this is a special syntax enabled by that package.

Or, we would just use it in macros. The brackets parse, and the package can use them. I could see it having some potential use in DataFramesMeta.jl, but I'm completely shooting from the hip here.

[MikeInnes]

You'd almost definitely want the brackets to parse to a macro call, since that gives the implementor the most flexibility (similar to @~ currently). Maybe I'm missing something but I don't think passing the called symbol to a function really makes sense. You'd probably want ⟦...⟧ and foo⟦...⟧ to dispatch to different macros, too.

You make a good point about the intended use affecting the parsing, though. I can definitely see it being useful to parse ⟦⟧ like an array as opposed to a tuple (with semicolons, for etc.), to allow packages to define a second array syntax.

[tonyhffong]

@one-more-minute , yes they all parse into macros first in the WIP PR. And you have a good point about passing the symbol, especially for a ref-like foo⟦...⟧ expression! It should parse into Brack_call( foo, args... ) instead of Brack_call( :foo, args... ), so it can dispatch on the type of foo. Cool.

And to your point foo⟦...⟧ dispatches to Brack_call( foo, args... ). And ⟦...⟧ dispatches to Brack_enclose( args... )

I would like to reserve Brace foo{| |} to pass the symbol :foo though, as I'm working on a package that could make good use of it.

[MikeInnes]

Sounds good. You may be aware of this but just in case, I'll mention that you don't need foo{| |} to be reserved in any way (by the parser or base) to pass the symbol, since a macro will let you do this and more. For example:

macro struckcurlycall(f, args...)
  Brack_call($(Expr(:quote, f)), $(args...))
end

Then foob⦃a, b, c⦄ is parsed to @struckcurlycall f a b c which expands to Brack_call(:f, a, b, c).

(Apologies if I just misunderstood what you meant by "reserved", though)

[tonyhffong]

@one-more-minute your suggestions have been incorporated into the PR.

[JeffBezanson]

+1 to @one-more-minute 's comments.

[tonyhffong]

Thanks for the input. To summarize, I now have square bracket do this

macro call_Brack( args... )
    esc( Expr( :call, :call_Brack, args... ) )
end

And, Brace bracket does this:

macro call_Brace( sym, args... )
    esc( Expr( :call, :call_Brace, Expr(:quote, sym), args... ) )
end

So we cater to two types of usage

• a[| b |] is more "ref-like". This is for working with an existing value a. Considering that this is likely to be the commonest usage, I have made the angle bracket behave the same way.
• a{| b |} is for working with the expression a that could be anything. e.g. (a*b){| c |} will be lowered to call_Brace( :(a*b), c ). It is up to individual methods to deal with that symbol or expression.

The lowering of the enclosing form (without prefix) is the same for all brackets. For example:

macro enclose_Brack( args... )
    esc( Expr( :call, :enclose_Brack, args... ) )
end

[tonyhffong]

Reaction is too mixed. I'm closing it.