allow identifiers to start with category No (Number, other)

[stevengj]

This PR allows identifiers to start with with numeric symbols in category No: Number, other, allowing you to have variables with names like ⅓ or ⅞U or ¹x₂. (Category No was already allowed in identifiers, just not as the first character.)

This is a strict increase in the set of allowed identifiers, so it shouldn't break any existing code. And it seems inconceivable to me that we would ever allow category No to represent numeric literals etc., so we might as well allow them to start identifiers.

I also added tab completion for the various "vulgar fractions," e.g. \7/8<tab> to get ⅞. (added in #25980)

[stevengj]

(Of course, if you are perverse, this allows you to define e.g. 🄂🄄4 = (1,3,4) since there are unicode characters in No for decimal digits followed by commas. However, there are lots of ways to write unreadable code, and I don't think this PR makes the situation any worse.)

[JeffBezanson]

Call me crazy, but many might expect ⅓ to equal 1//3. I'm fine with obscure identifier names, but I'm less sure about things that look very much like they mean one thing, but actually mean something else.

[stevengj]

We could certainly export const ⅓ = 1//3 at some point (given this PR), though I don't think it's worthwhile until #14324 is addressed.

However, it is quite common to have variable names like xhalf, and it would be nice to be able to use e.g. ½x instead. This PR allows us to do that and to define const ½ = 1//2 if the latter becomes efficient enough to be useful. (Of course, if you define ½x = 3x, that would be confusing, but so would xhalf = 3x.)

Identifiers starting with superscripts and subscripts also seems like a big win to me, since this is standard notation in a lot of areas (e.g. see hypergeometric functions).

[JeffreySarnoff]

I am queasy with the pairing of this current behavior
x = 1; 2x == 2 * x [allowed] and y = 6; 2y = 12 [error]
with this proposed
const ½ = 1//2;
x = 4; ½x == 1//2 * x [allowed] and y = 6; ½y = 3; [allowed]

[stevengj]

@JeffreySarnoff, the proposal does not imply ½x == ½ * x. Category No symbols like ½ are not numeric literals (either in the current or proposed behavior) and hence do not support multiplication by juxtaposition.

As for allowing people to define ½ = 7 … we already allow two = 17. Nothing will stop people from defining confusing variable names if they want to.

[stevengj]

One option would be to allow category No characters to start a variable name, but to require that the variable name also include non-No characters. i.e. allow ½x as a variable name, but disallow ½ by itself. This makes a lot of sense to me, since ¹x₂ makes sense as a variable name but ¹ does not.

This would be strictly more conservative … we could always allow ½ etcetera later.

Would that address your objections, Jeff?

[stevengj]

I've made this change more conservative as suggested above: identifiers can start with category No, but cannot consist solely of this category:

julia> ½ = 2
ERROR: syntax: invalid identifier ½

julia> ½x = 2
2

[stevengj]

Rebased; it would be nice to get a decision on this for 1.0, although since it is a non-breaking change it could technically wait for 1.1 or later. cc @ararslan.

[ararslan]

Marking for triage for a 1.0 decision but I agree that this could wait until after 1.0.

[JeffBezanson]

although since it is a non-breaking change it could technically wait for 1.1 or later

Those are the magic words :)

[stevengj]

This PR was postponed for post Julia 1.0. Can it be revisited and a decision made?

[JeffBezanson]

From triage: +1 to disallowing No characters by themselves. We also see no harm in allowing super- and subscripts, but we want to exclude the fractions for now so we can at least make some clear progress here.

[JeffBezanson]

Ok looking at the full list (https://www.compart.com/en/unicode/category/No) there is quite a lot of stuff in that category, e.g. parenthesized numbers. So I would lean towards only adding the superscripts and subscripts.