J2CL transpiled compiler Scanner.java doesn't understand non-ascii characters.

[brad4d]

JavaScript is supposed to allow non-ASCII unicode letters in identifiers, but the gwt-compiled version of Scanner.java doesn't recognize them because GWT's version of Character.isLetter() doesn't understand unicode..

https://github.com/google/closure-compiler/blob/master/src/com/google/javascript/jscomp/parsing/parser/Scanner.java#L775

Esprima has written their own logic for this in TypeScript here.
https://github.com/jquery/esprima/blob/master/src/character.ts

We should emulate this logic in Scanner.java instead of relying on Character, since it's buggy in GWT.

[ctjlewis]

We should emulate this logic in Scanner.java instead of relying on Character, since it's buggy in GWT.

Does this still hold? I figured getting Character.isAlphabetic and constant fields like Character.COMBINING_SPACING_MARK exposed to J2CL would definitely help with this, and let us rewrite the logic in Scanner.java to be future-proof. See google/j2cl#103, issue #3639, and PR #3645.

I'm focused on the isAlphabetic method because it single-handedly matches Lu, Ll, Lt, Lm, Lo, NI characters for IdentifierPart grammar in the ECMA spec. I'm happy to go through and hardcode all of this grammar, but it will break as new Unicode characters are added (hence #3639) - it seems like most long-term reward would be given by getting Character to interface well with J2CL but I could be very wrong.

[brad4d]

AFAIK, switching to J2CL did not make this situation better.
I doubt fixing it is ever going to be a priority.
Supporting the Java version of the compiler and the Graal-compiled native version are our focus.
The compiled-to-JS version is strictly a nice-to-have.