Case insensitivity doc

[parrt]

In response to the lively discussion on case insensitive lexing,I have created documentation that makes it very easy to deal with case insensitivity either completely in the grammar or by using all uppercase in the grammar and in conjunction with a special character stream.

[bramp]

Thanks for writing this up. Is the intention to exclude the CaseInsensitiveStreams from the core library? and require users to make a local copy?

[parrt]

Yep

…

Sent from my iPhone

On Dec 5, 2017, at 4:53 PM, Andrew Brampton ***@***.***> wrote: Thanks for writing this up. Is the intention to exclude the CaseInsensitiveStreams from the core library? and require users to make a local copy? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

[davesisson]

lgtm

[KvanTTT]

Why not include these streams to runtimes itself? It's more convenient and less error-prone. Moreover the size of this code too small, thus nothing worry about. I suggest moving these files to runtimes.

[ericvergnaud]

Duplicating my comment on another thread:
Wouldn’t it be more accurate and still simple to introduce a ‘caseless’ section in the Lexer grammar ?
Something like:
caseless {
SELECT: ‘select’;
WHERE: ‘where’;
}
The tool would generate the required intervals for each token letter.
I appreciate this requires changing the ANTLR grammar itself but it would remove any ambiguity on what is or isn’t caseless and would immediately work for all targets.
Only ascii would be supported.
What do you think?

[parrt]

@KvanTTT Every bit of code that goes into the main repository requires maintenance, documentation, synchronicity between language targets, testing and so on. It is always a trade-off between how many of the users need something and the maintenance effort on me. Remember that I've been doing this for 30 years now and everything we add I'm stuck supporting ;)

[parrt]

hi @ericvergnaud! An argument against that would be that changing the ANTLR metalanguage would break any existing tools. In the end, I think this is a minor issue for a small subset of the grammars. As @mike-lischke points out, you cut-and-paste once, if you are unlucky enough to work on an ancient language with case insensitive keywords, and never think about it again. :)

[parrt]

I introduced a syntax error are in the JavaScript so I'm waiting to get the tests to pass then will integrate... I will do a quick check for PRs but then will move towards a 4.7.1 release :)