Brand new TextMate grammar for providing C# syntax highlighting

[DustinCampbell]

Fixes #101, #225, #268, #316, #674, #706, #731, #746, #782, #802, #816, #829, #830, #861, #1078, #1084, #1086, #1091, #1096, #1097, #1106, #1108

This is a large change that addresses several syntax highlighting bugs and provides a brand new TextMate grammar for C#. While this grammar will initially be checked in here, it will ultimately be migrated to https://github.com/dotnet/csharp-tmLanguage where it will live as the official C# TextMate grammar from the C# team.

The new grammar is much more robust than the one previously used and is modeled after the grammar used for TypeScript: https://github.com/microsoft/TypeScript-tmLanguage.

To review the new grammar, please see syntaxes/csharp.tmLanguage.yml.

Major changes:

• The grammar is now authored as a YAML file which is more compact, supports comments, and allows us to split complex regular expressions across multiple lines for clarity.

• Type names are no longer matched as a single blob of text. Instead, they are matched recursively and each identifier and piece of punctuation is classified separately:

  

• Generally, each production is each matched separately (though care is taken to ensure that matches begin with non-terminal). This is a big improvement over the previous grammar because future changes are more localized and matches can be more contextual).

• ~400 unit tests to protect against future regressions.

cc @CyrusNajmabadi, @jasonmalinowski, @dpoeschl, @rchande, @mattwar, @MadsTorgersen, @jaredpar, @damieng, @alexandrudima, @jrieken, @david-driscoll, @filipw

Here's an example of how things look with the new grammar:

[gregg-miskelly]

You don't want this change

[DustinCampbell]

sigh I always forget those. Fixing.

[DustinCampbell]

Thanks for spotting those changes! They're reverted now.

[gregg-miskelly]

Or this one

[gregg-miskelly]

Or this

[damieng]

Wish I'd known this was coming - spent a lot of time fixing up the existing grammar atom/language-csharp#87

Any plans to move this to the grammar-only repo at https://github.com/dotnet/csharp-tmLanguage or is it going to reside here after all?

[DustinCampbell]

@damieng: Sorry you were working so hard on those changes! I worked on this mostly over holiday vacation time, so I apologize for not checking in with you. I suppose it's a little funny since we were likely working on our changes within a half-hour drive of one another. 😄

Yes, the new grammar will be moving to the CSharp-tmLanguage repo. For now, it's here simply for convenience and for kicking the tires while we get that repo up and running. Once it's there, this will become the official C# grammar and we can all contribute there.

[filipw]

excellent!

Fixes #101, #225, #268, #316, #674, #706, #731, #746, #782, #802, #816, #829, #830, #861, #1078, #1084, #1086, #1091, #1096, #1097, #1106, #1108

you might have broken a Github record 😄

[DustinCampbell]

@ivanz, @Pilchie: Could I get a sign off from either of you? I realize that providing a full review is probably unrealistic, but I'd love to get this merged so we can start kicking the tires for 1.7 of C# for VS Code.

[tverboon]

Would love to use a preview release and give feedback if I encounter any problems or see something strange. Working full time in VS Code 🎉 ❤️

[Pilchie]

I won't be able to take a look until Monday. Sorry :-/

[ivanz]

Amazing! As you say - it's massive to review. Conceptionally looks good and I am happy to approve, so it can be tested in practice :)

[DustinCampbell]

Thanks! Merging.

[jlmakes]

Epic!!

Is there a rough estimation of when these improvements will make it into a release?

[DustinCampbell]

I'm hoping to get a new beta release out here on the site that adds this and some other fixes for an upcoming .NET Core release. As far as the official release, the 1.7 milestone is currently slated for 1/31. We'll see if we make that date or not. 😄

[jasonmeisel]

Awesome! Great job!