annual upstream updates

[slimsag]

We haven't updated to account for upstream changes in a while, and a lot has changed!

Key highlights

• 228 commits / 1 year of improvements to the syntax highlighter library syntect
• 432 commits / 1 year of improvements to the base syntax definitions for ~36 languages sublimehq/Packages
• 30 new file extensions/names now properly detected
• Added support for: ASP vb.NET, ASP.net HTML, COBOL, jcl, CUDA C++, Solidity/Vyper (Ethereum), Smarty
• Likely fixes OCaml support and other major issues.

Updates

• Updates the upstream version of trishume/syntect (the syntax highlighter) which includes 228 commits/improvements over the past year
• Updates the upstream version of sublimehq/Packages (the base syntax definitions we use) which includes 432 commits/improvements to 36 languages
• Updates us to using the latest version of Rust nightly (~1 year of updates).
• Removes a fair amount of code that is no longer needed thanks to upstream changes in Syntect.
• Updates the sourcegraph/alpine base image from 3.9 -> 3.10

Effects

Now detects the following additional file types:

• Clojure: edn
• Erlang: escript
• Git Mailmap: .mailmap, mailmap
• Groovy: Jenkinsfile
• JavaScript: htc
• Perl: pc
• Python: vpy, wscript, bazel, bzl (TODO: Bazel inclusion is bug??)
• Starlark: build_defs, BUILD, WORKSPACE, sky, star, BUILD.bazel, WORKSPACE, WORKSPACE.bazel
• Ruby: Podfile
• Scala: sc
• Bash: .zlogin, .zlogout, .zprofile, .zshenv, .zshrc, PKGBUILD, .ebuild, .eclass
• XML: rng

There is now also support for the following languages:

• ASP vb.NET (vb)
• HTML (ASP.net) (aspx, ascx, master)
• COBOL (cbl, cpy, cob, dds, ss, wks, pco)
• jcl (jcl)
• CUDA C++ (cu, cuh)
• Solidity (sol) (ethereum)
• Vyper (vy) (ethereum)
• Smarty (tpl)

The following file types are no longer detected (I believe because it is no longer considered best out-of-the-box in Sublime, so likewise for us I presume):

• HTML: tmpl, tpl
• JSON: ipynb, Pipfile.lock
• Makefile: makefile.am, Makefile.am, makefile.in, Makefile.in
• Perl: PL (uppercase)
• R: s, S
• Bash: fish (is detected properly by fish-specific syntax)

Cool upstream changes

• Classed HTML support was added upstream which we could potentially start benefitting from in the future: Added classed HTML generator struct trishume/syntect#235 https://github.com/trishume/syntect/pull/274/files
• Fancy-regexp support was added, which means Syntect now isn't dependent entirely on onigiruma which is incredible: https://github.com/trishume/syntect/blob/master/CHANGELOG.md#version-400-2020-03-29
  • I didn't enable it for us here, because of the performance impact, but we could try it out in the future and I'm sure over time it'll only get better.
• We can start using not-a-fork of Syntect: Added classed HTML generator struct trishume/syntect#235 (comment)

Notable changes on our end

• Now treat .build_defs, .star, and BUILD files as Bazel/Starlark files (https://github.com/sourcegraph/sourcegraph/issues/8123)
• Added support for Solidarity / Etherium (https://github.com/sourcegraph/sourcegraph/issues/2440)
• Added support for Cobol (https://github.com/sourcegraph/sourcegraph/issues/10154)
• Added support for Cuda (https://github.com/sourcegraph/sourcegraph/issues/5907)
• Added support for Visual Basic / VB.net (https://github.com/sourcegraph/sourcegraph/issues/4901)
• Added support for ASP.NET (https://github.com/sourcegraph/sourcegraph/issues/4262)
• Added support for Smarty files (https://github.com/sourcegraph/sourcegraph/issues/2885)
• Changes upstream likely resolve an issue we saw somewhat frequently on certain files: https://github.com/sourcegraph/sourcegraph/issues/9557 (need to confirm)
• Changes upstream likely fix OCaml support issues (https://github.com/sourcegraph/sourcegraph/issues/3545)

Massive thanks

syntect_server just exposes the hard work others have done to make world-class syntax highlighting possible, so my deepest and most significant gratitude goes out to:

• To the author and contributors of Syntect
• To the authors of Sublime Text 3, and the numerous Sublime Text 3 syntax packages which provide syntax highlighting.

[slimsag]

Published:

sourcegraph/syntect_server:ff37f90@sha256:aa93514b7bc3aaf7a4e9c92e5ff52ee5052db6fb101255a69f054e5b8cdb46ff