Docs & Highlighters Status Report (Aug. 2021)

[tajmone]

@thoni56,

I've now updated all the style preview documents in _dev/styles-tests/ so that for the HTML backend a copy of each document is created using each supported syntax highlighter.

For each source file <fname>.asciidoc the build scripts will produce:

• HTML Format:
  • <fname>_rouge.html — using Rouge for code highlighting.
  • <fname>_highlight.html — using Highlight for code highlighting.
  • <fname>_hljs.html — using highlight.js for code highlighting.
• PDF Format:
  • <fname>.pdf — via asciidoctor-fopub.

This allowed me to discover that in the CSS for Highlight I had forgotten to include the styles for BNF blocks.

I haven't really yet had the time to carefully go through all nine HTML docs (3 different docs * 3 highlighters) and check how each style is covered in each, and which features are supported or not by each highlighter — but I'll be doing it in the next week, for I have to create a table with all the pros and cons of using each highlighter.

But I thought you might want to known about this update, so you can check how ALAN code is being rendered with Rouge vs previous highlighters (I can always fix anything out place in the CSS).

Roughly, the situation now is this:

• ALAN Block Comments — only supported by Rouge (PDF doesn't support them).
• Code highlighter markers & span — not supported by Rouge (the code block will simply be not highlighted).
• Callouts in Highlighted Code — unsupported by Highlight (they will be shown as raw comment, e.g. -- <1>, but outside the code block they references are correctly rendered).

As you can see, currently none of the highlighters supports all features, each one has its pros and cons, although none of them will break a document, just not rendered it fully as expected (but still readable).

Lack of support for ALAN block comments is something I'm trying to implement in Highlight, but struggling a bit with it (should eventually find a solution). I could more easily add them to highlight.js, but we're not using this highlighter in any documents right now.

I'm not sure if I can implement them for the PDF Fopub backend, because I don't remember how the highlighter worked (it might be trick if it's over-simplistic though, which I think it's the case).

Anyhow ... I just thought that after all the recent major changes in the repository it was a good idea to refresh ourselves about the documents status, their highlighters and their DOs and DON'Ts — but I'll be adding a reference doc in the repo soon, with all the details, once I've finished checking the styles tests docs and annotate everything down.

[thoni56]

I didn't find any instructions, or pointers, on how to install Rouge. Or did I just not look hard enough?

[tajmone]

No instruction yet, just linked the word Rouge to its website or repo. I'm currently reorganizing all the README docs, and didn't get round to that yet.

It's just a gem, so:

gem install rouge

[tajmone]

Updated Dependencies in README

Ok, I've update the Project Dependencies section, adding Rouge and all other highlighters, as well as Asciidoctor Diagram.

Also update the entry post, which was missing the links for the mentioned highlighters.

[tajmone]

Highlight Now Supports Block Comments

@thoni56, I've finally managed to try out the hack I had in mind to fix my previous attempts to update the Highlight syntax for ALAN to cover block comments.

Now it works correctly, as you can see from the tests in the ALAN Testbed:

• https://github.com/alan-if/Alan-Testbed/tree/master/syntax-hl/Highlight
• HTML LivePreview of sample.html

Now I only need to add the new syntax to the ALAN Docs repo, update the Sass themes to cover block comments. Before we were using the native ALAN syntax that ships with HL, but now I'll be adding our custom syntax, so that we can start using it before it's included in the next HL release package (of course, I'll submit it to HL too).

Now there's only highlight.js to update, so it can support block comments too. But since no docs are using it right now, and because the new system of independent syntaxes is a bit tedious to use, I'll procrastinate this for time being.

[tajmone]

New Status Doc

I've just added a new STATUS.md document that should simplify tracking all these problems with the different syntax highlighters, which features they do and do not support, etc.

I'll soon update the doc to also provide status info about all the documents in this repository, mentioning whether they are published on the website or not, which highlighter they require for the HTML backend, and which output formats they are available in and/or support.

This docs, with its lookup tables, should finally allow us to easy check all these various pieces on info in a single place, and then follow the links to the various Issues and Discussion to track the status of WIP features and pending tasks.

It was just becoming too tiresome having to track all these small things by digging into Issues and Discussion across all the different ALAN repositories. It also help me remember what's still pending implementation in this respect.