QSyntaxHighlighter and Pygments do not properly interoperate, leading to occasional mis-highlighting

[spyder-bot]

From [email protected] on 2015-01-18T11:36:30Z

As noted in my recent pull request [1], certain filetypes will not highlight well when using Pygments. This is because the base class, QSyntaxHighlighter [2] breaks text into blocks (individual lines) and tries to highlight them one at a time [3], whereas Pygments is set up to highlight whole files at once; when Pygments is asked to highlight individual lines, it can't always give the right results (see, for example, multi-line comments in a css file like this one [4]).

This disconnect will be tough to work around. IPython has a sophisticated attempt [5], but I don't know how well it can be adapted to Spyder. I'll investigate and hopefully have some sort of fix I can offer, but I'm not sure I can solve it.

[1] bitbucket.org/spyder-ide/spyderlib/pull-request/97/
[2] http://qt-project.org/doc/qt-4.8/qsyntaxhighlighter.html [3] https://bitbucket.org/spyder-ide/spyderlib/src/b5a5ef2bb4d59a437a111fff7cb03faf09e86d3f/spyderlib/widgets/sourcecode/syntaxhighlighters.py?at=default#cl-856 [4] https://raw.githubusercontent.com/daneden/animate.css/master/animate.css [5] https://github.com/ipython/ipython/blob/master/IPython/qt/console/pygments_highlighter.py

Original issue: http://code.google.com/p/spyderlib/issues/detail?id=2122

[spyder-bot]

From contrebasse on 2015-01-18T16:02:07Z

I don't understands the details, but it seems to me that QSyntaxHighlighter has a way to manage blocks bigger than one line since it highlights multiline strings just fine ?

[spyder-bot]

From [email protected] on 2015-01-18T18:30:21Z

Sorry, I forgot to mention that. QSyntaxHighlighter has a method for setting state from one block to the next. So if a block starts a multi-line comment, the next block knows that it is currently inside a multi-line comment and can be highlighted accordingly. But there's no (easy) way to connect that state mechanism with Pygments' highlighting. If Pygments sees a /* but no corresponding */, it will just think it's an error; it won't indicate that a comment has started, and has no way to tell later blocks that they are still inside a comment.

[spyder-bot]

From contrebasse on 2015-01-19T02:29:40Z

I understand now, thanks.

[spyder-bot]

From ccordoba12 on 2015-01-30T13:56:48Z

Status: Accepted
Labels: Cat-Editor MS-v2.4

[randyheydon]

Also see my unfinished pull request in relation to this issue.