Raw string highlighting

[jcrmatos]

Environment data

VS Code version: 1.18.1
Python Extension version: 0.80.0
Python Version: CPython 3.5.4 32b
OS and version: Windows 7 Pro SP1 x64

Actual behavior

I have the following line

fops.save_txt(environ['SystemRoot'] + r'\explorer.exe '
                      + bdexp.EXPORTS_DIR,
                      bpl.PROGS_TO_LAUNCH_PN)

and the r and \e shows in blue with the first ' and xplorer.exe ' shows in brown(?).
My point is that it isn't recognizing it as a string (raw in this case) like it does with the previous 'SystemRoot' which shows all brown(?) and the Intellisense recognizes as a string.

Expected behavior

It should show r'\explorer.exe ' all brown(?) and recognize it as a string.

I reported this first in VS Code Python at microsoft/vscode-python#337 (comment) but Brett Cannon told me to open it here because their syntax highlighting depends on MagicPython.

Thanks,

JM

[elprans]

This is a feature.

foo = r'\nregexp'  # this is parsed as a "regexp-string"
bar = R'\nregexp'  # this is parsed as a raw string

[jcrmatos]

Hello,

I'm sorry but I can't find that difference in the CPython spec.
In fact what I find in both Py2 and Py3 official docs is:

Both string and bytes literals may optionally be prefixed with a letter 'r' or 'R'; such strings are called raw strings and treat backslashes as literal characters.

...

Unless an 'r' or 'R' prefix is present, escape sequences in string and bytes literals are interpreted according to rules similar to those used by Standard C.

Please reconsider changing MagicPython behavior to be in sync with CPython.

Thanks,

JM

[jcrmatos]

Hello,

I forgot to include the CPython official docs links:
https://docs.python.org/3/reference/lexical_analysis.html
and
https://docs.python.org/2/reference/lexical_analysis.html

Thanks,

JM

[1st1]

Please reconsider changing MagicPython behavior to be in sync with CPython.

You're correct—there's no "regex" literal in Python, and r"string" is the same thing as R"string". However, TextMate originally, and Sublime Text later both highlighted r"strings" as regular expressions for years, and a lot of users expect MagicPython to do the same, as it's positioned as a "drop in replacement" for Python syntax highlighters in those editors. Therefore, unfortunately, we are not going to reconsider having this feature in MagicPython.

[jcrmatos]

Hello,

I understand.
Can I ask you please to add a Settings option to disable that behavior?

Thanks,

JM

[1st1]

Generally syntax highlighters don't have settings panels and aren't configurable. We had an idea how to do that for MP, but don't have time to pursue it. So unfortunately there's no ETA for this feature.

[memeplex]

There is a problem with the approach here that was not mentioned: it doesn't play well with the super popular black code formatter, which will turn R into r again.

[elprans]

Black can be told not to do that via --skip-string-normalization

[nerdfever]

This issue is now 5 years old and I came here to report it as well.

MagicPython is used in VSCode; the vast majority of VSCode users aren't (any longer at least) coming from TextMate or Sublime, and this looks broken to them.

Maybe it's time to reconsider. (Or make it configurable, somehow.)

[DiscipleOfEris]

I'm also coming here from VSCode to report this problem.

I will also mention that literally my very first experience with MagicPython (through VSCode) was this non-CPython compliant variation on raw strings. I had to stop coding for a minute to go check the docs to double-check that yes, it was just the syntax highlighter gaslighting me and not me misremembering raw strings.

In my experience, raw strings are extremely common for file system paths. I've obviously also used raw strings for regex, but I'd prefer syntax highlighting on regex to be visibly opt-in than for the syntax highlighter to incorrectly assume that raw strings are always regex. That said, if the price we pay for getting regex syntax highlighting at all is having to remember that raw strings for non-regex will be incorrectly highlighted, then I suppose there's an argument to be made that it would be worth it. However, I'm sure a comfortable middle ground exists between those two extremes.

[BBSeward]

Another user coming here from VSCode and agreeing with the past two comments. Would be great to consider this, or make it an option.

[nerdfever]

I do wish this would be re-opened.

For those coming here with the issue - note that using capital R for raw instead of r (R"string" vs. r"string) fixes this problem.

It's a workaround as r vs R has no meaning to Python, but if you use that convention you won't run into this problem.

[strubbly]

Sorry, just another VS Code user with the same issue. We have a company policy to use black with default values everywhere. That effectively means I can't avoid this incorrect highlighting on raw strings. So, many of my docstrings are highlighted incorrectly - and the same applies to most of my co-workers. I don't know what would be the best fix but I do think something needs to be done.

[mitchg17]

+1 for VS Code confusion

[pepa-husek]

the same issue - should definitely be at least configurable if not even turned off by default. yet again conservatism and backward compatibility are hindering normal progress.

[Jeitan]

Ditto here. This is still an issue. The rationale that "most users will be confused by making it correct" is no longer valid - rather, the opposite is now happening. Most users are confused by it not recognizing 'r' as a raw string.

[hutcho]

Came here after searching why the bug exists in vscode showing \ as escapes on r"" strings. Please add a config option to configure this behaviour. The R"" v r"" differentiation is not a feature, it's a strange design, and needs to be configurable.