Shell arithmetic syntax highlight breaks on multiple cases

[leagris]

Prerequisites

• Put an X between the brackets on this line if you have done all of the following:
  • Reproduced the problem in Safe Mode: https://flight-manual.atom.io/hacking-atom/sections/debugging/#using-safe-mode
  • Followed all applicable steps in the debugging guide: https://flight-manual.atom.io/hacking-atom/sections/debugging/
  • Checked the FAQs on the message board for common solutions: https://discuss.atom.io/c/faq
  • Checked that your issue isn't already filed: https://github.com/issues?utf8=✓&q=is%3Aissue+user%3Aatom
  • Checked that there is not already an Atom package that provides the described functionality: https://atom.io/packages

Description

Shell arithmetic syntax highlighting breaks on multiple cases:

1. Base marker base# is treated as a comment marker
2. When base marker is turned into a comment, the following lines, variables are not highlighted.
3. Parenthesis in arithmetic expression breaks the (( )) delimiters detection.

Steps to Reproduce

1. Edit the following Bash script:

#!/usr/bien/env bash
echo "$((16#ff))"   # base# handled as comment marker
echo "$((a=16#ff))" # variable highlighted ok
a=42                # variable highlighted ok
((a=42))            # variable not highlighted
a=42                # variable highlighted ok
((a=16#ff))         # variable not highlighted
a=42                # breaks further highlighting
a="$((~(-1 << 8)))" # parenthesis highlight broken

Expected behavior:

Same correct highlighting as sample code above as shown in github

Actual behavior:

Reproduces how often:

100% reproducible

Versions

Versions

• OS : Ubuntu 19.04
• Atom : 1.38.2
• Electron: 2.0.18
• Chrome : 61.0.3163.100
• Node : 8.9.3
• apm 2.1.7
• npm 6.2.0
• node 8.9.3 x64
• atom 1.38.2
• python 2.7.16
• git 2.20.1

Additional Information

none

[rsese]

Thanks again @leagris 👍 I can reproduce with on macOS 10.12.6 but not sure if this belongs in https://github.com/tree-sitter/tree-sitter-bash.

For example, I see that the issue with base# has already been reported: tree-sitter/tree-sitter-bash#41.

I think that leaves 2 issues from your report, I don't think these have been reported?

((a=42))            # variable not highlighted

and:

a="$((~(-1 << 8)))" # parenthesis highlight broken

I'll see which repo the other maintainers think the issue belongs.

[TriMoon]

I was about to file a bug report for exactly this issue...
Please fix asap 😉

if test 0 -ne $((bDebug & 2#0001)); then
        ....
fi

Atom's highlighting/parsing bugs after the hash sign of 2#0001
A simple fix IMHO is to check for digits right infront of the hash sign....

[leagris]

@TriMoon
or:

if ((bDebug & 2#0001)); then
  echo 'ok'
fi

or:

((bDebug & 2#0001)) && echo 'ok'

or: Oh! Well, it does not show the bug then!

((bDebug & 1)) && echo 'ok'

[TriMoon]

@leagris is that first example you give using if working on all shells? (eg. bash,zsh,dash,sh)
It needs to be shell agnostic...
Besides re-coding just to avoid highlighting bugs in some editor is not the way to go 😉

[leagris]

@TriMoon
Stand-alone arithmetic expressions like in 1 are undefined in POSIX but supported everywhere.
Base specification with <base>#<number> in arithmetic expressions is undefined in POSIX as well, so I opted to use POSIX undefined stand-alone expressions as well :)

[leagris]

A simple fix IMHO is to check for digits right infront of the hash sign....

The base value can be a variable name, so checking for [:digit:]# would break this way.

The correct fix is to turn-off comments parsing within an arithmetic expression, as there is no comment allowed within it.