Stop CRLF carriage-return leakage from unterminated single-line literals

[Copilot]

On CRLF input, unterminated single-line string/char literals were terminated only on \n, allowing \r to be captured inside StringLiteral tokens. That leaked a literal CR into HighlightedToken.Text, unlike sibling scanners that already stop on both line terminators.

• Tokenizer termination parity (CR + LF)

  • Updated CodeTokenizer.ScanStringBody to end non-multiline literals on either \r or \n, matching line-comment and directive scanning behavior.

• Regression coverage for real edit-time states

  • Added CRLF + unterminated literal cases in LineSplitterTests for:
    • C# unterminated string literal
    • C# unterminated char literal
    • Python unterminated string literal
  • Assertions verify first-line output contains no \r.

if (!rule.AllowMultiline && (current == '\n' || current == '\r'))
{
	return scan;
}

• Fixes CRLF normalization leaks a literal CR out of an unterminated single-line string literal #389

[sonarqubecloud]

Quality Gate passed

Issues
3 New issues
0 Accepted issues

Measures
0 Security Hotspots
100.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud