Tree-sitter match limit is insufficient sometimes

[woojiq]

Summary

In short: the text is colored not immediately, but if you scroll down a little.

Self describing video:

P.S. Give this issue a better name, I don't know what to call it properly 😂

Reproduction Steps

I tried this:

1. Clone the serde_json repo https://github.com/serde-rs/json
2. Open the src/lib.rs file
3. Go to the line +-310 and scroll down, you will see unhighlighted text, scroll down a bit and the text will be colored.

I expected this to happen:
Average beautiful highlighting:)

Instead, this happened:
Part of the text does not color out immediately.

Helix log

~/.cache/helix/helix.log

please provide a copy of `~/.cache/helix/helix.log` here if possible, you may need to redact some of the lines

nothing related even with `-vvv`

Platform

Linux/Nixos-unstable

Terminal Emulator

wezterm 20230326-111934-3666303c

Helix Version

helix 23.03

[the-mikedavis]

I think this is caused by the tree-sitter match limit (#4707), I saw similar behavior with Erlang when matching a complicated pattern against a large subtree (#4830).

[woojiq]

I've built the helix myself with the next change:

helix/helix-core/src/syntax.rs

Line 385 in 406c5c3

const TREE_SITTER_MATCH_LIMIT: u32 = 256;

-const TREE_SITTER_MATCH_LIMIT: u32 = 256; 
+const TREE_SITTER_MATCH_LIMIT: u32 = 512;

And now it works fine. Right now I don't know how to check the penalty speed properly, is it the start time of the helix app or what? In any case, I don't think just changing this variable is a good idea. In my opinion, it would be better to set these settings for different languages or give users access to this option, so it would be possible to change temporarily for one file because not everyone needs that much (personally, it doesn't spoil the quality of my life 🙂). Though I'm not sure treesitter can change its settings at runtime.