inject language based on file extension & shebang

[nrdxp]

Nodes can now be captured with "injection.filename". If this capture contains a valid file extension known to Helix, then the content will be highlighted as that language. In addition, a new "injection.shebang" is added which injects based on a valid shebang.

For consistency, both implementations use the same logic as used to detect the document filetype as a whole.

I'm not sure if the following is the most efficient way to capture the built in configuration, or if it can be passed in from somewhere else:
https://github.com/nrdxp/helix/blob/7f61de137f07cf5897d5a3240b1ece98febc0eb0/helix-core/src/syntax.rs#L1836

Also, it would probably be nice to extend this to user defined languages as well. 65bc9da

Also includes an additional query for Nix so one can test this behavior themselves. A few screens with that query:

This PR also includes a new document briefly describing language injeciton.

[the-mikedavis]

I think I would prefer injection.filename for this since the language is computed by the extension but the captured value is the full filename

[nrdxp]

I think I would prefer injection.filename for this since the language is computed by the extension but the captured value is the full filename

good point, I'll push a fix

[nrdxp]

Updated to "injection.filename" and also addressed the clippy lints

[nrdxp]

There appears to be one little bug I would like to get to the bottom of before merging. Mainly, if the capture happens more than once in a file, then the last value of injection.filename overrides all the other string contents highlighting. Not sure why this isn't an issue with injection.language since I didn't modify any of the highlighting logic.

To illustrate:

{
  f = writeText "foo.sh" ''
    echo $bar
  '';
  g = writeText "foo.py" ''
    def foo():
        print("hello")
  '';
}

Will highlight both strings as python, however, if I change "foo.py" to just "foo" then both are highlighted as haskell.

So essentially, the value is being applied to every string captured globally. This appears to have something to do with "injection.combined" since if I turn it off, this doesn't occur, but the downside is that highlighting after interpolation isn't correct either.

edit
After review the original issue thread for combined injections it would appear this is intentional. I'm still a bit confused because this does not happen with my comment style injection queries which also use injection.combined. So I wonder if I am perhaps misusing injections.combined and instead my query should somehow be improved to be more specific perhaps?

Also, if we really wanted to, we could make the implementation even simpler by just allowing "injection.language" to fallback to filetype detection if language name detection fails with this simple diff:

  diff --git a/helix-core/src/syntax.rs b/helix-core/src/syntax.rs
  index e0a984d2..f7fb8e7a 100644
  --- a/helix-core/src/syntax.rs
  +++ b/helix-core/src/syntax.rs
  @@ -539,6 +539,12 @@ pub fn language_configuration_for_injection_string(
               let configuration = &self.language_configs[i];
               return Some(configuration.clone());
           }
  +
  +        let path = Path::new(string);
  +
  +        if let Some(config) = self.language_config_for_file_name(path) {
  +            return Some(config);
  +        }
           None
       }

And that would be all that is required.

[the-mikedavis]

injection.combined is for combining all nodes captured with injection.content together. There isn't currently any scoping for it - it applies for the whole document. I don't think the comment highlighting or this should be using it now that I think about it. Does something break if we remove it?

I think injection.language should be separate from this

[the-mikedavis]

the downside is that highlighting after interpolation isn't correct either

Ah sorry, reading is hard 😅

To cover this you most likely want to use injection.include-children instead of injection.combined

[nrdxp]

@the-mikedavis, thanks for the feedback, I'll try to fix the queries and then it should be ready. Also, fyi I changed the implementation just a little bit. I think it's definitely better. It should also be more efficient since we don't have to traverse the languages once to get the name and then again to get the config.

Instead I introduced a new enum InjectionCapture, to represent each @injection capture. This should make it easier to extend again in the future if we wanted to.

edit
"include-children" doesn't seem to function the way I need. Say for example in this nix string containing bash:

''
  echo "${bar}"
  echo baz
''

The second echo will not highlight correctly, because the result is not combined, even with include-children meaning the bash highlighter mistakenly thinks the string after the interpolation is the opening of a shell quote instead of the closing. In other words it thinks the quotes are unbalanced when they are not.

Perhaps we need a new capture, something like "injection.combine-siblings", that does the samething as "injection.combined" but not globally, only within the scope of the current query. Or maybe "injection.combined" could just be modified to do the same, if it doesn't break anybodies use-case. I'm not sure where a global capture like that would be useful where a scoped one would not, but maybe there is.

Either way though, it might be best to implement in a seperate PR. I think it's rare enough to define two scripts of a diffferent language in one file that merging this wouldn't be horrible in its current state.

[nrdxp]

My last refactor made it fairly trivial to add another commit to highlight based on a captured shebang. In addition, I wrote up a brief doc explaining language injection queries.

[nrdxp]

@the-mikedavis, I address your comments and cleaned up the commit history. Also, the PRs I submitted to upstream tree-sitter-nix were just merged today, so I added a small commit to bump it and included some small query fixes.

[nrdxp]

Thanks for the thorough review! I addressed all of your comments, and hopefully it is ready now 🤞

[pascalkuthe]

LGTM now

[David-Else]

There are few spelling and typing errors I noticed.

[nrdxp]

Thank you @David-Else. Looks like we really need a spellchecker for Helix as well 😅

[David-Else]

Thank you @David-Else. Looks like we really need a spellchecker for Helix as well sweat_smile

You can use https://github.com/helix-editor/helix/wiki/How-to-install-the-default-language-servers#ltex-ls , for Markdown, it is excellent!