Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generate tree-sitter parsers if they don't exist in git repo #1859

Closed
slinlee opened this issue Mar 22, 2022 · 2 comments
Closed

Generate tree-sitter parsers if they don't exist in git repo #1859

slinlee opened this issue Mar 22, 2022 · 2 comments
Labels
A-tree-sitter Area: Tree-sitter C-enhancement Category: Improvements

Comments

@slinlee
Copy link
Contributor

slinlee commented Mar 22, 2022

The problem is that the grammar .gitignores the src/ directory (see here). Helix's build process for grammars currently requires the generated src/ directory which is all the C/C++ files you need to compile the parser.

Originally posted by @the-mikedavis in #1853 (comment)

@the-mikedavis the-mikedavis added C-enhancement Category: Improvements A-tree-sitter Area: Tree-sitter labels Mar 22, 2022
@the-mikedavis
Copy link
Member

the-mikedavis commented Mar 22, 2022

tree-sitter/tree-sitter#1641 has some info on how nvim-treesitter deals with grammars. They have a system simliar to ours where the repositories are cloned and in some cases, the src files get generated by calling out to tree-sitter and sometimes even npm. I've been thinking about how to solve some of those issues and I think a central registry which packages the generated src directories is a reasonable increment, so that's what I've been tinkering with in helix-editor/tspm. The idea there is that you can package the grammars into a minimal and reproducible tarball with a uniform structure, so all of the packaging concerns like getting dependencies (submodules, npm, etc.), generating the src files and specifying the ABI live in TSPM instead of editors / tree-sitter consumers. Then consumers download and extract the tarball and compile the C files.

That doesn't solve all of the issues there (in particular around safety), but I think it's a good middle-ground between the current state and the ideal eventual state (see the "Native Library, WASM parsers" part of tree-sitter/tree-sitter#930).

@the-mikedavis
Copy link
Member

We could do this either by embedding tree-sitter-cli or shelling out to it. I think both options are less favorable to me than #1970, though, so closing out in favor of that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-tree-sitter Area: Tree-sitter C-enhancement Category: Improvements
Projects
None yet
Development

No branches or pull requests

2 participants