Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixes line encoding for strict clients (neovim) #24

Merged
merged 3 commits into from
Jul 22, 2024

Conversation

chiefnoah
Copy link
Contributor

The line ending is always \r\n as far as neovim is concerned. Futher, these values are effectively constants, no reason to wrap them in function calls.

Without this change, neovim will ignore responses and behave very strangely.

I also intend to submit a PR to nvim-lspconfig to do some basic setup.

@chiefnoah
Copy link
Contributor Author

Ope, I think I forget to fix the tests

@chiefnoah
Copy link
Contributor Author

There's a few rather odd changes that come from my use of nvim-paredit with strict paren matching enabled. It's mostly pedantic cleanup of tabs and removing some odd vestigial bits of code. If undesired, simply remove those changes.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This breaks the tests in this file.

project.janet Outdated
@@ -5,7 +5,8 @@
:dependencies ["https://github.com/janet-lang/spork.git"
"https://github.com/CFiggers/jayson.git"
"https://github.com/ianthehenry/judge.git"
"https://github.com/CFiggers/cmd.git"])
"https://github.com/CFiggers/cmd.git"
"https://github.com/CFiggers/jgraph.git"])
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

jgraph isn't really a dependency of this project, it's just referenced in some sample code used to test the formatter. I wouldn't include it here (if having it as part of the formatting example is causing problems, I'd rather get a new formatting sample).

Copy link

@sogaiu sogaiu Jul 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FWIW, I had noticed that without jgraph installed the tests (at least those invokable via jpm test) don't run without errors:

$ jpm test
running test/test-format-file-after.janet ...
error: could not find module jgraph:
    build/jgraph.so
    /home/user/.local/lib/janet/jgraph.jimage
    /home/user/.local/lib/janet/jgraph.janet
    /home/user/.local/lib/janet/jgraph/init.janet
    /home/user/.local/lib/janet/jgraph.so
  in require-1 [boot.janet] on line 3028, column 20
  in import* [boot.janet] on line 3072, column 15
  in thunk [test/test-format-file-after.janet] (tail call) on line 3, column 1
non-zero exit code in test/test-format-file-after.janet: 1
running test/test-format-file-before.janet ...
error: could not find module jgraph:
    build/jgraph.so
    /home/user/.local/lib/janet/jgraph.jimage
    /home/user/.local/lib/janet/jgraph.janet
    /home/user/.local/lib/janet/jgraph/init.janet
    /home/user/.local/lib/janet/jgraph.so
  in require-1 [boot.janet] on line 3028, column 20
  in import* [boot.janet] on line 3072, column 15
  in thunk [test/test-format-file-before.janet] (tail call) on line 3, column 1
non-zero exit code in test/test-format-file-before.janet: 1
running test/test-integration.janet ...
running test/test-lookup.janet ...
running test/test-main.janet ...
Starting LSP

May be that's what you meant by:

it's just referenced in some sample code used to test the formatter.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I don't use jpm test in this project—I'm using ianthehenry/judge for everything (admittedly not all of those are passing at the moment either... 😅)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The tests were running with jpm test when I ran them (with jgraph installed). I didn't realize judge had its own executable.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tests set up with Judge ((test) and (deftest) macros) don't mean anything when run with jpm testjpm test essentially just evaluates .janet files, it doesn't do any special test running. The (test) and (deftest) macros in Judge evaluate to no-ops when you're not using the judge binscript. So even failing Judge tests will not complain when jpm test is run.

Copy link

@sogaiu sogaiu Jul 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FWIW, I disabled jpm test in a project like this.

One could customize output to indicate the use of other options I suppose :)

Copy link
Contributor Author

@chiefnoah chiefnoah Jul 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we use jpm to invoke judge instead? I'm usually a pretty big fan of consistent interfaces :)

Copy link

@sogaiu sogaiu Jul 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

judge's documentation mentions a specific way it can be used via project.janet:

You can also add this to your project.janet file:

(task "test" [] (shell "jpm_tree/bin/judge"))

To run Judge with a normal jpm test invocation.

May be that or something similar is suitable for janet-lsp's case?

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

judge's documentation mentions a specific way it can be used via project.janet:

You can also add this to your project.janet file:

(task "test" [] (shell "jpm_tree/bin/judge"))

To run Judge with a normal jpm test invocation.

May be that or something similar is suitable for janet-lsp's case?

I'm trying to get this working and right now I'm only getting obscure build errors whenever this task is included in project.janet.

@CFiggers
Copy link
Owner

Thanks for the PR. I'm glad you're enjoying the LSP!

Could you provide more detail on what "ignore responses and behave very strangely" looks like in practice? And how you're confident that this change fixes the problem?

Before merging this I'll need to test it with Janet++ to make sure it doesn't break anything in the VS Code implementation.

@chiefnoah
Copy link
Contributor Author

chiefnoah commented Jul 16, 2024

Could you provide more detail on what "ignore responses and behave very strangely" looks like in practice? And how you're confident that this change fixes the problem?

I believe it was effectively this:

Until the server has responded to the initialize request with an InitializeResult, the client must not send any additional requests or notifications to the server.

from the LSP Spec

The neovim LSP client would start the LSP process and send an initialize request, which I was able to see using the debug console (or stderr), which janet-lsp would respond with the appropriate message. And then nothing.

Neovim would show that the server was running (confirmed via ps), but the LSP would not attach to the buffer and the nvim lsp client wouldn't send any more messages. It looks like this:

screenshot-20240716-135748Z-selected

So without any new requests from the client, the LSP server isn't able to... well do anything, and so none of the LSP functionality works.

With my changes (really just the line ending changes, the rest are unrelated), the expected behavior happens and I'm able to get docstrings and stuff (notice, the buffer ID):

screenshot-20240716-135410Z-selected

screenshot-20240716-140604Z-selected

I believe neovim is somewhat picky here and always expects \r\n. To be fair, the spec only seems to give one example and doesn't really expand on it as far as I can tell.

@chiefnoah
Copy link
Contributor Author

I'll fix up the PR after work today. I'm not able to test on VSCode at the moment, but I can also attempt that if someone else doesn't get to it first.

@CFiggers
Copy link
Owner

Hmmm... I've been using Janet LSP together with Neovim for some time now (at least a couple months) and haven't encountered that issue. (Just to state the obvious, I'm still interested in fixing this because if you're hitting it then other folks might be as well. 🙂)

I'm curious, are you running under Windows or a Unix-like (Mac, Linux, BSD)?

@chiefnoah
Copy link
Contributor Author

Hmmm... I've been using Janet LSP together with Neovim for some time now (at least a couple months) and haven't encountered that issue. (Just to state the obvious, I'm still interested in fixing this because if you're hitting it then other folks might be as well. 🙂)

I'm curious, are you running under Windows or a Unix-like (Mac, Linux, BSD)?

I'm using Linux (NixOS, specifically). I have a mac that I can test on later.

@chiefnoah
Copy link
Contributor Author

It may also be a bug or change in the latest release of nvim, I'm using 0.10

@chiefnoah
Copy link
Contributor Author

Interestingly, I do not appear to have this problem on MacOS. Perhaps this is a platform-specific issue! I'll see if other Linux distros give me trouble

@chiefnoah
Copy link
Contributor Author

On a mostly unrelated note, nvim-lspconfig now has a basic config for janet_lsp

@chiefnoah
Copy link
Contributor Author

Interestingly, I do not appear to have this problem on MacOS. Perhaps this is a platform-specific issue! I'll see if other Linux distros give me trouble

I was unable to reproduce on a Ubuntu VM so it has to be something funky with NixOS. Perhaps it's a locale thing? I'm really not sure.

@chiefnoah
Copy link
Contributor Author

I'm starting to think this is somehow a locale or kernel-level issue. I'm running archlinux/archlinux from docker.io in podman and am seeing this issue:

screenshot-20240717-140834Z-selected

Will test with some other platforms. It could be that this is new behavior in neovim itself.

@chiefnoah
Copy link
Contributor Author

Ok, I have reproduced it using the release-0.10 branch of neovim on a clean Ubuntu 24.04 (Noble) VM and on a Archlinux container. The issue appears to be a change in LSP client expectations/behavior in neovim then. I could do a bisect, but that starts to get more complicated.

Regardless, the \n\n vs \r\n\r\n behavior is invalid behavior, the spec is more explicit about this than I originally thought.

@chiefnoah
Copy link
Contributor Author

Some more analysis: the neovim RPC client always splits on \r\n and probably wouldn't get valid results for \n\n. Are you using the builtin neovim LSP client or something like coc.nvim which implements its own client?

@CFiggers
Copy link
Owner

Regardless, the \n\n vs \r\n\r\n behavior is invalid behavior, the spec is more explicit about this than I originally thought.

That's pretty definitive. 😁 Thanks for finding that.

Once we get the other stuff cleaned up I'll be happy to merge.

The line ending is *always* \r\n as far as neovim is concerned. Futher,
these values are effectively constants, no reason to wrap them in
function calls.
@chiefnoah
Copy link
Contributor Author

Alright, should be cleaned up now. Let me know if there's anything else this needs!

@CFiggers
Copy link
Owner

CFiggers commented Jul 22, 2024

It looks like you still have a call to read-offset as a function and not a value in src/main.janet line 234.

https://github.com/chiefnoah/janet-lsp/blob/71df200073f7193597879919e1b3a4c8cc0ac6ea/src/main.janet#L232-L236

@chiefnoah
Copy link
Contributor Author

Oh huh, whoops

@CFiggers CFiggers merged commit 52dc61f into CFiggers:master Jul 22, 2024
@CFiggers
Copy link
Owner

As it turns out, it seems like I'm going to need to revert this pull request—whatever the specification may say, experimentally Windows needs line-ending to be "\n\n" and not "\r\n\r\n".

I don't know exactly why this would be— at this point I suspect it has to do with the way STDIO works on Windows? One debug test I did had four separate line breaks showing up, making me think that perhaps Windows is somehow interpreting both \r and \n as a full carriage return (CRLF) each? I really don't know. But testing both in Neovim and VS Code, everything works on Windows when line-ending is "\n\n" and fails when it is "\r\n\r\n".

The current master branch, which is using "\r\n\r\n" everywhere per this PR, should be working fine on Linux.

Notably, it looks like I myself introduced a bug related to this back in 90a33d2 , which might be why you first encountered issues with carriage returns.

At any rate, I'll be pushing a patch to master later today.

@chiefnoah
Copy link
Contributor Author

I suspect there is something else in the chain turning \n into \r\ns for Windows (perhaps in Janet itself, who knows). Looks very similar to this issue

@CFiggers
Copy link
Owner

Yep, that looks very similar indeed. Great find!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants