gopls/vulncheck: use result from gopls.vulncheck, rather than gopls.fetch_vulncheck_result

[findleyr]

In https://go.dev/cl/621055, I added support async request handling to gopls, using it to make long running command handlers run asynchronous to other LSP requests. This means that we no longer need our hacky gopls.fetch_vulncheck_result command to collect vulncheck results: the client can simply await the command result.

I didn't remove the gopls.fetch_vulncheck_result command. However, there IS an observable change in behavior for vulncheck integration in gopls: previously the gopls.run_vulncheck command would return quickly. Now it does not.

Therefore, we need to:

1. Check that this blocking of the workspace/executeCommand request doesn't cause problems for VS Code.
2. Eventually stop using gopls.fetch_vulncheck_result in VS Code.

[findleyr]

Decisions, from our meeting:

1. We'll revert gopls.run_vulncheck to its old behavior, since the new behavior breaks special handling to stream vulncheck results to a terminal.
2. We'll add a new gopls.vulncheck command that implements the new behavior. So the old behavior uses two commands (gopls.run_vulncheck and gopls.fetch_vulncheck_result), and the new behavior has just one (gopls.vulncheck).
3. We'll generalize the special handling that displays vulncheck results in a terminal output. Specifically, we'll document special metadata that gopls may embed in a header of the workDoneProgressBegin message.

In pseudocode:

workDoneProgressBegin{
  title: "⚡ govulncheck -C path/to/dir",
  message: "style: log\n\nStarting govulncheck...",
}

In this case, the header has a single piece of metadata: style: log, which signals to clients that the preferred output style for this progress notification is a log (=append) style. Of course, most clients will not have any special handling, but the default UX will be somewhat reasonable: the progress notification will show the most recent log message. Previously, we had discussed style: terminal, but @adonovan points out that this may make it unclear whether the output should support ANSI escape codes.

CC @hyangah @h9jiang @adonovan

[adonovan]

FWIW this is the upstream issue for the LSP protocol change that would make this unnecessary:

• support for executeCommand to produce additional output streams through the progress notification mechanism microsoft/language-server-protocol#2014

(Which will happen first: this proposal getting approved, or the sun exhausting its nonmetallic fuel? Only time will tell!)

[hyangah]

I don't know what is the current status of gopls.
Our CI started to be failing with vulncheck unit tests and I guess it's related to gopls v0.17.0-pre.1 release.
https://go-review.git.corp.google.com/c/vscode-go/+/627276

[findleyr]

Shoot, I forgot to fix this before the -pre.1 release. I'll fix now, and we can cut -pre.2.

[gopherbot]

Change https://go.dev/cl/627556 mentions this issue: gopls/internal/server: revert the gopls.run_govulncheck command

[h9jiang]

Hi @findleyr

I have done some reading in vscode-go side but I need more clarification on this issue. Based on my understanding, the current state is:

• VSCode-Go is still using the old way, gopls.run_govulncheck and gopls.fetch_vulncheck_result to run and get the result. They are being printed in the terminal today in vscode.
• Gopls have three commands run_govulncheck gopls.fetch_vulncheck_result and gopls.vulncheck. The first two are paired together and the last one is a sync call so it will return result without have to async.

The end goal of this is:

• Set up a framework for the async behavior (where gopls have to do it using async call). VSCode-Go can watch for workprogress's message when workprogress.kind == begin. If the message have special metadata, style = log, this is something we what to print to terminal. We will create one terminal per token, attach message to the terminal as more workprogress coming to the client side. One the workprogress.kind == end shows up, we will print the final result to the terminal and finish (leave the terminal open for user to check result, but will no longer print any more logs).
• Migrate the async call which uses gopls.run_govulncheck and gopls.fetch_vulncheck_result to sync call gopls.vulncheck. But after migration, vulncheck will not use the framework defined above.

Other things:

• There might be some versions where both async and sync call need to be supported from vscode-go. Maybe for one or two releases.
• Gopls today does not send style=log metadata info to message. So some gopls side change is needed.

[findleyr]

Thanks, @h9jiang, I think that sounds approximately correct. Responses below.

Set up a framework for the async behavior (where gopls have to do it using async call). VSCode-Go can watch for workprogress's message when workprogress.kind == begin.

You mean, this is how it currently behaves, right?

There might be some versions where both async and sync call need to be supported from vscode-go. Maybe for one or two releases.
Gopls today does not send style=log metadata info to message. So some gopls side change is needed.

Good point. Do you want to start by adding support to VS Code for detecting "style=log", and then add this to gopls? Perhaps we should inject a setting into the gopls initialization options to indicate that the client support this type of progress redirection. That way, we don't show this in other clients.

[h9jiang]

You mean, this is how it currently behaves, right?

Today, the current implementation is limited only for vulncheck. It's kind of customized for vulncheck. What I'm asking is, should generalize this kind of customization so it can be used for other async calls that gopls may support in the future?

We have some pre-defined text customize for vulncheck, to generalize this, these text would be better to integrated in gopls instead of pre-defining them in vscode-go. Like Code.

My intention here is, minimum customization logic in vscode-go so it's more generalize. This means gopls will need to report some extra text back to vscode-go.

Do you want to start by adding support to VS Code for detecting "style=log", and then add this to gopls?

Sure. I will start working on this. But maybe one step at a time,

• First generalize the solution for other async calls.
• Migrate the vulncheck async call to sync call.

Thank you for the clarification.

[findleyr]

Generalizing sounds good. In that case, should we include the command that is running in the structured initial progress message?

message: "style: log\ncommand:govulncheck -C ${dir} ./...\n\nStarting govulncheck..."

[gopherbot]

Change https://go.dev/cl/644896 mentions this issue: gopls/internal/server: gopls.vulncheck return both vuln report and token

[h9jiang]

With some more changes in vscode-go and gopls, the behavior looks below:

This is the desired behavior. Trigger run govulncheck from go.mod file through gopls.vulncheck. enabled by

  "gopls": {
    "ui.codelenses": {
      // "run_govulncheck": true,
      "vulncheck": true,
    },
  }

This is the current behavior. Trigger run govulncheck from go.mod file through gopls.run_govulncheck. enabled by

  "gopls": {
    "ui.codelenses": {
      "run_govulncheck": true,
      // "vulncheck": true,
    },
  }

As shown above, they behave very similar. But the benefit of the first solution is, it's extensible for any workDoneProgress that prefer to stream logs to terminal. As long as gopls send in workDoneProgress{kind: begin, message: style: log\n\nfoo}, vscode-go will stream the to terminal.

Some LSP protocol extends workDoneProgress (e.g. executeCommand). If gopls send the same token in executeCommand's response, vscode-go will attach the result to the same terminal.

By default, the result will be dumped to the terminal, gopls.vulncheck have some special handling since the vulncheck is too noisy.

For gopls.vulncheck the communication looks like below:

• User click code lens.
• VSCode-Go save the workspace and send the executeCommand to gopls. VSCode-Go waiting for response...
• Gopls received the request but didn't respond.
• Gopls report workDoneProgress started. token: "aabb", kind: "begin", message: "style: log\n\nRunning..."
• VSCode-Go create a terminal, writing Running to the terminal.
• Gopls reports progress workDoneProgress. token: "aabb", kind: "reports", message: "vulncheck is running"
• VSCode-Go append vulncheck is running to the same terminal.
• Gopls report workDoneProgress finished. token: "aabb", kind: "end", message: "completed"
• VSCode-Go append completed to the same terminal.
• Gopls send the response for executeCommand to client. token: "aabb", result: "a very long report"
• VSCode-Go locate the terminal using the token and append simplified report to the same token.

CL will sent out for review soon.

[gopherbot]

Change https://go.dev/cl/645116 mentions this issue: extension/src: support stream $/progress message in log style

[gopherbot]

Change https://go.dev/cl/645695 mentions this issue: gopls/internal/server: embed style metadata in vulncheck progress