daemon.md

Gopls: Running as a daemon

Note: this feature is new. If you encounter bugs, please file an issue.

If you just want to try this out, skip ahead to the quickstart.

Background: gopls execution modes

Gopls was originally implemented as an LSP sidecar: a process started by editors or editor plugins, and communicated with using jsonrpc 2.0 over stdin/stdout. By executing as a stateful process, gopls can maintain a significant amount of cache and can eagerly perform analysis on the source code being edited.

This execution mode does not work as well when there are many separate editor processes or when editor processes are short-lived, as is often the case for users of non-IDE editors such as Vim or Emacs. Having many processes means having many caches, consuming a significant amount of system resources. Using short-lived sessions means paying a start-up cost each time a session is created.

To support these types of workflows, a new mode of gopls execution is supported wherein a single, persistent, shared gopls "daemon" process is responsible for managing all gopls sessions. In this mode, editors still start a gopls sidecar, but this sidecar merely acts as a thin "forwarder", responsible for forwarding the LSP to the shared gopls instance and recording metrics, logs, and rpc traces.

Quickstart

To use a shared gopls instance you must either manage the daemon process yourself, or let the gopls forwarder processes start the shared daemon as needed.

Running with -remote=auto

Automatic management of the daemon is easiest, and can be done by passing the flag -remote=auto to the gopls process started by your editor. This will cause this process to auto-start the gopls daemon if needed, connect to it, and forward the LSP. For example, here is a reasonable gopls invocation, that sets some additional flags for easier debugging:

gopls -remote=auto -logfile=auto -debug=:0 -remote.debug=:0 -rpc.trace

Note that the shared gopls process will automatically shut down after one minute with no connected clients.

Managing the daemon manually

To manage the gopls daemon process via external means rather than having the forwarders manage it, you must start a gopls daemon process with the -listen=<addr> flag, and then pass -remote=<addr> to the gopls processes started by your editor.

For example, to host the daemon on the TCP port 37374, do:

gopls -listen=:37374 -logfile=auto -debug=:0

And then from the editor, run

gopls -remote=:37374 -logfile=auto -debug=:0 -rpc.trace

If you are on a POSIX system, you can also use unix domain sockets by prefixing the flag values with unix;. For example:

gopls -listen="unix;/tmp/gopls-daemon-socket" -logfile=auto -debug=:0

And connect via:

gopls -remote="unix;/tmp/gopls-daemon-socket" -logfile=auto -debug=:0 -rpc.trace

(Note that these flag values MUST be enclosed in quotes, because ';' is a special shell character. For this reason, this syntax is subject to change in the future.)

Debugging

Debugging a shared gopls session is more complicated than a singleton session, because there are now two gopls processes involved with handling the LSP. Here are some tips:

Finding logfiles and debug addresses

When running in daemon mode, you can use the gopls inspect sessions command to find the logfile and debug port for your gopls daemon instance (as well as for all its connected clients). By default, this inspects the default daemon (i.e. -remote=auto). To inspect a different daemon, use the -remote flag explicitly: gopls -remote=localhost:12345 inspect sessions.

This works whether or not you have enabled -remote.debug.

Traversing debug pages

When -debug=:0 is passed to gopls, it runs a webserver that serves stateful debug pages (see troubleshooting.md). You can find the actual port hosting these pages by either using the gopls inspect sessions command, or by checking the start of the logfile -- it will be one of the first log messages. For example, if using -logfile=auto, find the debug address by checking head /tmp/gopls-<pid>.log.

By default, the gopls daemon is not started with -debug. To enable it, set the -remote.debug flag on the forwarder instance, so that it invokes gopls with -debug when starting the daemon.

The debug pages of the forwarder process will have a link to the debug pages of the daemon server process. Correspondingly, the debug pages of the daemon process will have a link to each of its clients.

This can help you find metrics, traces, and log files for all of the various servers and clients.

Using logfiles

The gopls daemon is started with logging disabled by default. To customize this, pass -remote.logfile to the gopls forwarder. Using -remote.logfile=auto, the daemon will log to a default location (on posix systems: /tmp/gopls-daemon-<pid>.log).

The gopls daemon does not log session-scoped messages: those are instead reflected back to the forwarder so that they can be accessed by the editor. Daemon logs will only contain global messages, for example logs when sessions connect and disconnect.

It is recommended to start the forwarder gopls process with -rpc.trace, so that its logfile will contain rpc trace logs specific to the LSP session.

Using multiple shared gopls instances

There may be environments where it is desirable to have more than one shared gopls instance. If managing the daemon manually, this can be done by simply choosing different -listen addresses for each distinct daemon process.

On POSIX systems, there is also support for automatic management of distinct shared gopls processes: distinct daemons can be selected by passing -remote="auto;<id>". Any gopls forwarder passing the same value for <id> will use the same shared daemon.

FAQ

Q: Why am I not saving as much memory as I expected when using a shared gopls?

A: As described in implementation.md, gopls has a concept of view/session/cache. Each session and view map onto exactly one editor session (because they contain things like edited but unsaved buffers). The cache contains things that are independent of any editor session, and can therefore be shared.

When, for example, three editor session are sharing a single gopls process, they will share the cache but will each have their own session and view. The memory savings in this mode, when compared to three separate gopls processes, corresponds to the amount of cache overlap across sessions.

Because this hasn't mattered much in the past, it is likely that there is state that can be moved out of the session/view, and into the cache, thereby increasing the amount of memory savings in the shared mode.

Q: How do I customize the daemon instance when using -remote=auto?

The daemon may be customized using flags of the form -remote.* on the forwarder gopls. This causes the forwarder to invoke gopls with these settings when starting the daemon. As of writing, we expose the following configuration:

• -remote.logfile: the location of the daemon logfile
• -remote.debug: the daemon's debug address
• -remote.listen.timeout: the amount of time the daemon should wait for new connections while there are no current connections, before shutting down. Must be set to a valid time.Duration (e.g. 30s or 5m). If 0, listen indefinitely. Default: 1m.

Note that once the daemon is already running, setting these flags will not change its configuration. These flags only matter for the forwarder process that actually starts the daemon.