Presentation Compiler / Language Server API

[seantalts]

This issue is meant to track the various language server APIs we think are most important and want to expose via the stanc3 compiler. We'll write out the API specs (stealing them from dart's API as needed) and see if we can come up with a high level description of how we can answer these questions either with existing functions and datastructures or figuring out what we need to change in order to be able to implement these API calls. We can have discussion below and keep this top-level comment updated with the proposals for APIs and their implementation techniques.

• analysis.getErrors
  . variety of types, including compile time error, hint, lint, static type warning, static warning, syntax error, todo
• search.findReferences - give the server an offset within the filepath, get back a list of locations, kind of reference (declaration, invocation, read, read_write, reference, unknown, write)
• search.findDeclarations - takes in a var name, returns search results as above
• edit.format - format the file, also give it a current selection so that the current selection can be updated after formatting (but preserving the selection doesn't seem super urgent). returns a list of SourceEdits (offsets, lengths, string to replace that region with)
• edit.getAssists - this one returns SourceChanges, which are slightly different from SourceEdits. These also have a message and a way of grouping edits together that I don't understand.
• edit.getFixes given a location in a file, return a list of AnalysisErrorFixes, which are just an AnalysisError and a SourceChange.
• completion.getSuggestions

The responses often include these types:

• Location
  • basically just need start location within the source file
• AnalysisError
  • location, message, correction suggestion text
• SourceChange
  • start and end location of the region to be modified, replacement string for the region
• CompletionSuggestion
  • identifier to be inserted, offset where it should be inserted, information about the element to be inserted (parameter names, returnType, typeParameters, declaration location)
  • can be one of these kinds of completion: argument list, import, identifier, invocation, keyword, named argument, optional argument, parameter

These all tend to allow someone to stream results and subscribe to certain things, but we can worry more about that when actually implementing the language server. I think for now it's enough to think about returning lists or using generators of some kind.

@matklad you mentioned on the hangout that we should maintain purity and keep file IO to the extreme boundaries. So when we're thinking about these APIs, how should we refer to files or buffers internally? I think that's a major question for me - the LSP and dart APIs seem to talk about files and locations within files a lot, so I'm not sure if we should internally just pass an entire AST and a node pointer everywhere? Can you tell us more about the internal representations of files for this kind of API?

Thanks @matklad for putting this on our radar and helping guide us through it! (from #65)

[matklad]

The list look good to me. Two relatively easy to implement, but important for usability features I would add to the list are

• Outline (a tree of symbols in the current file): this is useful to have a quick "jump to symbol in file by fuzzy name" feature

• Extend Selection. There's no API for this in Dart, however, there's a proposed extension to the LSP. The API allows you to extend selection to larger syntax nodes. This + multiple cursors is like vim text objects, but with superpowers.

how should we refer to files or buffers internally?

Good question! One thing you should probably not do is to use bare file-systems path. A super-concrete use-case here is Source Graph, which stores actual files on remote machines an generally refers to files by URLs. Another problem with paths is that they are platform specific and non-canonical.

Using an URL (or just an opaque wrapper around a string) to refer to files would work OK (as long as you use a special URL type, and not string). In Rust analyzer, actually go further and use integer ids to refer to files. A mapping from ids to paths is stored outside of the core analyzer, so there's simply no way to read data from files, neighboring to the current one. This is probably an overkill though.