Bsp improve

[lefou]

Reworked and improved BSP Support.

Fix #1035 - TaskFinish is never sent
Fix #986 - Support workspace/reload
Fix #1497 - compilation while BSP synchronization
Fix #1034 - BSP TaskProgress is not implemented correctly

[lefou]

Regarding SemanticDB: As a first step I added a new trait mill.scalalib.bsp.ScalaMetalsSupport with which you can mix in some essential scalac options including SemanticDB configuration in your Scala module.

object myScalaModule extends ScalaModule with ScalaMetalsSupport {
  def scalaVersion = ...
  def semanticDbVersion = ...
  ...
}

It's the simplest and most effective way to support Metals, if needed. We can come up with more sophisticated setups, but any clean solution is much more complex and probably involves redundant compile targets. I'm not saying we will never have them, but for now I'm mostly interested in out-of-the-box full support of (core) BSP.

[lefou]

When installing, you can now select the parallel jobs to use for the BSP server, e.g. select 0 to use as much threads as you have CPU cores.

mill mill.bsp.BSP/install --jobs 0

[tgodzik]

So the users would have to enable it specifically in their codebase by adding the trait? That might hard to sell to users.

Maybe, with this approach it would make sense to just have instead of ScalaMetalsSupport -> SemanticdbSupport ? Would also benefit scalafix etc..

We could also add a way to disable it, which people could do on CI? Something long the lines of:
def semanticEnabled = env.get("CI") == true

Edit: Rather semanticEnabled or metalsEnabled

[lefou]

So the users would have to enable it specifically in their codebase by adding the trait? That might hard to sell to users.

Yeah, you're probably right. It's the fast option to benefit right now, until we have an alternative. But I explicitly wanted to avoid some quick and dirty switch here. Most users already have a configuration trait, and it's a one-liner for them.

Maybe, with this approach it would make sense to just have instead of ScalaMetalsSupport -> SemanticdbSupport ? Would also benefit scalafix etc..

Yeah. I thought the same, but then, I also enabled some other scalac options and even added a filter for -Xfatal-warnings which is in favor of Metals.

We could also add a way to disable it, which people could do on CI? Something long the lines of: def semanticDbVersion = env.get("CI") == true

Yeah. Makes sense.

[lefou]

I was thinking if we could also check for existence/content of some file (roughly like .semanticdb.conf) and also initially create it from Metals (if not present), so we get switchable but stable compile results.

[lefou]

Or make it a persistent (external) target to avoid extra files in the repo root.

[tgodzik]

I was thinking if we could also check for existence/content of some file (roughly like .semanticdb.conf) and also initially create it from Metals (if not present), so we get switchable but stable compile results.

If that helps we could create it, there is always .metals/metals.log file in the workspace that indicates that user is using Metals. It could something like .metals/semanticdb.conf with the expected semanticdb version.

.metals is anyway ignored in git by users.

[lefou]

I'd like to summarize:

Mill has various IDE support:

• generates IDEA projects directly, supports all Languages and features, IDEA has
• provides a BSP server, supports all BSP clients
  ** IDEA, which supports Languages Java and Scala
  ** Metals as Language Server used by many different editors, supports only Language Scala; requires SemanticDB generator

As SemanticDB generation isn't cheap in terms of processing time and may have unforseen side-effects, we only want to enable it if it is in the interest of the user, which means: only enable it when the user is using Metals.

Three options to handle SemanticDB:

a) leaf it to the user to enable it in their build, e.g. use ScalaMetalsSupport trait.

pros:

• simple

cons:

• too simple, bad user experience

b) have a dedicated compile target for Metals (we could detect it in the BSP initialize request or by env var)

pros:

• only used when needed (by Metals)

cons:

• compilation in IDE differs from commandline
• customized toolchains need extra care

c) automatically enable SemanticDB in case the user uses Metals

pros:

• code compilation is the same for IDE and commandline (although changes once after Metals was started the first time)

cons:

• we need to be clever to not repeatedly flip the enablement when on commandline, so we need to make the enablement persistent
• we need to be clever to not enable it when inappropriate, e.g. in CI
• we need provide a way to disable it explicitly
• we need to handle Scala versions unsupported by the SemanticDB version
• invasive into existing targets: scalacOptions, scalacPluginIvyDeps, ...

How could we implement c):

i) depend on external info, e.g. existence of .metals directory or some file in it.

pros:

• easy to implement

cons:

• unexpected for Mill users, which assume the build.sc (and the actual mill args and cache) as single source of truth
• hard to disable, once the file exists

ii) persist the fact that we were once executed from Metals

pros:

• easy to reset with mill clean or mill clean <persistent-target>

cons:

• more complicated to implement
• probably hard to understand by user
• we will need some persistent targets to keep the state (combined with some input task to capture the env)

[lefou]

Because it is so much clearer, separated and properly cached, I would still prefer option b) (dedicated compile target). As a side note: when we use direct IDEA support, we also compile twice (from IDE and in mill). For me this is no issue, as I almost never hit the compile button in the IDE (probably an old Eclipse habit). :p

If we want to go route c) (auto-detect), than I'd prefer ii) (keep enablement in an persistent target).

[tgodzik]

b) and c) are both fine from Metals perspective. c) might be nicer for people using CLI a lot, but whatever seems better suited for Mill should be more important here.

[ckipp01]

When installing, you can now select the parallel jobs to use for the BSP server, e.g. select 0 to use as much threads as you have CPU cores.

mill mill.bsp.BSP/install --jobs 0

Reading back through all this again, I just stumbled on this. In Metals we now auto-generate this if a user is in a Mill workspace that supports BSP. We just use the default for the # of jobs. Do you think that's fine? And if a user does want more control over it they just create it manually?

[lefou]

When installing, you can now select the parallel jobs to use for the BSP server, e.g. select 0 to use as much threads as you have CPU cores.

mill mill.bsp.BSP/install --jobs 0

Reading back through all this again, I just stumbled on this. In Metals we now auto-generate this if a user is in a Mill workspace that supports BSP. We just use the default for the # of jobs. Do you think that's fine? And if a user does want more control over it they just create it manually?

Parallel job processing is rather stable and thus safe to use. Our reworked BSP is now usable on the happy path, but if something goes wrong, we have still rough edges (for example when we have errors in build.sc, #1560). Understanding these issues and debugging these will probably become a lot harded, when work and output is generated in a concurrent and simultaneous way. So it's probably a bit early to make it the default. It's better for now, to let the power-users enable it manually, but run with only one thread by default. This is wasting some performance potential, yes.