initial support for running rules on Scala 3 sources #210

bjaglin · 2021-05-10T21:45:08Z

See open questions / limitations in scalacenter/scalafix#1316 (comment)

Note that most of the built-in Scalafix rules do not support Scala 3 sources at the moment.

bjaglin · 2021-05-10T22:06:06Z

src/main/scala/scalafix/internal/sbt/SemanticRuleValidator.scala

      interface: ScalafixInterface
  ): Seq[String] = {
    if (files.isEmpty) Nil
    else {
      val errors = ListBuffer.empty[String]
      val hasSemanticdb =
-        dependencies.exists(_.name.startsWith("semanticdb-scalac"))
+        dependencies.exists(_.name.startsWith("semanticdb-scalac")) ||
+          scalacOpts.contains("-Xsemanticdb")


this will bring a false positive error with early dotty & 3.0.0-M* compilers, but I guess it's fine

The goal is to support the final release of scala 3, so I agree it's fine

bjaglin · 2021-05-10T22:14:36Z

src/main/scala/scalafix/sbt/ScalafixEnable.scala

+  lazy val command =
+    if (isSemanticdbPluginAvailable) withSemanticdbPluginCommand
+    else withoutSemanticdbPluginCommand
+
+  private lazy val withSemanticdbPluginCommand = Command.command(
+    "scalafixEnable",
+    briefHelp = "Configure SemanticdbPlugin and scalaVersion for scalafix.",
+    detail =
+      """1. enables SemanticdbPlugin with the right version supported by scalafix
+        |2. sets scalaVersion to latest Scala version supported by scalafix""".stripMargin


This changes the implementation of scalafixEnable for sbt 1.3+ users, so it has the potential of breaking a few things, especially on complex projetcts with many modules and/or scala versions. I wonder if we could smoke-test it with scala steward that relies on it?

Enabling semanticdb with the setting key semanticdbEnabled before Scala 3, is supposed to be equivalent to adding it through enablePlugin (for sbt 1.3+). As far as I understood, it translates to adding -Xplugin:...
@adpi2 do you think we need to be more careful with this change ?

Yes, using SemanticdbPlugin should be equivalent, but there are subtle changes in terms of scopes that might make it regress (see my comment below also).

In the amended commit, I have reworked how SemanticdbPlugin is enabled to make sure it's matching where semanticdb-scalac was currently injected prior to this PR (AND on scala3 obviously). I also added a dedicated scripted to test regressions of scalafixEnable.

bjaglin · 2021-05-10T22:18:02Z

src/sbt-test/sbt-1.5/scala-3/project/plugins.sbt

+
+//FIXME: remove when scalafixVersion is bumped to 0.9.28
+resolvers += Resolver.sonatypeRepo("snapshots")
+dependencyOverrides += "ch.epfl.scala" % "scalafix-interfaces" % "0.9.27+52-6c9eeec9-SNAPSHOT"


https://github.com/scalacenter/scalafix/commits/6c9eeec9bc99522dc2073ae54290488c6d06401f

bjaglin · 2021-05-11T07:16:29Z

src/sbt-test/sbt-1.5/scala-3/project/build.properties

@@ -0,0 +1 @@
+sbt.version=1.5.2


The way scripted are tested against different versions of sbt is quite messy at the moment, but I would rather wait for scalacenter/scalafix#1333 to clean it up.

bjaglin · 2021-05-11T12:19:12Z

src/main/scala/scalafix/sbt/ScalafixEnable.scala

+      inConfig(_)(ScalafixPlugin.relaxScalacOptionsConfigSettings)
+    )
+    val settings = for {
+      p <- extracted.structure.allProjectRefs


This is enabling SemanticdbPlugin on all projects of the build on purpose, since projectsWithMatchingScalaVersion would ignore Scala 3 (as it's driven by the full Scala version semanticdb-scalac is published against, so irrelevant as semanticdb is baked-in in the Scala 3 compiler). I can make it slightly more precise though, by adding an explicit case for Scala 3, to avoid enabling semanticdb on Scala 2.10 projects (which would fail).

Done in the amended commit

mlachkar · 2021-05-11T11:16:20Z

src/main/scala/scalafix/internal/sbt/SemanticRuleValidator.scala

      interface: ScalafixInterface
  ): Seq[String] = {
    if (files.isEmpty) Nil
    else {
      val errors = ListBuffer.empty[String]
      val hasSemanticdb =
-        dependencies.exists(_.name.startsWith("semanticdb-scalac"))
+        dependencies.exists(_.name.startsWith("semanticdb-scalac")) ||
+          scalacOpts.contains("-Xsemanticdb")


The goal is to support the final release of scala 3, so I agree it's fine

mlachkar · 2021-05-11T11:27:39Z

src/main/scala/scalafix/sbt/ScalafixEnable.scala

+  lazy val command =
+    if (isSemanticdbPluginAvailable) withSemanticdbPluginCommand
+    else withoutSemanticdbPluginCommand
+
+  private lazy val withSemanticdbPluginCommand = Command.command(
+    "scalafixEnable",
+    briefHelp = "Configure SemanticdbPlugin and scalaVersion for scalafix.",
+    detail =
+      """1. enables SemanticdbPlugin with the right version supported by scalafix
+        |2. sets scalaVersion to latest Scala version supported by scalafix""".stripMargin


Enabling semanticdb with the setting key semanticdbEnabled before Scala 3, is supposed to be equivalent to adding it through enablePlugin (for sbt 1.3+). As far as I understood, it translates to adding -Xplugin:...
@adpi2 do you think we need to be more careful with this change ?

mlachkar · 2021-05-11T11:28:51Z

src/main/scala/scalafix/sbt/ScalafixEnable.scala

+
+  private lazy val isSemanticdbPluginAvailable = Try {
+    Class.forName("sbt.plugins.SemanticdbPlugin")
+    true


So is it the same as verifying that the scalacOptions contain -Xplugin:sematicdb ?

No, this is just a way of detecting that the semanticdb* keys we redeclare above (since they are not available in the sbt 1.2.x we build against for retro compatibility) will have an effect. I find it more relevant to check for existence of that class in the classpath than the sbt version, but we could do that too:

sbt-scalafix/src/main/scala/scalafix/internal/sbt/SemanticRuleValidator.scala

Lines 41 to 45 in be4486b

val recommendedSetting = CrossVersion.partialVersion(sbtVersion) match {

case Some((1, n)) if n < 3 => atMostSbt12

case Some((0, _)) => atMostSbt12

case _ => atLeastSbt13(scalaVersion)

}

.

(we should probably reuse the same helper for both cases actually since the intent is the same)

that code was extracted to SemanticdbPlugin to be more explicit and shared

- use sbt 1.3+ SemanticdbPlugin to offload the choice of the right compiler options to get semanticdb output from the compiler or the compiler plugin (with the right semanticdb and Scala full version) depending on the Scala binary version - extend SemanticRuleValidator to detect Scala 3 compiler flag injected by SemanticdbPlugin - always pass the scala version for scalafix to parse with the right dialect

bjaglin · 2021-05-11T22:56:16Z

src/sbt-test/sbt-1.5/scalafixEnable/build.sbt

@@ -0,0 +1,19 @@
+ThisBuild / scalafixDependencies += "ch.epfl.scala" %% "example-scalafix-rule" % "1.6.0"


This contains a very simple semantic rule so it's good for an initial scripted until built-in rules pick up actual support. The only downside is that the rule does not even evaluate the semantic document, so I am not sure it really tests the generation of semanticdb compiler output... Maybe we could tag a 1.6.1 with https://github.com/scalacenter/sbt-scalafix/pull/125/files#diff-38f3b339536ccc14af957f820b7bfda9be5e1226210bb2d8938d7fab17003911R7 to lift this doubt?

bjaglin · 2021-05-11T22:59:27Z

src/sbt-test/sbt-1.5/scala-3/test

@@ -0,0 +1,4 @@
+# check that we can run a syntactic rule against a Scala 3 dialect source file
+-> scalafix --check LeakingImplicitClassVal
+> scalafix LeakingImplicitClassVal


I found a built-in syntactic rewriting rule that works on a scala3 source with significant indentation!

[info] [info] Running scalafix on 1 Scala sources [info] [error] --- /tmp/sbt_704f6c79/src/main/scala/SignificantIndentation.scala [info] [error] +++ <expected fix> [info] [error] @@ -1,3 +1,3 @@ [info] [error] object SignificantIndentation: [info] [error] - implicit class XtensionVal(val str: String) extends AnyVal: [info] [error] + implicit class XtensionVal(private val str: String) extends AnyVal: [info] [error] def doubled: String = str + str

🎉

bjaglin · 2021-05-11T23:03:02Z

src/main/scala/scalafix/sbt/ScalafixPlugin.scala

@@ -383,6 +383,7 @@ object ScalafixPlugin extends AutoPlugin {
            Arg.PrintStream(errorLogger),
            Arg.Config(scalafixConf),
            Arg.Rules(shell.rules),
+            Arg.ScalaVersion(scalaVersion.value),


I verified that this change is required for the syntactic-only invocation in the scala-3 scripted to successfully parse a source with significant indentation.

bjaglin · 2021-05-11T23:09:33Z

src/main/scala/scalafix/sbt/ScalafixEnable.scala


-/** Command to automatically enable semanticdb-scalac for shell session */
+/** Command to automatically enable semanticdb compiler output for shell session


This class is getting quite complex as it currently handles:

different sbt/Zinc versions

1.x with x >= 3 SemanticdbPlugin available

1.x with x < 3 SemanticdbPlugin unavailable

0.13.x SemanticdbPlugin unavailable & forcing usage of Compat

different scalac versions and how they may support semanticdb

3.x built-in

2.x with x >= 11 supported by semanticdb-scalac

2.x with x < 11 unsupported by semanticdb-scalac

bjaglin · 2021-05-11T23:12:44Z

.scalafmt.conf

@@ -1,5 +1,6 @@
-version = "2.7.5"
+version = "3.0.0-RC2"


bumped in a preliminary commit to support the sources with significant indentation in the scripted tests

bump scalafmt to support scala3 in the source tree

dcaed38

bjaglin changed the title ~~initial support for running rules on scala3 sources~~ initial support for running rules on Scala 3 sources May 10, 2021

bjaglin force-pushed the scala3 branch from bed8c0f to a053719 Compare May 10, 2021 22:04

bjaglin commented May 10, 2021

View reviewed changes

bjaglin force-pushed the scala3 branch from a053719 to be4486b Compare May 10, 2021 22:12

bjaglin commented May 10, 2021

View reviewed changes

bjaglin commented May 11, 2021

View reviewed changes

mlachkar reviewed May 11, 2021

View reviewed changes

bjaglin mentioned this pull request May 11, 2021

Scala 3 support scalacenter/scalafix#1316

Closed

5 tasks

bjaglin force-pushed the scala3 branch from be4486b to 4f8b090 Compare May 11, 2021 22:27

bjaglin force-pushed the scala3 branch from 4f8b090 to 5e14eb5 Compare May 11, 2021 22:55

bjaglin commented May 11, 2021

View reviewed changes

bjaglin marked this pull request as ready for review May 11, 2021 23:14

bjaglin requested a review from mlachkar May 11, 2021 23:14

mlachkar approved these changes May 12, 2021

View reviewed changes

bjaglin merged commit fdba46f into scalacenter:main May 12, 2021

This was referenced May 12, 2021

make it possible to and do build against a scalafix SNAPSHOT #211

Merged

Test Scala 3 input/output with rule built against Scala 2.12 scalacenter/scalafix.g8#19

Merged

mlachkar mentioned this pull request Dec 17, 2021

Scala 3 support cosmicsilence/gradle-scalafix#60

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

initial support for running rules on Scala 3 sources #210

initial support for running rules on Scala 3 sources #210

bjaglin commented May 10, 2021 •

edited

Loading

bjaglin May 10, 2021 •

edited

Loading

mlachkar May 11, 2021

bjaglin May 10, 2021

mlachkar May 11, 2021

bjaglin May 11, 2021

bjaglin May 11, 2021 •

edited

Loading

bjaglin May 10, 2021

bjaglin May 11, 2021

bjaglin May 11, 2021 •

edited

Loading

bjaglin May 11, 2021

mlachkar May 11, 2021

mlachkar May 11, 2021

mlachkar May 11, 2021

bjaglin May 11, 2021 •

edited

Loading

bjaglin May 11, 2021

bjaglin May 11, 2021

mlachkar May 12, 2021

bjaglin May 11, 2021

bjaglin May 11, 2021

bjaglin May 11, 2021 •

edited

Loading

bjaglin May 11, 2021

	val recommendedSetting = CrossVersion.partialVersion(sbtVersion) match {
	case Some((1, n)) if n < 3 => atMostSbt12
	case Some((0, _)) => atMostSbt12
	case _ => atLeastSbt13(scalaVersion)
	}

		@@ -0,0 +1,19 @@
		ThisBuild / scalafixDependencies += "ch.epfl.scala" %% "example-scalafix-rule" % "1.6.0"


		/** Command to automatically enable semanticdb-scalac for shell session */
		/** Command to automatically enable semanticdb compiler output for shell session

		@@ -0,0 +1 @@
		sbt.version=1.5.2

initial support for running rules on Scala 3 sources #210

initial support for running rules on Scala 3 sources #210

Conversation

bjaglin commented May 10, 2021 • edited Loading

bjaglin May 10, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bjaglin May 11, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bjaglin May 11, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bjaglin May 11, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bjaglin May 11, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bjaglin commented May 10, 2021 •

edited

Loading

bjaglin May 10, 2021 •

edited

Loading

bjaglin May 11, 2021 •

edited

Loading

bjaglin May 11, 2021 •

edited

Loading

bjaglin May 11, 2021 •

edited

Loading

bjaglin May 11, 2021 •

edited

Loading