Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Fixes #227; add mill clean #315

Merged
merged 8 commits into from
May 16, 2018
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions docs/pages/1 - Intro to Mill.md
Original file line number Diff line number Diff line change
Expand Up @@ -456,6 +456,26 @@ $ mill show foo.compileDepClasspath
`show` is also useful for interacting with Mill from external tools, since the
JSON it outputs is structured and easily parsed & manipulated.

### clean

```bash
$ mill clean
```

`clean` deletes all the cached outputs of previously executed tasks. It can
apply to the entire project, entire modules, or specific tasks.

```bash
mill clean # clean all outputs
mill clean foo # clean all outputs for module 'foo' (including nested modules)
mill clean foo.compile # only clean outputs for task 'compile' in module 'foo'
mill clean foo.{compile,run}
mill clean "foo.{compile,run}"
mill clean foo.compile foo.run
mill clean _.compile
mill clean __.compile
```

## IntelliJ Support

Mill supports IntelliJ by default. Use `mill mill.scalalib.GenIdea/idea` to
Expand Down
42 changes: 41 additions & 1 deletion main/src/mill/main/MainModule.scala
Original file line number Diff line number Diff line change
@@ -1,10 +1,12 @@
package mill.main

import ammonite.ops.Path
import mill.define.{NamedTask, Task}
import mill.eval.{Evaluator, Result}
import mill.util.{EitherOps, ParseArgs, PrintLogger, Watched}
import mill.util.{PrintLogger, Watched}
import pprint.{Renderer, Truncated}
import upickle.Js

object MainModule{
def resolveTasks[T](evaluator: Evaluator[Any], targets: Seq[String], multiSelect: Boolean)
(f: List[NamedTask[Any]] => T) = {
Expand Down Expand Up @@ -35,6 +37,9 @@ trait MainModule extends mill.Module{
println(res)
res
}

private val OutDir: String = "out"

/**
* Resolves a mill query string and prints out the tasks it resolves to.
*/
Expand Down Expand Up @@ -177,4 +182,39 @@ trait MainModule extends mill.Module{
}
}
}

/**
* Deletes the given targets from the out directory. Providing no targets
* will clean everything.
*/
def clean(evaluator: Evaluator[Any], targets: String*) = mill.T.command {
val rootDir = ammonite.ops.pwd / OutDir

val KeepPattern = "(mill-.+)".r.anchored
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here's an idea: what if instead of this regex, we simply treated mill clean as the same as mill clean __? That should correctly delete everything that needs to be deleted while leaving the not-to-be-deleted files untouched

Copy link
Contributor Author

@guilgaly guilgaly May 14, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You just beat me to it! I was starting to think along those line, mainly for better consistency with the other tasks in the main module, but you're right that it should also remove the special-case code with the somewhat arbitrary regex. 👌

I'll try to implement that and see if there is any issue.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@guilgaly do you want to try out this one in this PR?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rockjam I'd say this PR is already useful, so I'll rather try to write the improved version for another future PR.

@lihaoyi & @rockjam : one thing I'm wondering about is wether we have a way to list all available external modules (like the GenIdea one)? As we'll want to clean their outputs too when doing mill clean __.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we have a way to list all of them right now; they just live arbitrarily on the classpath, and JVM classpath-scanning is not well supported (slow, unreliable, ...). Currently, their target directories are also randomly scattered throughout the out/ folder.

Perhaps we could just put all the ExternalModules into an mill-external folder, and add that folder to the hardcoded list of things to rm?


def keepPath(path: Path) = path.segments.lastOption match {
case Some(KeepPattern(_)) => true
case _ => false
}

val pathsToRemove =
if (targets.isEmpty)
Right(ammonite.ops.ls(rootDir).filterNot(keepPath))
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Turns out the error on "filter out" was caused by deleting out/mill-worker-1. For now I have hard-coded a filter for anything like out/mill-worker-N, but:

  • I'm not sure this is what we'd want?
  • If we do, I don't know if there is a better way to filter those folders ?

else
RunScript.resolveTasks(
mill.main.ResolveSegments, evaluator, targets, multiSelect = true
).map(
_.map { segments =>
Evaluator.resolveDestPaths(rootDir, segments).out
})

pathsToRemove match {
case Left(err) =>
Result.Failure(err)
case Right(paths) =>
paths.foreach(ammonite.ops.rm)
Result.Success(())
}
}

}
70 changes: 70 additions & 0 deletions main/src/mill/main/Resolve.scala
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,76 @@ object ResolveMetadata extends Resolve[String]{
}
}

object ResolveSegments extends Resolve[Segments] {

override def endResolveCross(obj: Module,
revSelectorsSoFar: List[Segment],
last: List[String],
discover: Discover[_],
rest: Seq[String]): Either[String, Seq[Segments]] = {
obj match{
case c: Cross[Module] =>
last match{
case List("__") => Right(c.items.map(_._2.millModuleSegments))
case items =>
c.items
.filter(_._1.length == items.length)
.filter(_._1.zip(last).forall{case (a, b) => b == "_" || a.toString == b})
.map(_._2.millModuleSegments) match {
case Nil =>
Resolve.errorMsgCross(
c.items.map(_._1.map(_.toString)),
last,
revSelectorsSoFar
)
case res => Right(res)
}
}
case _ =>
Left(
Resolve.unableToResolve(Segment.Cross(last), revSelectorsSoFar) +
Resolve.hintListLabel(revSelectorsSoFar)
)
}
}

def endResolveLabel(obj: Module,
revSelectorsSoFar: List[Segment],
last: String,
discover: Discover[_],
rest: Seq[String]): Either[String, Seq[Segments]] = {
val target =
obj
.millInternal
.reflect[Target[_]]
.find(_.label == last)
.map(t => Right(t.ctx.segments))

val command =
Resolve
.invokeCommand(obj, last, discover, rest)
.headOption
.map(_.map(_.ctx.segments))

val module =
obj.millInternal
.reflectNestedObjects[Module]
.find(_.millOuterCtx.segment == Segment.Label(last))
.map(m => Right(m.millModuleSegments))

command orElse target orElse module match {
case None =>
Resolve.errorMsgLabel(
singleModuleMeta(obj, discover, revSelectorsSoFar.isEmpty),
last,
revSelectorsSoFar
)

case Some(either) => either.right.map(Seq(_))
}
}
}

object ResolveTasks extends Resolve[NamedTask[Any]]{


Expand Down
11 changes: 11 additions & 0 deletions readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,17 @@ You can get Mill to show the JSON-structured output for a particular `Target` or

Output will be generated into a the `./out` folder.

You can clean the project using `clean`:

```bash
# Clean entire project.
mill clean
# Clean a single target.
mill clean main
# Clean multiple targets.
mill clean main core
```

If you are repeatedly testing Mill manually by running it against the `build.sc`
file in the repository root, you can skip the assembly process and directly run
it via:
Expand Down