Split documentation into individual files in docs/

docs/notes/custom-mps-distribution.md

@@ -0,0 +1,41 @@
+## Custom MPS Distribution
+
+Features that perform an action inside an MPS project, like the `modelcheck` or `generate-models` plugin, require
+an MPS available to them. While for vanilla MPS it is enough to pass in a reference to the MPS dependency via the
+`mpsConfig` property, this doesn't work for custom distributions of MPS. A custom distribution of MPS is also called
+an MPS RCP. If you like to use your own MPS distribution with preinstalled plugins and your own versioning scheme
+then this is possible but requires additional steps in the build script.
+
+When you are using a custom distribution of MPS you can still use the `mpsConfig` property and rely on
+the plugin resolving it. However, you may need to configure explicit `mpsVersion` for the plugin. You can also use a
+custom `mpsLocation` with no value set for `mpsConfig`. In this case you _must_ configure `mpsVersion` as well.
+
+If you set `mpsVersion` but also set `mpsConfig` then `mpsVersion` will take precedence over the version of the
+dependency in the configuration. The plugin will resolve the specified configuration into `mpsLocation`.
+
+`mpsVersion` needs to be set to the exact MPS version your custom distribution is based on. For example, if you build an
+RCP with MPS 2020.3.3 you need to set this property to `2020.3.3`. `mpsLocation` needs to point to the location
+where you extracted your custom MPS distribution into e.g. `$buildDir/myAwesomeMPS` if you extracted into that location.
+
+Each of the plugins creates a `resolveMpsFor<name>` task in the build. When `mpsVersion` and `mpsLocation` are set
+this task is still present in the task graph but becomes a noop. The task is present to be able to add your own task(s)
+as dependency to it. This is useful for extracting your custom distribution before its being used. A minimal example
+could look like this:
+
+```
+def myCustomLocation = "$buildDir/myAwesomeMPS"
+
+task downloadAndExtractCustomMPS() {
+    // your logic to download and extract here
+}
+
+modelcheck {
+    mpsLocation = myCustomLocation
+    mpsVersion = "2020.3.3"
+    projectLocation = file("$rootDir/mps-prj")
+    modules = ["my.solution.with.errors"]
+    junitFile = file("$buildDir/TEST-modelcheck-results.xml")
+}
+
+tasks.getByName("resolveMpsForModelcheck").dependsOn(downloadAndExtractCustomMPS)
+```

docs/notes/mps-vs-idea-environment.md

@@ -0,0 +1,6 @@
+## MPS vs IDEA environment
+
+Since plugin version 1.10, the `environmentKind` parameter allows to choose between MPS or IDEA environment for
+generation or model check. The default is to use the IDEA environment because that was the case in earlier versions, but
+the MPS environment is lighter and likely to be more performant. On the other hand, the MPS environment does not load
+plugins (only extensions) and may lead to different results.

docs/plugins/download-jbr.md

@@ -0,0 +1,58 @@
+## Download JetBrains Runtime
+
+When building MPS projects with the JetBrains Runtime, the JDK/JRE used by MPS and other intellij based IDEs, it's
+required to download the correct version of the runtime. Since the runtime is platform dependent it's required to
+download a platform dependent binary. While it's possible to add the logic to your own build script we provide a convenient
+way of doing this with a Gradle plugin.
+
+The download-jbr plugin will add new dependencies and a task to your build. It will add a dependency to `com.jetbrains.jdk:jbr`
+to your build, you need to make sure that it is available in your dependency repositories. The itemis Maven repository at
+https://artifacts.itemis.cloud/repository/maven-mps provides this dependency, but you can create your own with
+the scripts located in [mbeddr/build.publish.jdk](https://github.com/mbeddr/build.publish.jdk).
+
+For easy consumption and incremental build support the plugin creates a task `downloadJbr` which exposes the location of
+the java executable via the `javaExecutable` property. See
+[the tests](../../src/test/kotlin/test/others/JBRDownloadTest.kt) for an example of how to use it.
+
+### Usage
+
+
+Kotlin:
+```
+plugins {
+    id("download-jbr")
+}
+
+repositories {
+    mavenCentral()
+    maven {
+        url = URI("https://artifacts.itemis.cloud/repository/maven-mps")
+    }
+}
+
+downloadJbr {
+    jbrVersion = "11_0_6-b520.66"
+}
+```
+
+Groovy:
+```
+apply plugin: 'download-jbr'
+...
+
+repositories {
+    maven { url 'https://artifacts.itemis.cloud/repository/maven-mps' }
+    mavenCentral()
+}
+
+downloadJbr {
+    jbrVersion = '11_0_6-b520.66'
+}
+```
+
+### Parameters
+* `jbrVersion` - version of the JBR to download. While this supports maven version selectors we highly recommend not
+  using wildcards like `*` or `+` in there for reproducible builds.
+* `distributionType` - optional distribution type for the JBR to use. Will default to `jbr_jcef` if omitted.
+* `downloadDir` - optional directory where the downloaded JBR is downloaded and extracted to. The plugin defaults to
+  `build/jbrDownload`. 

docs/plugins/generate.md

@@ -0,0 +1,67 @@
+## `generate`
+
+Generate a specific or all models in a project without the need for an MPS model.
+
+While technically possible generating languages with this task makes little sense as there is no way of packaging the
+generated artifacts into JAR files. We only recommend using this for simple tasks where user defined models should be
+generated in the CI build or from the commandline.
+
+### Usage
+
+A minimal build script to generate an MPS project with no external plugins would look like this:
+
+```
+apply plugin: 'generate-models'
+
+configurations {
+    mps
+}
+
+ext.mpsVersion = '2018.3.6'
+
+generate {
+    projectLocation = new File("./mps-prj")
+    mpsConfig = configurations.mps
+}
+
+dependencies {
+    mps "com.jetbrains:mps:$mpsVersion"
+}
+```
+
+Parameters:
+* `mpsConfig` - the configuration used to resolve MPS. Custom plugins are supported via the `pluginLocation` parameter.
+* `mpsLocation` - optional location where to place the MPS files if `mpsConfig` is specified, or where to take them from
+  otherwise.
+* `mpsVersion` - optionally overrides automated version detection from `mpsConfig`. Required if you use
+  a [custom distribution](../notes/custom-mps-distribution.md) of MPS.
+* `javaExec` - optional `java` executable to use.
+* `pluginLocation` - location where to load the plugins from. Structure needs to be a flat folder structure similar to the
+  `plugins` directory inside of the MPS installation.
+* `plugins` - deprecated, use `pluginsProperty`.
+* `pluginsProperty` - optional list of plugins to load before generation is attempted.
+  The notation is `new Plugin("pluginID", "somePath")`. The first parameter is the plugin id.
+  For the second parameter `"somePath"` there are several options:
+  * if it's an absolute path, the plugin is loaded from that path
+  * if it's a folder located under `pluginLocation` the plugin is loaded from that folder
+  * otherwise it should be a plugin folder located under the default `mps/plugins`
+* `models` - optional list of models to generate. If omitted all models in the project will be generated. Only full name
+  matched are supported and no RegEx or partial name matching.
+* `excludeModels` (since 1.14) - optional list of models to exclude from generating. RegEx can be used for matching multiple models.
+* `modules` (since 1.14) - optional list of modules to generate. Expects ordinary name (w/o virtual folders). RegEx can be used for matching multiple modules.
+  If both parameters, `models` and `modules`, are omitted - all models in the project will be generated, except as
+  excluded by `excludeModels` and `excludeModules`.
+* `excludeModules` (since 1.14) - optional list of modules to exclude from generate. RegEx can be used for matching multiple modules.
+* `macros` - optional list of path macros. The notation is `new Macro("name", "value")`.
+* `projectLocation` - location of the MPS project to generate.
+* `parallelGenerationThreads` (since 1.17) - optional number of threads to use for parallel generation. Defaults to `0`,
+  which means that parallel generation is turned off.
+* `debug` - optionally allows to start the JVM that is used to generated with a debugger. Setting it to `true` will cause
+  the started JVM to suspend until a debugger is attached. Useful for debugging classloading problems or exceptions during
+  the build.
+* `backendConfig` - optional configuration providing the backend. If not given, the `execute-generators` backend from
+  [mps-build-backends](https://github.com/mbeddr/mps-build-backends) will be used.
+* `environmentKind` - optional kind of environment (MPS or IDEA) to execute the generators in. IDEA environment is used
+  by default for backwards compatibility but MPS environment may be faster. See [MPS vs IDEA environment](../notes/mps-vs-idea-environment.md).
+* `maxHeap` (since 1.15) - maximum heap size setting for the JVM that executes the generator. The value is a string
+  understood by the JVM command line argument `-Xmx` e.g. `3G` or `512M`.

docs/plugins/modelcheck.md

@@ -0,0 +1,99 @@
+## `checkModels`
+
+Run the model check on a subset or all models in a project directly from gradle.
+
+This functionality currently runs all model checks (typesystem, structure, constrains, etc.) from Gradle. By default, if
+any of checks fails, the complete build is failed. All messages (Info, Warning or Error) are reported through log4j to
+the command line.
+
+### Usage
+
+A minimal build script to check all models in an MPS project with no external plugins would look like this:
+
+```
+apply plugin: 'modelcheck'
+
+configurations {
+    mps
+}
+
+dependencies {
+    mps "com.jetbrains:mps:$mpsVersion"
+}
+
+ext.mpsVersion = '2018.3.6'
+
+modelcheck {
+    projectLocation = new File("./mps-prj")
+    mpsConfig = configurations.mps
+    macros = [Macro("mypath", "/your/path")]
+}
+```
+
+Parameters:
+* `mpsConfig` - the configuration used to resolve MPS. Custom plugins are supported via the `pluginLocation` parameter.
+* `mpsLocation` - optional location where to place the MPS files if `mpsConfig` is specified, or where to take them from
+  otherwise.
+* `mpsVersion` - optionally overrides automated version detection from `mpsConfig`. Required if you use
+  a [custom distribution](../notes/custom-mps-distribution.md) of MPS.
+* `javaExec` - optional `java` executable to use.
+* `pluginLocation` - location where to load the plugins from. Structure needs to be a flat folder structure similar to the
+  `plugins` directory inside of the MPS installation.
+* `plugins` - deprecated, use `pluginsProperty`.
+* `pluginsProperty` - optional list of plugins to load before generation is attempted.
+  The notation is `new Plugin("pluginID", "somePath")`. The first parameter is the plugin id.
+  For the second parameter `"somePath"` there are several options:
+  * if it's an absolute path, the plugin is loaded from that path
+  * if it's a folder located under `pluginLocation` the plugin is loaded from that folder
+  * otherwise it should be a plugin folder located under the default `mps/plugins`
+* `models` - optional list of models to check. RegEx can be used for matching multiple models.
+* `excludeModels` - optional list of models to exclude from checking. RegEx can be used for matching multiple models.
+* `modules` - optional list of modules to check. Expects ordinary name (w/o virtual folders). RegEx can be used for matching multiple modules.
+  If both parameters, `models` and `modules`, are omitted - all models in the project will be checked, except as
+  excluded by `excludeModels` and `excludeModules`.
+* `excludeModules` - optional list of modules to exclude from checking. RegEx can be used for matching multiple modules.
+* `macros` - optional list of path macros. The notation is `new Macro("name", "value")`.
+* `projectLocation` - location of the MPS project to check.
+* `errorNoFail` - report errors but do not fail the build.
+* `warningAsError` - handles warnings as errors and will fail the build if any is found when `errorNoFail` is not set.
+* `debug` - optionally allows to start the JVM that is used to load MPS project with a debugger. Setting it to `true` will cause
+  the started JVM to suspend until a debugger is attached. Useful for debugging classloading problems or exceptions during
+  the build.
+* `junitFile` - allows storing the results of the model check as a JUnit XML file. By default, the file will contain one
+  testcase for each model that was checked (s. `junitFormat`).
+* `junitFormat` - allows to change the format of the JUnit XML file, how the model checking errors will be reported. Possible options:
+  * `model` (default) - generates one testcase for each model that was checked. If the model check reported any error for the model,
+    the testcase will fail and the message of the model checking error will be reported.
+  * `message` - generates one testcase for each model check error. For uniqueness reasons, the name of the testcase will reflect the specific
+    model check error and the name of the test class will be constructed from the checked node ID and its containing root node.
+    Full error message and the node URL will be reported in the testcase failure. Checked models will be mapped to test suites with this option.
+* `parallel` (since 1.20) - runs model checker in parallel mode. Supported in MPS 2021.3.4. Default is `false`.
+* `maxHeap` - maximum heap size setting for the JVM that executes the model checker. This is useful to limit the heap usage
+  in scenarios like containerized build agents where the OS reported memory limit is not the maximum
+  to be consumed by the container. The value is a string understood by the JVM command line argument `-Xmx` e.g. `3G` or `512M`.
+* `backendConfig` - optional configuration providing the backend. If not given, the `modelcheck` backend from
+  [mps-build-backends](https://github.com/mbeddr/mps-build-backends) will be used.
+* `environmentKind` - optional kind of environment (MPS or IDEA) to execute the generators in. IDEA environment is used
+  by default for backwards compatibility but MPS environment may be faster. See [MPS vs IDEA environment](../notes/mps-vs-idea-environment.md).
+
+### Additional Plugins
+
+By default, only the minimum required set of plugins is loaded. This includes base language and some utilities like the
+HTTP server from MPS. If your project requires additional plugins to be loaded this is done by setting plugin location
+to the place where your jar files are placed and adding your plugin id and folder name to the `plugins` list:
+
+```
+apply plugin: 'modelcheck'
+...
+
+modelcheck {
+    pluginLocation = new File("path/to/my/plugins")
+    pluginsProperty = [new Plugin("com.mbeddr.core", "mbeddr.core")]
+    projectLocation = new File("./mps-prj")
+    mpsConfig = configurations.mps
+}
+
+```
+
+Dependencies of the specified plugins are automatically loaded from the `pluginLocation` and the plugins directory of
+MPS. If they are not found the build will fail.

docs/tasks/BundleMacosJdk.md

@@ -0,0 +1,33 @@
+## BundleMacosJdk
+
+(Linux/macOS) Creates a .tar.gz by combining an RCP artifact and a JDK.
+This task is intended as a substitute for the macOS-specific CreateDmg
+task.
+
+### Usage
+
+```
+task bundleMacosJdk(type: de.itemis.mps.gradle.BundleMacosJdk) {
+    rcpArtifact file('path/to/RCP.tgz')
+
+    jdkDependency "com.jetbrains.jdk:jdk:${jdkVersion}:osx_x64@tgz"
+    // -or -
+    jdk file('path/to/jdk.tgz')
+
+    outputFile file('output.tar.gz')
+}
+```
+
+Parameters:
+* `rcpArtifact` - the path to the RCP artifact produced by a build script.
+* `jdkDependency` - the coordinates of a JDK in case it's available in
+  a repository and can be resolved as a Gradle dependency.
+* `jdk` - the path to a JDK .tgz file.
+* `outputFile` - the path and file name of the output gzipped tar archive.
+
+### Operation
+
+The task unpacks `rcpArtifact` into a temporary directory, unpacks
+the JDK given by `jdkDependency`/`jdk` under the `jre` subdirectory of
+the unpacked RCP artifact, fixes file permissions and creates missing
+symlinks. Finally, the file is repackaged again as tar/gzip.

docs/tasks/CreateDmg.md

@@ -0,0 +1,46 @@
+## CreateDmg
+
+(macOS only) Creates a .dmg installer by combining an RCP artifact (as
+created by an MPS-generated Ant script), a JDK, and a background image.
+
+### Usage
+
+```
+task buildDmg(type: de.itemis.mps.gradle.CreateDmg) {
+    rcpArtifact file('path/to/RCP.tgz')
+
+    jdkDependency "com.jetbrains.jdk:jdk:${jdkVersion}:osx_x64@tgz"
+    // -or -
+    jdk file('path/to/jdk.tgz')
+
+    backgroundImage file('path/to/background.png')
+    dmgFile file('output.dmg')
+
+    signKeyChain file("/path/to/my.keychain-db")
+
+    signKeyChainPassword "my.keychain-db-password"
+
+    signIdentity "my Application ID Name"
+}
+```
+
+Parameters:
+* `rcpArtifact` - the path to the RCP artifact produced by a build script.
+* `jdkDependency` - the coordinates of a JDK in case it's available in
+  a repository and can be resolved as a Gradle dependency.
+* `jdk` - the path to a JDK .tgz file.
+* `backgroundImage` - the path to the background image.
+* `dmgFile` - the path and file name of the output DMG image. Must end
+  with `.dmg`.
+* `signKeyChain (optional)` - the path and file name of the keychain which contains a code signing certificate.
+* `signKeyChainPassword (optional)` - the password which should be used to unlock the keychain.
+* `signIdentity (optional)` - the application ID of the code signing certificate.
+
+### Operation
+
+The task unpacks `rcpArtifact` into a temporary directory, unpacks
+the JDK given by `jdkDependency`/`jdk` under the `jre` subdirectory of
+the unpacked RCP artifact, fixes file permissions and creates missing
+symlinks. If the additional properties for code signing (`signKeyChain`, `signKeyChainPassword`, `signIdentity`) are defined,
+the application will be signed with the given certificate. Afterwards a DMG image is created and its layout is configured using the
+background image. Finally, the DMG is copied to `dmgFile`.

docs/tasks/GenerateLibrariesXml.md

@@ -0,0 +1,28 @@
+## GenerateLibrariesXml
+
+Generates a `.mps/libraries.xml` file using data from property files.
+
+### Usage
+
+```
+task generateLibrariesXml(type: de.itemis.mps.gradle.GenerateLibrariesXml) {
+    defaults rootProject.file('projectlibraries.properties')
+    overrides rootProject.file('projectlibraries.overrides.properties')
+    destination file('.mps/libraries.xml')
+}
+```
+
+Parameters:
+* `defaults` - path to default properties (checked in to version control)
+* `overrides` - path to property overrides (ignored, not checked in to
+  version control, absent by default)
+* `destination` - path to the output `libraries.xml`
+
+### Operation
+
+The task reads properties file `defaults`, then `overrides` (if
+present). `destination` is then generated based on the properties.
+
+Each property represents an entry in `destination` (a project library),
+where the property name is the library name and the property value is
+the path to the library.