diff --git a/README.md b/README.md index c66d30c1d3..cdcf1b7b76 100644 --- a/README.md +++ b/README.md @@ -1,77 +1,13 @@ -[![Dart CI](https://github.com/dart-lang/build/actions/workflows/dart.yml/badge.svg)](https://github.com/dart-lang/build/actions/workflows/dart.yml) +_Questions? Suggestions? Found a bug? Please +[file an issue](https://github.com/dart-lang/build/issues) or + [start a discussion](https://github.com/dart-lang/build/discussions)._ -These packages provide libraries for generating, compiling and serving Dart code. +Code generation for Flutter and Dart packages. -[Getting started with build_runner](https://github.com/dart-lang/build/blob/master/docs/getting_started.md) +See the +[build_runner package documentation](https://pub.dev/packages/build_runner) +for how to use `build_runner` builders in your package to add serialization, +data classes, data binding, dependency injection, mocking and more. -[General FAQ](https://github.com/dart-lang/build/blob/master/docs/faq.md) - -[Windows FAQ](https://github.com/dart-lang/build/blob/master/docs/windows_faq.md) - -[FAQ for Builder authors](https://github.com/dart-lang/build/blob/master/docs/builder_author_faq.md). - -[Additional Docs](https://github.com/dart-lang/build/blob/master/docs/) - -## [build](https://github.com/dart-lang/build/blob/master/build/README.md) - -Defines the interfaces for creating a `Builder` which is a way of doing codegen -that is compatible across build systems (pub, bazel, standalone runner). - -For packages doing code generation this should generally be the only package -against which there is a public dependency. Packages may have a dev_dependency on -one or more of the other packages. - -## [build_config](https://github.com/dart-lang/build/blob/master/build_config/README.md) - -Support for parsing `build.yaml` files. Used by `build_runner`. - -## [build_modules](https://github.com/dart-lang/build/blob/master/build_modules/README.md) - -Support for discovering the sub-modules within packages and creating summaries -of those modules. Used by `build_web_compilers` but should not be used directly -by most users. - -## [build_resolvers](https://github.com/dart-lang/build/blob/master/build_resolvers/README.md) - -An implementation of the `Resolver` interface to use the analyzer during build -steps. - -## [build_runner](https://github.com/dart-lang/build/blob/master/build_runner/README.md) - -Provides utilities to enact builds and a way to automatically run builds based -on configuration. - -This package should generally be a dev_dependency as it is used to run -standalone builds. The only exception would be wrapping the `build` and `watch` -methods with some other package. - -## [build_test](https://github.com/dart-lang/build/blob/master/build_test/README.md) - -Stub implementations for classes in `Build` and some utilities for running -instances of builds and checking their outputs. - -This package generally only be a dev_dependency as it introduces a dependency on -package:test. The exception to that would be if you were creating another -testing-only package that wraps this one. - -## [build_web_compilers](https://github.com/dart-lang/build/blob/master/build_web_compilers/README.md) - -Provides the `dart2js` and `dartdevc` support for your package. To use this -package you should add it as a dev_dependency. - -If you are using the automated build scripts, your project will automatically -start being compiled with dartdevc, and you can start developing with chrome -without any configuration. - - - -## Examples - -The [example](https://github.com/dart-lang/build/tree/master/example) -directory has an example of a build with custom builders which generate outputs -into both the source tree and a hidden generated directory. Try a build with -`dart run build_runner build -o web:deploy` to see what the output looks like. - -Most projects should not need custom builders. +See the [build package documentation](https://pub.dev/packages/build) to create +your own builder. \ No newline at end of file diff --git a/build/CHANGELOG.md b/build/CHANGELOG.md index 15114b77ee..4fa3b10c17 100644 --- a/build/CHANGELOG.md +++ b/build/CHANGELOG.md @@ -1,6 +1,7 @@ ## 3.0.2-wip - Use `build_runner_core` 9.3.0. +- Documentation revamp. ## 3.0.1 diff --git a/build/README.md b/build/README.md index d79b3f109f..9a7f062689 100644 --- a/build/README.md +++ b/build/README.md @@ -1,46 +1,50 @@ -Defines the basic pieces of how a build happens and how they interact. +_Questions? Suggestions? Found a bug? Please +[file an issue](https://github.com/dart-lang/build/issues) or + [start a discussion](https://github.com/dart-lang/build/discussions)._ -## [`Builder`][dartdoc:Builder] +Package for writing code generators, called _builders_, that run with +[build_runner](https://pub.dev/packages/build_runner). -The business logic for code generation. Most consumers of the `build` package -will create custom implementations of `Builder`. +- [See also: source_gen](#see-also-source_gen) +- [The `Builder` interface](#the-builder-interface) + - [Using the analyzer](#using-the-analyzer) +- [Examples](#examples) -## [`BuildStep`][dartdoc:BuildStep] +## See also: source_gen -The way a `Builder` interacts with the outside world. Defines the unit of work -and allows reading/writing files and resolving Dart source code. +The [source_gen](https://pub.dev/packages/source_gen) package provides helpers +for writing common types of builder, for example builders that are triggered by +a particular annotation. -## [`Resolver`][dartdoc:Resolver] class +Most builders should use `source_gen`, but it's still useful to learn about the +underlying `Builder` interface that `source_gen` plugs into. -An interface into the dart [analyzer][pub:analyzer] to allow resolution of code -that needs static analysis and/or code generation. +## The `Builder` interface -## Implementing your own Builders - -A `Builder` gets invoked one by one on it's inputs, and may read other files and -output new files based on those inputs. - -The basic API looks like this: +A builder implements the +[Builder](https://pub.dev/documentation/build/latest/build/Builder-class.html) +interface. ```dart abstract class Builder { - /// You can only output files that are configured here by suffix substitution. - /// You are not required to output all of these files, but no other builder - /// may declare the same outputs. + /// Declares inputs and outputs by extension. Map> get buildExtensions; - /// This is where you build and output files. + /// Builds all outputs for a single input. FutureOr build(BuildStep buildStep); } ``` -Here is an implementation of a `Builder` which just copies files to other files -with the same name, but an additional extension: +Its `buildExtensions` getter declares what inputs it runs on and what outputs +it might produce. -```dart -import 'package:build/build.dart'; +During a build its `build` method gets called once per matching input. + +It uses `buildStep` to read, analyze and write files. -/// A really simple [Builder], it just makes copies of .txt files! +For example, here is a builder that copies input `.txt` files to `.txt.copy` files: + +```dart class CopyBuilder implements Builder { @override final buildExtensions = const { @@ -49,106 +53,34 @@ class CopyBuilder implements Builder { @override Future build(BuildStep buildStep) async { - // Each `buildStep` has a single input. - var inputId = buildStep.inputId; + // Create the output ID from the build step input ID. + final inputId = buildStep.inputId; + final outputId = inputId.addExtension('.copy'); - // Create a new target `AssetId` based on the old one. - var copy = inputId.addExtension('.copy'); - var contents = await buildStep.readAsString(inputId); - - // Write out the new asset. - await buildStep.writeAsString(copy, contents); + // Read from the input, write to the output. + final contents = await buildStep.readAsString(inputId); + await buildStep.writeAsString(outputId, contents); } } ``` -It should be noted that you should _never_ touch the file system directly. Go -through the `buildStep#readAsString` and `buildStep#writeAsString` methods in -order to read and write assets. This is what enables the package to track all of -your dependencies and do incremental rebuilds. It is also what enables your -[`Builder`][dartdoc:Builder] to run on different environments. +Outputs are optional. For example, a builder that is activated by a particular +annotation will output nothing if it does not find that annotation. ### Using the analyzer -If you need to do analyzer resolution, you can use the `BuildStep#resolver` -object. This makes sure that all `Builder`s in the system share the same -analysis context, which greatly speeds up the overall system when multiple -`Builder`s are doing resolution. - -Here is an example of a `Builder` which uses the `resolve` method: - -```dart -import 'package:build/build.dart'; - -class ResolvingCopyBuilder implements Builder { - // Take a `.dart` file as input so that the Resolver has code to resolve - @override - final buildExtensions = const { - '.dart': ['.dart.copy'] - }; - - @override - Future build(BuildStep buildStep) async { - // Get the `LibraryElement` for the primary input. - var entryLib = await buildStep.inputLibrary; - // Resolves all libraries reachable from the primary input. - var resolver = buildStep.resolver; - // Get a `LibraryElement` for another asset. - var libFromAsset = await resolver.libraryFor( - AssetId.resolve(Uri.parse('some_import.dart'), - from: buildStep.inputId)); - // Or get a `LibraryElement` by name. - var libByName = await resolver.findLibraryByName('my.library'); - } -} -``` - -Once you have gotten a `LibraryElement` using one of the methods on `Resolver`, -you are now just using the regular `analyzer` package to explore your app. - -### Sharing expensive objects across build steps - -The build package includes a `Resource` class, which can give you an instance -of an expensive object that is guaranteed to be unique across builds, but may -be re-used by multiple build steps within a single build (to the extent that -the implementation allows). It also gives you a way of disposing of your -resource at the end of its lifecycle. - -The `Resource` constructor takes a single required argument which is a -factory function that returns a `FutureOr`. There is also a named argument -`dispose` which is called at the end of life for the resource, with the -instance that should be disposed. This returns a `FutureOr`. - -So a simple example `Resource` would look like this: - -```dart -final resource = Resource( - () => createMyExpensiveResource(), - dispose: (instance) async { - await instance.doSomeCleanup(); - }); -``` - -You can get an instance of the underlying resource by using the -`BuildStep#fetchResource` method, whose type signature looks like -`Future fetchResource(Resource)`. - -**Important Note**: It may be tempting to try and use a `Resource` instance to -cache information from previous build steps (or even assets), but this should -be avoided because it can break the soundness of the build, and may introduce -subtle bugs for incremental builds (remember the whole build doesn't run every -time!). The `build` package relies on the `BuildStep#canRead` and -`BuildStep#readAs*` methods to track build step dependencies, so sidestepping -those can and will break the dependency tracking, resulting in inconsistent and -stale assets. +The `buildStep` passed to `build` gives easy access to the analyzer for +processing Dart source code. -## Features and bugs +Use `buildStep.resolver.compilationUnitFor` to parse a single source file, or +`libraryFor` to fully resolve it. This can be used to introspect the full API +of the source code the builder is running on. For example, a builder can learn +enough about a class's field and field types to correctly serialize it. -Please file feature requests and bugs at the [issue tracker][tracker]. +## Examples -[tracker]: https://github.com/dart-lang/build/issues +There are some simple examples +[in the build repo](https://github.com/dart-lang/build/blob/master/example). -[dartdoc:Builder]: https://pub.dev/documentation/build/latest/build/Builder-class.html -[dartdoc:BuildStep]: https://pub.dev/documentation/build/latest/build/BuildStep-class.html -[dartdoc:Resolver]: https://pub.dev/documentation/build/latest/build/Resolver-class.html -[pub:analyzer]: https://pub.dev/packages/analyzer +For real examples of builders, see the list of popular builders in the +[build_runner package documentation](https://pub.dev/packages/build_runner). \ No newline at end of file diff --git a/build_config/CHANGELOG.md b/build_config/CHANGELOG.md index a4ae2a5696..9ce41d6ef2 100644 --- a/build_config/CHANGELOG.md +++ b/build_config/CHANGELOG.md @@ -5,6 +5,7 @@ for more information. - Bump the min sdk to 3.7.0. - Remove unused dep: `yaml`. +- Documentation revamp. ## 1.1.2 diff --git a/build_config/README.md b/build_config/README.md index e1d7441112..92f3475385 100644 --- a/build_config/README.md +++ b/build_config/README.md @@ -1,14 +1,32 @@ -# Customizing builds - -Customizing the build behavior of a package is done by creating a `build.yaml` -file, which describes your configuration. - -The full format is described in the +_Questions? Suggestions? Found a bug? Please +[file an issue](https://github.com/dart-lang/build/issues) or + [start a discussion](https://github.com/dart-lang/build/discussions)._ + +Configuration file format for +[build_runner](https://pub.dev/packages/build_runner) +builds. + +- [See also: build_yaml_format.md](#see-also-build_yaml_formatmd) +- [Dividing a package into build targets](#dividing-a-package-into-build-targets) +- [Configuring builders applied to your package](#configuring-builders-applied-to-your-package) +- [Configuring builders globally](#configuring-builders-globally) +- [Defining builders to apply to depenents](#defining-builders-to-apply-to-dependents) +- [Defining post process builders](#defining-post-process-builders) +- [Adjusting builder ordering](#adjusting-builder-ordering) +- [Triggers](#triggers) +- [Publishing build.yaml files](#publishing-buildyaml-files) +- [FAQ](#faq) + - [How do I avoid running builders on unnecessary inputs?](#how-do-i-avoid-running-builders-on-unnecessary-inputs) + - [How is the configuration for a builder resolved?](#how-is-the-configuration-for-a-builder-resolved) + - [How can I include additional sources in my build?](#how-can-i-include-additional-sources-in-my-build) + +## See also: build_yaml_format.md + +See also [docs/build_yaml_format.md](https://github.com/dart-lang/build/blob/master/docs/build_yaml_format.md) -file, while this documentation is more focused on specific usage scenarios of -the file. +for a more technical description of the format. -## Dividing a package into Build targets +## Dividing a package into build targets When a `Builder` should be applied to a subset of files in a package the package can be broken up into multiple 'targets'. Targets are configured in the @@ -35,7 +53,8 @@ Each target may also contain the following keys: (from the `pubspec.yaml`). - **builders**: Map, Optional. See "configuring builders" below. -## Configuring `Builder`s applied to your package +## Configuring builders applied to your package + Each target can specify a `builders` key which configures the builders which are applied to that target. The value is a Map from builder to configuration for that builder. The key is in the format `'$packageName:$builderName'`. The @@ -63,7 +82,7 @@ configuration may have the following keys: depending on the particular builder. The values in this map override Builder defaults or non mode-specific options when the build is done in release mode. -## Configuring `Builder`s globally +## Configuring builders globally Target level builder options can be overridden globally across all packages with the `global_options` section. These options are applied _after_ all Builder defaults and target level configuration, and _before_ `--define` command line @@ -83,7 +102,7 @@ arguments. depending on the particular builder. The values in this map override all other values per-key when the build is done in release mode. -## Defining `Builder`s to apply to dependents +## Defining builders to apply to dependents If users of your package need to apply some code generation to their package, then you can define `Builder`s and have those applied to packages with a @@ -158,7 +177,7 @@ builders: some_key: "Some value the users will want in release mode" ``` -## Defining `PostProcessBuilder`s +## Defining post process builders `PostProcessBuilder`s are configured similarly to normal `Builder`s, but they have some different/missing options. @@ -209,7 +228,7 @@ post_process_builders: [adjusting builder ordering]: #adjusting-builder-ordering -### Adjusting Builder Ordering +## Adjusting builder ordering Both `required_inputs` and `runs_before` can be used to tweak the order that Builders run in on a given target. These work by indicating a given builder is a @@ -307,7 +326,7 @@ triggers: - annotation NewAnnotation ``` -# Publishing `build.yaml` files +## Publishing `build.yaml` files `build.yaml` configuration should be published to pub with the package and checked in to source control. Whenever a package is published with a @@ -315,3 +334,99 @@ checked in to source control. Whenever a package is published with a the package consuming the config has a compatible version. Breaking version changes which do not impact the configuration file format will be clearly marked in the changelog. + +## FAQ + +### How do I avoid running builders on unnecessary inputs? + +You can skip unnecessary inputs and so speed up your build using the +`generate_for` option of the builder: + +``` +targets: + $default: + builders: + # Typically the builder key is just the package name, run + # `dart run build_runner doctor` to check your config. + : + generate_for: + # Example glob for only the Dart files under `lib/models` + - lib/models/*.dart +``` + +### How is the configuration for a builder resolved? + +Builders are constructed with a map of options which is resolved from the +builder specified defaults and user overrides. The configuration is specific to +a `target` and build mode. The configuration is "merged" one by one, where +the higher precedence configuration overrides values by String key. The order +of precedence from lowest to highest is: + +- Builder defaults without a mode. +- Builder defaults by mode. +- Target configuration without a mode. +- Target configuration by mode. +- Global options without a mode. +- Global options by mode. +- Options specified on the command line. + +For example: + +```yaml +builders: + some_builder: + # Some required fields omitted + defaults: + options: + some_option: "Priority 0" + release_options: + some_option: "Priority 1" + dev_options: + some_option: "Priority 1" +targets: + $default: + builders: + some_package:some_builder: + options: + some_option: "Priority 2" + release_options: + some_option: "Priority 3" + dev_options: + some_option: "Priority 3" + +global_options: + some_package:some_builder: + options: + some_option: "Priority 4" + release_options: + some_option: "Priority 5" + dev_options: + some_option: "Priority 5" +``` + +And when running the build: + +``` +dart run build_runner build --define=some_package:some_builder=some_option="Priority 6" +``` + +### How can I include additional sources in my build? + +The `build_runner` package defaults the included source files to directories +derived from the +[package layout conventions](https://dart.dev/tools/pub/package-layout). + +If you have additional files which you would like to be included as part of the +build, you can do that with the `sources` field on the `$default` target: + +```yaml +targets: + $default: + sources: + - my_custom_sources/** + - lib/** + - web/** + # Note that it is important to include these in the default target. + - pubspec.* + - $package$ +``` \ No newline at end of file diff --git a/build_runner/CHANGELOG.md b/build_runner/CHANGELOG.md index 60ba7aef8f..56e10e0460 100644 --- a/build_runner/CHANGELOG.md +++ b/build_runner/CHANGELOG.md @@ -7,6 +7,7 @@ for more information. - Remove interactive prompts for whether to delete files. - Ignore `-d` flag: always delete files as if `-d` was passed. +- Documentation revamp. ## 2.6.1 diff --git a/build_runner/README.md b/build_runner/README.md index 4bbe396666..b8c592170b 100644 --- a/build_runner/README.md +++ b/build_runner/README.md @@ -1,243 +1,205 @@ -

- Standalone generator and watcher for Dart using package:build. -
- - Issues related to build_runner - - - Pub Package Version - - - Latest Dartdocs - -

- -The `build_runner` package provides a concrete way of generating files using -Dart code. Files are always generated directly on disk, and -rebuilds are _incremental_ - inspired by tools such as [Bazel][]. - -> **NOTE**: Are you a user of this package? You may be interested in -> simplified user-facing documentation, such as our -> [getting started guide][getting-started-link] and [faq][faq-link]. - -[getting-started-link]: https://github.com/dart-lang/build/blob/master/docs/getting_started.md -[faq-link]: https://github.com/dart-lang/build/blob/master/docs/faq.md -[other-docs-link]: https://github.com/dart-lang/build/tree/master/docs - -* [Installation](#installation) -* [Usage](#usage) - * [Built-in commands](#built-in-commands) - * [Inputs](#inputs) - * [Outputs](#outputs) - * [Source control](#source-control) - * [Publishing packages](#publishing-packages) -* [Docs](#docs) -* [Contributing](#contributing) - * [Testing](#testing) - -## Installation - -This package is intended to support development of Dart projects with -[`package:build`][]. In general, add it to your [`pubspec.yaml`][pubspec] -as a [dev_dependencies][] by running the following command. - -```console -$ dart pub add dev:build_runner +_Questions? Suggestions? Found a bug? Please +[file an issue](https://github.com/dart-lang/build/issues) or + [start a discussion](https://github.com/dart-lang/build/discussions)._ + +Code generation for Dart and Flutter packages. + +- [Builders](#builders) +- [Getting started](#getting-started) + - [Install builders](#install-builders) + - [Build and watch](#build-and-watch) + - [Output files](#output-files) + - [Internal files](#internal-files) + - [Additional configuration](#additional-configuration) +- [Writing your own builder](#writing-your-own-builder) + +## Builders + +A `build_runner` code generator is called a _builder_. + +Usually, a builder adds some capability to your code that is inconvenient to +add and maintain in pure Dart. Examples include serialization, data classes, +data binding, dependency injection, and mocking. + +Here is a selection of the most-used builders on [pub.dev](https://pub.dev). +Except as noted, they are not owned or specifically endorsed by Google. + +|Builder|Adds capabilities|Notes| +|-|-|- +|[auto_route_generator](https://pub.dev/packages/auto_route)|Flutter navigation| +|[built_value_generator](https://pub.dev/packages/built_value)|data classes with JSON serialization|[Flutter Favourite](https://docs.flutter.dev/packages-and-plugins/favorites) by Google +|[chopper_generator](https://pub.dev/packages/chopper)|REST HTTP client|[Flutter Favourite](https://docs.flutter.dev/packages-and-plugins/favorites) +|[copy_with_extension_gen](https://pub.dev/packages/copy_with_extension_gen)|`copyWith` extension methods| +|[dart_mappable_builder](https://pub.dev/packages/dart_mappable)|data classes with JSON serialization| +|[drift_dev](https://pub.dev/packages/drift_dev)|reactive data binding and SQL| +|[envied_generator](https://pub.dev/packages/envied)|environment variable bindings| +|[freezed](https://pub.dev/packages/freezed)|data classes, tagged unions, nested classes, cloning|[Flutter Favourite](https://docs.flutter.dev/packages-and-plugins/favorites) +|[flutter_gen_runner](https://pub.dev/packages/flutter_gen_runner)|Flutter asset bindings| +|[go_router_builder](https://pub.dev/packages/go_router_builder)|Flutter navigation|by Google +|[hive_ce_generator](https://pub.dev/packages/hive_ce)|key-value database| +|[injectable_generator](https://pub.dev/packages/injectable_generator)|dependency injecton| +|[json_serializable](https://pub.dev/packages/json_serializable)|JSON serialization|[Flutter Favourite](https://docs.flutter.dev/packages-and-plugins/favorites) by Google +|[mockito](https://pub.dev/packages/mockito)|mocks and fakes for testing|by Google +|[retrofit_generator](https://pub.dev/packages/retrofit_generator)|REST HTTP client| +|[riverpod_generator](https://pub.dev/packages/riverpod)|reactive caching and data binding|[Flutter Favourite](https://docs.flutter.dev/packages-and-plugins/favorites) +|[slang_build_runner](https://pub.dev/packages/slang)|type-safe i18n| +|[swagger_dart_code_generator](https://pub.dev/packages/swagger_dart_code_generator)|dart types from Swagger/OpenAPI schemas| +|[theme_tailor](https://pub.dev/packages/theme_tailor)|Flutter themes and extensions| +|[webdev](https://pub.dev/packages/webdev)|compilation to javascript|by Google + +## Getting started + +### Install builders + +Find builders that look useful, perhaps via the list above, and follow their +"getting started" guides. + +The guides will take you through adding the necessary dependencies to your +package, then how to write code that activates the builder's capabilities. +Most builders are activated via an annotation that tells the builder to run +and what exactly it should do. + +For example, after following the `json_serializable` guide you will have these +dependencies in your `pubspec.yaml`: + +```yaml +dependencies: + json_annotation: ^4.9.0 + +dev_dependencies: + build_runner: ^2.6.0 + json_serializable: ^6.10.0 ``` -## Usage +and activate it with code like -When the packages providing `Builder`s are configured with a `build.yaml` file -they are designed to be consumed using an generated build script. Most builders -should need little or no configuration, see the documentation provided with the -Builder to decide whether the build needs to be customized. If it does you may -also provide a `build.yaml` with the configuration. See the -`package:build_config` README for more information on this file. +```dart +import 'package:json_annotation/json_annotation.dart'; -To have web code compiled to js add a `dev_dependency` on `build_web_compilers`. +// Include the file that the builder will generate. +part 'example.g.dart';. -## Docs +// Activate the builder. +@JsonSerializable() +class Person { + final String name; + final DateTime? dateOfBirth; -More comprehensive documentation is stored in this repository, under the -[docs][other-docs-link] directory. If you find something is undocumented, please -file an issue. + Person({required this.name, this.dateOfBirth}); -### Built-in Commands + // Wire up the generated `toJson` in `example.g.dart`. + Map toJson() => _$PersonToJson(this); -The `build_runner` package exposes a binary by the same name, which can be -invoked using `dart run build_runner `. - -The available commands are `build`, `watch`, `serve`, and `test`. - -- `build`: Runs a single build and exits. -- `watch`: Runs a persistent build server that watches the files system for - edits and does rebuilds as necessary. -- `serve`: Same as `watch`, but runs a development server as well. - - By default this serves the `web` and `test` directories, on port `8080` and - `8081` respectively. See below for how to configure this. -- `test`: Runs a single build, creates a merged output directory, and then runs - `dart run test --precompiled `. See below for instructions - on passing custom args to the test command. - -#### Command Line Options - -All the above commands support the following arguments: - -- `--help`: Print usage information for the command. -- `--[no-]fail-on-severe`: Whether to consider the build a failure on an error - logged. By default this is false. -- `--build-filter`: Build filters allow you to choose explicitly which files to - build instead of building entire directories. See further documentation on - this feature [here][partial_builds]. - -Some commands also have additional options: - -##### serve + // Wire up the generated `fromJson` in `example.g.dart`. + factory Person.fromJson(Map json) => _$PersonFromJson(json); +} +``` -- `--hostname`: The host to run the server on. -- `--live-reload`: Enables automatic page reloading on rebuilds. +—see [the json_serializable documentation](https://pub.dev/packages/json_serializable) for more detail. -Trailing args of the form `:` are supported to customize what -directories are served, and on what ports. +### Build and watch -For example to serve the `example` and `web` directories on ports 8000 and 8001 -you would do `dart run build_runner serve example:8000 web:8001`. +Once you have installed builders in your package, use the terminal to do a single build -##### test +```bash +cd +dart run build_runner build +``` -The test command will forward any arguments after an empty `--` arg to the -`dart run test` command. +or to launch "watch mode", which runs a build whenever your source code changes: -For example if you wanted to pass `-p chrome` you would do -`dart run build_runner test -- -p chrome`. +```bash +cd +dart run build_runner watch +``` -### Inputs +So, for example, in the `json_serializable` example, watch mode updates the +generated `toJson` and `fromJson` as you add or remove fields from the +`Person` class. + +```dart +@JsonSerializable() +class Person { + final String name; + final DateTime? dateOfBirth; + // Added. + final int age; + + // Updated manually. + Person({required this.name, this.dateOfBirth, this.age}); + + // No change needed, the generated implementations referenced get updated. + Map toJson() => _$PersonToJson(this); + factory Person.fromJson(Map json) => _$PersonFromJson(json); +} +``` -Valid inputs follow the general dart package rules. You can read any files under -the top level `lib` folder any package dependency, and you can read all files -from the current package. +If you have multiple packages then you need to run `build` or `watch` in each +package separately; there is an +[open feature request for workspace support](https://github.com/dart-lang/build/issues/3804). -In general it is best to be as specific as possible with your `InputSet`s, -because all matching files will be checked against a `Builder`'s -[`buildExtensions`][build_extensions] - see [outputs](#outputs) for more -information. +### Output files -### Outputs +Output is written directly to your package source, for example under `lib`. +This makes it immediately available to all tools including compilers and IDEs. -* You may output files anywhere in the current package. +You can choose whether or not to check generated files into source control. -> **NOTE**: When a `BuilderApplication` specifies `hideOutput: true` it may -> output under the `lib` folder of _any_ package you depend on. +If you publish your package, you must publish the generated files with it. +Users getting your package via `pub` cannot run the build step themselves. -* Builders are not allowed to overwrite existing files, only create new ones. -* Outputs from previous builds will not be treated as inputs to later ones. -* You may use a previous `BuilderApplications`'s outputs as an input to a later - action. +### Internal files -### Source control +`build_runner`uses a folder called `.dart_tool` in your package for internal files. +These are private to `build_runner` and should not be edited, checked in, +published or used in any other way. -This package creates a top level `.dart_tool` folder in your package, which -should not be submitted to your source control repository. You can see [our own -`.gitignore`](https://github.com/dart-lang/build/blob/master/.gitignore) as an -example. +So, tools such as `git` must be configured to ignore them. Make `git` ignore `.dart_tool` by adding to your `.gitignore` file: -```git -# Files generated by dart tools +```bash .dart_tool ``` -When it comes to _generated_ files it is generally best to not submit them to -source control, but a specific `Builder` may provide a recommendation otherwise. - -It should be noted that if you do submit generated files to your repo then when -you change branches or merge in changes you may get a warning on your next build -about declared outputs that already exist. This will be followed up with a -prompt to delete those files. You can type `l` to list the files, and then type -`y` to delete them if everything looks correct. If you think something is wrong -you can type `n` to abandon the build without taking any action. - -### Publishing packages - -In general generated files **should** be published with your package, but this -may not always be the case. Some `Builder`s may provide a recommendation for -this as well. - +### Additional configuration -## Legacy Usage +Builders can be further configured with a `build.yaml` file in your package's +root folder. -If the generated script does not do everything you need it's possible to -manually write one. With this approach every package which *uses* a -[`Builder`][builder] must have it's own script, they cannot be reused -from other packages. A package which defines a [`Builder`][builder] may have an -example you can reference, but a unique script must be written for the consuming -packages as well. You can reference the generated script at -`.dart_tool/build/entrypoint/build.dart` for an example. +For example, you can restrict which files in your package a builder runs for: -Your script should the [**`run`**][run_fn] functions defined in this library. - -### Configuring - -[`run`][run_fn] has a required parameter which is a `List`. -These correspond to the `BuilderDefinition` class from `package:build_config`. -See `apply` and `applyToRoot` to create instances of this class. These will be -translated into actions by crawling through dependencies. The order of this list -is important. Each Builder may read the generated outputs of any Builder that -ran on a package earlier in the dependency graph, but for the package it is -running on it may only read the generated outputs from Builders earlier in the -list of `BuilderApplication`s. - -**NOTE**: Any time you change your build script (or any of its dependencies), -the next build will be a full rebuild. This is because the system has no way -of knowing how that change may have affected the outputs. - -## Contributing - -We welcome a diverse set of contributions, including, but not limited to: - -* [Filing bugs and feature requests][file_an_issue] -* [Send a pull request][pull_request] -* Or, create something awesome using this API and share with us and others! - -For the stability of the API and existing users, consider opening an issue -first before implementing a large new feature or breaking an API. For smaller -changes (like documentation, minor bug fixes), just send a pull request. - -### Testing - -All pull requests are validated against CI, and must pass. The -`build_runner` package lives in a mono repository with other `build` packages, -and _all_ of the following checks must pass for _each_ package. - -Ensure code passes all our [analyzer checks][analysis_options]: - -```sh -$ dart analyze . +```yaml +targets: + $default: + builders: + json_serializable: + generate_for: + # Only run `json_serializable` on source under `lib/models`. + - lib/models/*.dart ``` -Ensure all code is formatted with the latest [dev-channel SDK][dev_sdk]. - -```sh -$ dart format . +Occasionally when using multiple builders you will need to specify which order +they run in. For full details on this and other options see the +[build_config documentation](https://pub.dev/packages/build_config). + +Some settings apply to a specific builder, for example `freezed`: + +```yaml +targets: + $default: + builders: + freezed: + options: + # Do format output. + format: true + # Don't generate `copyWith` or `operator==`. + copy_with: false + equal: false ``` -Run all of our unit tests: +—see each builder's documentation for details. -```sh -$ dart run test -``` +## Writing your own builder + +For advanced use cases it's possible to write your own builder. -[Bazel]: https://bazel.build/ -[`package:build`]: https://pub.dev/packages/build -[analysis_options]: https://github.com/dart-lang/build/blob/master/analysis_options.yaml - -[builder]: https://pub.dev/documentation/build/latest/build/Builder-class.html -[run_fn]: https://pub.dev/documentation/build_runner/latest/build_runner/run.html -[builder_application]: https://pub.dev/documentation/build_runner/latest/build_runner/BuilderApplication-class.html -[build_extensions]: https://pub.dev/documentation/build/latest/build/Builder/buildExtensions.html -[partial_builds]: https://github.com/dart-lang/build/blob/master/docs/partial_builds.md - -[dev_sdk]: https://dart.dev/get-dart -[dev_dependencies]: https://dart.dev/tools/pub/dependencies#dev-dependencies -[pubspec]: https://dart.dev/tools/pub/pubspec -[file_an_issue]: https://github.com/dart-lang/build/issues/new -[pull_request]: https://github.com/dart-lang/build/pulls +Get started with the [build package documentation](https://pub.dev/packages/build). diff --git a/build_test/CHANGELOG.md b/build_test/CHANGELOG.md index b49065a3d6..aa716a6e70 100644 --- a/build_test/CHANGELOG.md +++ b/build_test/CHANGELOG.md @@ -2,6 +2,7 @@ - Use `build` 3.0.2. - Use `build_runner` 2.7.0. +- Documentation revamp. ## 3.3.1 diff --git a/build_test/README.md b/build_test/README.md index 0576737a4e..8e1bce881d 100644 --- a/build_test/README.md +++ b/build_test/README.md @@ -1,87 +1,26 @@ -

- Testing utilities for users of package:build. -
- - Issues related to build_test - - - Pub Package Version - - - Latest Dartdocs - - - Join the chat on Gitter - -

+_Questions? Suggestions? Found a bug? Please +[file an issue](https://github.com/dart-lang/build/issues) or + [start a discussion](https://github.com/dart-lang/build/discussions)._ -## Installation +Test helpers for [build_runner](https://pub.dev/packages/build_runner) builders. -This package is intended to only be as a [development dependency][] for users -of [`package:build`][], and should not be used in any production code. Simply -add to your `pubspec.yaml`: +- [In-memory builds](#in-memory-builds) +- [In-memory builds with real sources](#in-memory-builds-with-real-sources) +- [Resolving source code](#resolving-source-code) -```yaml -dev_dependencies: - build_test: ^3.0.0 -``` - -## Running tests - -To run tests, you should go through the `dart run build_runner test` command. -This will compile all your tests to a temp directory and run them using -`dart run test`. If you would like to see the output directory, you can use the -`--output=` option to force the output to go to a specific place. - -### Forwarding additional args to `dart run test` - -It is very common to need to pass some arguments through to the eventual call -to `dart run test`. To do this, add all those args after an empty `--` arg. - -For example, to run all chrome platform tests you would do -`dart run build_runner test -- -p chrome`. - -## Debugging web tests - -This package will automatically create `*.debug.html` files next to all your -`*_test.dart` files, which can be loaded in a browser from the normal -development server (`dart run build_runner serve`). - -**Note:** In order to run the tests this way, you will need to configure them -to be compiled (by default we only compile `*.browser_test.dart` files). You -can do this in your build.yaml file, with something like the following: - -```yaml -targets: - $default: - builders: - build_web_compilers:entrypoint: - generate_for: - - test/**_test.dart - - web/**.dart -``` +### In-memory builds -You may also view an index of links to every `*.debug.html` file by navigating -to `http://localhost:8081` (or wherever your `test` folder is being served). +The [testBuilders][api:testBuilders] method provides a way to run builds in +memory for small, self contained tests. See the `test` folder in the `build` package for examples. -## Writing tests for your custom Builder +### In-memory builds with real sources -In addition to assiting in running normal tests, this package provides some -utilities for testing your custom `Builder` classes. +To pass sources on disk to `testBuilders`, create a +[TestReaderWriter][api:TestReaderWriter]. -_See the `test` folder in the `build` package for more examples_. - -### Run a `Builder` within a test environment - -Using [`testBuilder`][api:testBuilder], you can run a functional test of a -`Builder`, including feeding specific assets, and more. It automatically -creates an in-memory representation of various utility classes. - -### Exposing actual package sources to `testBuilder` - -To pass sources on disk to `testBuilder`, create a `TestReaderWriter`. You can -write individual sources to it from a `PackageAssetReader`, or write all sources -to it with `loadIsolateSources`: +You can write individual sources to it from a +[PackageAssetReader][api:PackageAssetReader], or write all sources to it with +`loadIsolateSources`: ```dart final readerWriter = TestReaderWriter(rootPackage: 'test_package'); @@ -98,12 +37,10 @@ class TestClass {} ); ``` -### Resolve source code for testing +### Resolving source code -Using [`resolveAsset`][api:resolveAsset] and -[`resolveSource`][api:resolveSource], you can resolve Dart source code into a -static element model, suitable for probing and using within tests of code you -might have written for a `Builder`: +Use [resolveAsset][api:resolveAsset] or +[resolveSource][api:resolveSource] to resolve code with the analyzer: ```dart test('should resolve a simple dart file', () async { @@ -117,19 +54,8 @@ test('should resolve a simple dart file', () async { }); ``` -### Various test implementations of classes - -* [`FakeWatcher`][api:FakeWatcher] -* [`TestReaderWriter`][api:TestReaderWriter] -* [`PackageAssetReader`][api:PackageAssetReader] - -[development dependency]: https://dart.dev/tools/pub/dependencies#dev-dependencies -[`package:build`]: https://pub.dev/packages/build - -[api:FakeWatcher]: https://pub.dev/documentation/build_test/latest/build_test/FakeWatcher-class.html -[api:TestReaderWriter]: https://pub.dev/documentation/build_test/latest/build_test/TestReaderWriter-class.html [api:PackageAssetReader]: https://pub.dev/documentation/build_test/latest/build_test/PackageAssetReader-class.html - [api:resolveAsset]: https://pub.dev/documentation/build_test/latest/build_test/resolveAsset.html [api:resolveSource]: https://pub.dev/documentation/build_test/latest/build_test/resolveSource.html -[api:testBuilder]: https://pub.dev/documentation/build_test/latest/build_test/testBuilder.html +[api:testBuilders]: https://pub.dev/documentation/build_test/latest/build_test/testBuilders.html +[api:TestReaderWriter]: https://pub.dev/documentation/build_test/latest/build_test/TestReaderWriter-class.html diff --git a/build_web_compilers/CHANGELOG.md b/build_web_compilers/CHANGELOG.md index 19e961bb71..85e4276991 100644 --- a/build_web_compilers/CHANGELOG.md +++ b/build_web_compilers/CHANGELOG.md @@ -1,3 +1,7 @@ +## 4.2.2-wip + +- Documentation revamp. + ## 4.2.1 - Remove unused deps: `build_config`. diff --git a/build_web_compilers/README.md b/build_web_compilers/README.md index d03d6ab246..ab5a7ec3aa 100644 --- a/build_web_compilers/README.md +++ b/build_web_compilers/README.md @@ -1,24 +1,12 @@ -

- Web compilers for users of package:build. -
- - Issues related to build_web_compilers - - - Pub Package Version - - - Latest Dartdocs - - - Join the chat on Gitter - -

- -* [Installation](#installation) -* [Usage](#usage) -* [Configuration](#configuration) -* [Manual Usage](#manual-usage) + _Questions? Suggestions? Found a bug? Please +[file an issue](https://github.com/dart-lang/build/issues) or + [start a discussion](https://github.com/dart-lang/build/discussions)._ + + Web compilers for [build_runner](https://pub.dev/packages/build_runner). + +- [Installation](#installation) +- [Usage](#usage) +- [Configuration](#configuration) ## Installation @@ -234,80 +222,6 @@ targets: - -DANOTHER_VAR=true ``` -### Legacy builder options - -Previous versions of `build_web_compilers` only supported a single enabled -compiler that would be enabled with the `compiler` option. -If you only want to use `dart2js` for all builds, you can use that option: - -```yaml -targets: - $default: - builders: - build_web_compilers:entrypoint: - # These are globs for the entrypoints you want to compile. - generate_for: - - test/**.browser_test.dart - - web/**.dart - options: - compiler: dart2js - # List any dart2js specific args here, or omit it. - dart2js_args: - - -O2 -``` - -Similarly, only compiling with `dart2wasm`: - -```yaml -targets: - $default: - builders: - build_web_compilers:entrypoint: - options: - compiler: dart2wasm - # List flags that should be forwarded to `dart compile wasm` - dart2wasm_args: - - -O2 -``` - -When no option is set, the `compiler` option is implicitly set to `dart2js` on -release builds and to `dartdevc` otherwise. -Note that the `compilers` option takes precedence over the `compiler` option -when set. - -## Manual Usage - -If you are using a custom build script, you will need to add the following -builder applications to what you already have, almost certainly at the end of -the list (unless you need to post-process the js files). - -```dart -[ - apply( - 'build_web_compilers:ddc', - [ - (_) => new ModuleBuilder(), - (_) => new UnlinkedSummaryBuilder(), - (_) => new LinkedSummaryBuilder(), - (_) => new DevCompilerBuilder() - ], - toAllPackages(), - // Recommended, but not required. This makes it so only modules that are - // imported by entrypoints get compiled. - isOptional: true, - hideOutput: true), - apply('build_web_compilers:entrypoint', - // You can also use `WebCompiler.Dart2Js`. If you don't care about - // dartdevc at all you may also omit the previous builder application - // entirely. - [(_) => new WebEntrypointBuilder(WebCompiler.DartDevc)], toRoot(), - hideOutput: true, - // These globs should match your entrypoints only. - defaultGenerateFor: const InputSet( - include: const ['web/**', 'test/**.browser_test.dart'])), -] -``` - [development dependency]: https://dart.dev/tools/pub/dependencies#dev-dependencies [Dart development compiler]: https://dart.dev/tools/dartdevc [`package:build`]: https://pub.dev/packages/build diff --git a/build_web_compilers/pubspec.yaml b/build_web_compilers/pubspec.yaml index a2958a7543..66171ea101 100644 --- a/build_web_compilers/pubspec.yaml +++ b/build_web_compilers/pubspec.yaml @@ -1,5 +1,5 @@ name: build_web_compilers -version: 4.2.1 +version: 4.2.2-wip description: Builder implementations wrapping the dart2js and DDC compilers. repository: https://github.com/dart-lang/build/tree/master/build_web_compilers resolution: workspace diff --git a/docs/builder_author_faq.md b/docs/builder_author_faq.md deleted file mode 100644 index 1de979c372..0000000000 --- a/docs/builder_author_faq.md +++ /dev/null @@ -1,115 +0,0 @@ -## When should a Builder build to `cache` vs `source`? - -Outputs to `source` will generally be published with the package on pub. **This -can be risky.** If the generated code depends on _any_ details from the current -pub solve - that is it reads information from dependencies - then a consumer of -the package which has different versions of dependencies in their pub solve -**may be broken**. If the generated code imports any libraries, including from -the package providing the builder, it must only use the API surface area which -is guaranteed to not break without a major version bump. - -Outputs to `cache` will never be published with the package, and if they are -required to compile or run, then the build system must be used by every consumer -of the code. **This is always safe**, but may place limitations on the end user. -Any outputs which are used _during_ a build to produce other outputs, but don't -need to be compiled or seen by the user, should also be built to `cache`. - -## How expensive is it to get a resolved library (ex: buildStep.inputLibrary) - -This can be a very expensive operation, depending on a number of factors. - -The first time this is done in the build process, all transitive imports of the -file have to be parsed and analyzed, and even the entire SDK may have to be -analyzed. - -We have some optimizations to make this cheaper for subsequent builders, but -they can be circumvented by certain package structures. You should assume the -cost will be at least on the order of the number of all transitive imports. - -The build step will also be invalidated if any transitive import of the resolved -library changes, even in the best case scenarios. - -You should only use a resolved library if you absolutely need on. If you can -use simply a parsed compilation unit instead, you should consider using that. - -## How can I have temporary outputs only used during the Build? - -Due to build restrictions - namely that a builder can't read the outputs -produced by other builders in the same phase - it's sometimes necessary to write -information to a temporary file in one phase, then read that in a subsequent -phase, but the intermediate result is not a useful output outside of the build. - -Use a `PostProcessBuilder` to "delete" files so they are not included in the -merged output directory or available through the development server. Note that -files are never deleted from disk, instead a "delete" by a `PostProcessBuilder` -acts a filter on what assets can be seen in the result of the build. This works -best if temporary assets have a unique extension. - -The `FileDeletingBuilder` from the `build` package is designed for this case and -only needs to be configured with the extensions it should remove. In some cases -the builder should only operate in release mode so the files can see be seen in -development mode - use the `isEnabled` argument to the constructor rather than -returning a different builder or passing a different set of extensions - if the -extensions change between modes it will invalidate the entire build. - -For example: - -```dart -// In lib/builder.dart -PostProcessBuilder temporaryFileCleanup(BuilderOptions options) => - FileDeletingBuilder(const ['.used_during_build'], - isEnabled: options.config['enabled'] as bool? ?? false); -Builder writesTemporary([_]) => ... -Builder readsTemporaryWritesPermanent([_]) => ... -``` - -```yaml -builders: - my_builder: - import: "package:my_package/builders.dart" - builder_factories: - - writesTemporary - - readsTemporaryWritesPermanent - build_extensions: - .dart: - - .used_during_build - - .output_for_real - auto_apply: dependents - applies_builders: - - my_package|temporary_file_cleanup -post_process_builders: - temporary_file_cleanup: - import: "package:my_package/builders.dart" - builder_factory: temporaryFileCleanup - defaults: - release_options: - enabled: true -``` - -## How can I debug my builder? - -After running a build, or by running the `generate-build-script` command, a -build script will be written to `.dart_tool/build/entrypoint/build.dart`. This -is a Dart VM application that can be run manually, including with debugging -enabled. See the [devtool docs][] or [IntelliJ debugging docs][] -for usage instructions. The build script takes the same arguments as `dart run -build_runner`, for example: - -`dart --observe --pause-isolates-on-start .dart_tool/build/entrypoint/build.dart -build` - -[devtool docs]:https://dart.dev/tools/dart-devtools -[IntelliJ debugging docs]:https://www.jetbrains.com/help/idea/dart.html#dart_run_debug_command_line_application - -## Why can't my builder resolve code output by another builder? - -A builder may only read a generated output if it is ordered _after_ the builder -that emitted it. Ordering can be adjusted using the `runs_before` or -`required_inputs` configuration options for builders. In particular this can be -tricky for the `SharedPartBuilder` from `package:source_gen`. When using this -utility it is the `source_gen:combining_builder` which emits the Dart code in a -`.g.dart` file, and this always runs after all builders emitting `.g.part` -files. No `SharedPartBuilder` will be able to resolve the Dart code emitted by -another `SharedPartBuilder`. In order for emitted code to be used with the -`Resolver` by later build steps, it must write to a `.dart` file directly, and -should be a different file extension than `.g.dart`. diff --git a/docs/faq.md b/docs/faq.md deleted file mode 100644 index d5bcccc014..0000000000 --- a/docs/faq.md +++ /dev/null @@ -1,454 +0,0 @@ -## How do I avoid running builders on unnecessary inputs? - -Slow builds are often the result of builders that run on all Dart files in your -package, and analyze them. In this case you can speed up your builds by telling -those builders exactly which files they should run on. - -You can do this in your `build.yaml` file, by configuring the `generate_for` -option of the builder: - -``` -targets: - $default: - builders: - # Typically the builder key is just the package name, run - # `dart run build_runner doctor` to check your config. - : - generate_for: - # Example glob for only the Dart files under `lib/models` - - lib/models/*.dart -``` - -## Where are the generated files? - -There are 3 places you might see files getting generated. On _every_ build files -might go to _cache_ (`.dart_tool/build/generated/*`), and _source_ (the working -tree of your package). These are determined by the `build_to` configuration of -the Builders you are using. When a Builder specifies `build_to: source` you will -see them next to your other files but should not edit them by hand, and they are -generally meant to be published with your package. If you see a warning about -"conflicting outputs" they refer to _source_ outputs, because when the build -tool is starting with no prior information it can't tell if it might be -overwriting some file you wrote by hand. - -Separately the `--output` option can specify a directory to create a _merged_ -view of all the files in the build system. This copies _hand written source_ -files, along with _source_ and _cache_ outputs and puts them all together in the -same directory structure. - -## How can I debug my release mode web app (dart2js)? - -By default, the `dart2js` compiler is only enabled in `--release` mode, which -does not include source maps or the original `.dart` files. If you need to debug -an error which only happens in `dart2js`, you will want to change your debug -mode compiler to `dart2js`. You can either do this using the `--define` command -line option: - -```text ---define "build_web_compilers:entrypoint=compiler=dart2js" -``` - -Or by editing your `build.yaml` file: - -```yaml -targets: - $default: - builders: - build_web_compilers:entrypoint: - options: - compiler: dart2js -``` - -## How can I build with multiple configurations? - -The build system supports two types of builds, "dev" and "release". By default -with `build_runner` the "dev" version is built by regardless of the command -used, build in release mode by passing the `--release` flag. With `webdev` the -default mode for the `serve` command is dev, and the default mode for the -`build` command is release. The `build` command can use dev mode with the -`--no-release` flag. - -Options can be configured per mode, and they are -[merged by key](#how-is-the-configuration-for-a-builder-resolved) with the -defaults provided by the builder and global overrides. The `options` field -defines configuration used in all modes, and the `dev` and `release` fields -defines the overrides to those defaults for the specific mode chosen. Builders -can define their own defaults by mode which is overridden by user config. For -example `build_web_compilers` defines options that use `dartdevc` compiler in -dev mode, and `dart2js` in release mode. - -The following configuration builds with `dart2js` always, passes `--no-minify` -in dev mode, and passed `-O3` in release mode: - -```yaml -targets: - $default: - builders: - build_web_compilers:entrypoint: - options: - compiler: dart2js - dev_options: - dart2js_args: - - --no-minify - release_options: - dart2js_args: - - -O3 -``` - -If you need other configurations in addition to dev and release, you can define -multiple `build.yaml` files. For instance if you have a `build.debug.yaml` file -you can build with `--config debug` and this file will be used instead of the -default `build.yaml`. The dev and release flavors still apply. `dart run -build_runner serve --config debug` will use the `dev_options` in -`build.debug.yaml`, while `dart run build_runner build --config debug --release` -will use the `release_options` in `build.debug.yaml`. - -Only one build flavor can be built at a time. It is not possible to have -multiple targets defined which set different builder options for the same set of -sources. Builds will overwrite generated files in the build cache, so flipping -between build configurations may be less performant than building the same build -configuration repeatedly. - -## How is the configuration for a builder resolved? - -Builders are constructed with a map of options which is resolved from the -builder specified defaults and user overrides. The configuration is specific to -a `target` and [build mode](#how-can-i-build-with-multiple-configurations). The -configuration is "merged" one by one, where the higher precedence configuration -overrides values by String key. The order of precedence from lowest to highest -is: - -- Builder defaults without a mode. -- Builder defaults by mode. -- Target configuration without a mode. -- Target configuration by mode. -- Global options without a mode. -- Global options by mode. -- Options specified on the command line. - -For example: - -```yaml -builders: - some_builder: - # Some required fields omitted - defaults: - options: - some_option: "Priority 0" - release_options: - some_option: "Priority 1" - dev_options: - some_option: "Priority 1" -targets: - $default: - builders: - some_package:some_builder: - options: - some_option: "Priority 2" - release_options: - some_option: "Priority 3" - dev_options: - some_option: "Priority 3" - -global_options: - some_package:some_builder: - options: - some_option: "Priority 4" - release_options: - some_option: "Priority 5" - dev_options: - some_option: "Priority 5" -``` - -And when running the build: - -``` -dart run build_runner build --define=some_package:some_builder=some_option="Priority 6" -``` - -## How can I include additional sources in my build? - -The `build_runner` package defaults the included source files to directories -derived from the -[package layout conventions](https://dart.dev/tools/pub/package-layout). - -If you have additional files which you would like to be included as part of the -build, you will need a custom `build.yaml` file. You will want to modify the -`sources` field on the `$default` target: - -```yaml -targets: - $default: - sources: - - my_custom_sources/** - - lib/** - - web/** - # Note that it is important to include these in the default target. - - pubspec.* - - $package$ -``` - -## Why do Builders need unique outputs? - -`build_runner` relies on determining a static build graph before starting a -build - it needs to know every file that may be written and which Builder would -write it. If multiple Builders are configured to (possibly) output the same file -you can: - -- Add a `generate_for` configuration for one or both Builders so they do not - both operate on the same primary input. -- Disable one of the Builders if it is unneeded. -- Contact the author of the Builder and ask that a more unique output - extension is chosen. -- Contact the author of the Builder and ask that a more unique input extension - is chosen, for example only generating for files that end in - `_something.dart` rather than all files that end in `.dart`. - -## How can I use my own development server to serve generated files? - -There are 2 options for using a different server during development: - -1. Run `build_runner serve web:` and proxy the requests to it from your - other server. This has the benefit of delaying requests while a build is - ongoing so you don't get an inconsistent set of assets. - -2. Run `build_runner watch --output web:build` and use the created `build/` - directory to serve files from. This will include a `build/packages` - directory that has these files in it. - -## How can I fix `AssetNotFoundException`s for swap files? - -Some editors create swap files during saves, and while build_runner uses some -heuristics to try and ignore these, it isn't perfect and we can't hardcode -knowledge about all editors. - -One option is to disable this feature in your editor. Another option is you can -explicitly ignore files with a given extension by configuring the `exclude` -option for your targets `sources` in `build.yaml`: - -```yaml -targets: - $default: - sources: - exclude: - # Example that excludes intellij's swap files - - **/*___jb_tmp___ -``` - -## Why are some logs "(cached)"? - -`build_runner` will only run actions that have changes in their inputs. When an -action fails, and a subsequent build has exactly the same inputs for that action -it will not be rerun - the previous error messages, however, will get reprinted -to avoid confusion if a build fails with no printed errors. To force the action -to run again make an edit to any file that is an input to that action, or throw -away all cached values with `dart run build_runner clean` before starting the -next build. - -## How can I resolve "Skipped compiling" warnings? - -These generally come up in the context of a multi-platform package (generally -due to a mixture of vm and web tests). It can also happen if you have imports -to the `lib/src` directory of any package, even your own. - -The errors tend to look like this: - -```text -[WARNING] build_web_compilers:entrypoint on example|test/my_test.dart: - -Skipping compiling example|test/my_test.dart for the web because some of its -transitive libraries have sdk dependencies that not supported on this platform: - -example|test/imports_dart_io.dart -``` - -### Multi-platform packages - -While we are smart enough not to attempt to compile your web tests for the vm, -it is slow for us to figure that out so we print this warning to encourage you -to set up the proper configuration so we will never even attempt to compile -these tests. - -You can set up this configuration in your `build.yaml` file, using the -`generate_for` option on the builder. It helps a lot if you separate your web -and vm tests into separate directories, but you don't have to. - -For example, your `build.yaml` might look like this: - -```yaml -targets: - $default: - builders: - build_web_compilers:entrypoint: - generate_for: - - test/multiplatform/**_test.dart - - test/web/**_test.dart - - web/**.dart -``` - -### Imports of "private" libraries under lib/src - -We group libraries together into "modules" to optimize compilation, and that -grouping is based on the public imports of the package, as well as things like -any file containing a main function gets its own module. - -This means any time you have a package import that starts with `src/` in the -path, you might end up actually pulling in a module which contains additional -libraries which were not depended on by the actual library you imported. - -This can be very confusing, because tools that search for transitive imports -will not find the libraries that the error is complaining about not being -supported on the current platform. - -The best solution to this problem, is to import only public libraries. If you -cannot do that, you can opt out of this module grouping, by using the `fine` -module strategy, but this will come with a compile cost. - -If the import is in your own package, you can add this to your build.yaml: - -```yaml -targets: - $default: - builders: - build_web_compilers:ddc_modules: - options: - strategy: fine - build_web_compilers:dart2js_modules: - options: - strategy: fine -``` - -If it is to another package, you will need to do this as either a global -override (which will likely have a large negative effect on build times): - -```yaml -global_options: - builders: - build_web_compilers:ddc_modules: - options: - strategy: fine - build_web_compilers:dart2js_modules: - options: - strategy: fine -``` - -Or, override the entire `build.yaml` file for the package you depend on. -This would look like the local package override, but would go in a -`.build.yaml` file. If that package has its own build.yaml file -already, you will need to copy over its contents before adding the additional -configuration. - -## Why can't I see a file I know exists? - -A file may not be served or be present in the output of a build because: - -- You may be looking for it in the wrong place. For example if a server for - the `web/` directory is running on port `8080` then the file at - `web/index.html` will be loaded from `localhost:8080/index.html`. -- It may have be excluded from the build entirely because it isn't present as - in the `sources` for any `target` in `build.yaml`. Only assets that are - present in the build (as either a source or a generated output from a - source) can be served. -- It may have been removed by a `PostProcessBuilder`. For example in release - modes, by default, the `build_web_compilers` package enables a - `dart_source_cleanup` builder that removes all `.dart` source files. - -## Configuring the number of compiler processes - -Some builders run multiple compiler processes in order to speed up compilation. - -The amount of parallelism per task can be configured using the environment -variable `BUILD_MAX_WORKERS_PER_TASK`. - -The "tasks" in this case refer to different types of compilation (ie: different -compilers). There are least 3 different compilers which may be a part of any -given build - `kernel`, `dartdevc`, and `dart2js`. - -## How can I reduce the amount of memory used by the build process? - -By default most files in a typical build are cached in memory, but this can -cause problems in memory constrained environments (such as CI systems). - -You can pass the `--low-resources-mode` to disable this file caching. - -We may add future optimizations for this mode as well, with the general -principle being making the tradeoff of worse build times for less resource -usage on the machine. - -See also [Configuring the number of compiler processes](#configuring-the-number-of-compiler-processes). - -## How can I setup my editor to work with `build.yaml` files? - -A schema for valid keys and values in build configuration files is available on -[SchemaStore.org](https://www.schemastore.org/dart-build.json), but it is not -assigned to any file names by default. You can configure your editor to use the -correct schema. - -### VS Code - -When using VS Code, you can use the [YAML extension] by RedHat to validate the -schema of build configuration files. -Add the following snippet to your `settings.json` to use the correct schema for -`build.yaml` files: - -```json -"yaml.schemas": { - "https://json.schemastore.org/dart-build": [ - "build.yaml", - "*.build.yaml", - "build.*.yaml" - ] -} -``` - -### IntelliJ-based editors (e.g. IDEA, Android Studio, Webstorm) - -JetBrains IDEs have builtin support for YAML schema verification, but the -correct schema for `build.yaml` files needs to be configured here too. -To do so, open Settings and select "Languages & Frameworks", "Schemas and -DTDs" and "JSON Schema Mappings". -Add a new configuration with `https://json.schemastore.org/dart-build.json` as -a schema URL. The configuration should look similar to this: - -![Schema configuration for IDEA based IDEs](images/idea_schema_config.png) - -[YAML extension]: https://marketplace.visualstudio.com/items?itemName=redhat.vscode-yaml - -## How can I adjust builder ordering? - -You can configure certain builders to run before other builders globally, using the -`global_options` configuration in your `build.yaml` file: - -```yaml -global_options: - some_package:some_builder: - runs_before: - - some_other_package:some_other_builder -``` - -## Unable to read asset, could not compute transitive deps - -If you see this error, it means that we tried to compute the transitive -dependencies of a library, but some of those dependencies did not (yet) exist. - -Typically this would be caused by having multiple `target` definitions in your -`build.yaml` file, but not setting up the proper `dependencies` between them. - -For example, lets say you have 2 targets "a" and "b". Target "b" does some code -generation, and target "a" imports some of those generate files. You will need -to add a dependency on the "b" target from the "a" target like so: - -```yaml -targets: - a: - sources: - - lib/a.dart - dependencies: - - :b - b: - sources: - - lib/b.dart // Assume some builder generates lib/b.g.dart in this target. -``` - -If you are still having problems, you can file an issue on this repo and ask for -help. diff --git a/docs/getting_started.md b/docs/getting_started.md deleted file mode 100644 index 9d3be163fd..0000000000 --- a/docs/getting_started.md +++ /dev/null @@ -1,116 +0,0 @@ -# Getting started with `build_runner` - -To use `build_runner`, you need a 2.x version of the Dart SDK. - -* [Automated installers](https://dart.dev/get-dart#install) -* [Direct downloads](https://dart.dev/get-dart/archive#dev-channel) - -If you have issues using `build_runner`, see the -[Troubleshooting section](#troubleshooting), below. - -- [Getting started with `build_runner`](#getting-started-with-build_runner) - - [Using `build_runner` as a development server](#using-build_runner-as-a-development-server) - - [Creating an output directory](#creating-an-output-directory) - - [Using other `build_runner` commands](#using-other-build_runner-commands) - - [Switching to dart2js](#switching-to-dart2js) - - [Troubleshooting](#troubleshooting) - - [Diagnosing build times](#diagnosing-build-times) - - [build_runner has no versions that match...](#build_runner-has-no-versions-that-match) - - [Too many open files](#too-many-open-files) - -## Using `build_runner` as a development server - -1. Edit your package's **pubspec.yaml** file, adding dev dependencies on - **build_runner** and **build_web_compilers**: - - ```sh - dart pub add dev:build_runner dev:build_web_compilers - ``` - -2. Get package dependencies: - - ```sh - dart pub get - ``` - -3. Start the server: - - ```sh - dart run build_runner serve - ``` - -While the `serve` command runs, every change you save triggers a rebuild. - -The first build is the slowest. After that, assets are cached on disk and -incremental builds are faster. - -## Creating an output directory - -Build with `--output ` to write files into a merged output -directory with file paths that match internally referenced URIs. This can be -used with the `build`, `watch`, and `serve` commands. This directory can also be -used with a different server if the `serve` command is insufficient. - -To output only part of the package, for example to output only the `web` -directory, use `--output web:`. - -## Using other `build_runner` commands - -In addition to **serve** you can use: - -- **build:** Runs a single build and exits. This is most useful if your build - also generates output to your source directory. With `--output ` - this also creates a merged output directory with all sources and generated - assets. - -- **watch:** Like `build` but reruns after file changes. With `--output - ` the merged output directory will be kept up to date with changes. - This can be used to keep the outputs updated for use with another - filed-based development server. - -- **test:** Creates an output directory and runs `dart run test` within it. - This command requires a dev dependency on `build_test`. - -## Switching to dart2js - -By default `build_web_compilers` uses dartdevc. To switch to dart2js, pass -`--release` to `dart run build_runner build` (or `serve`). Pass args to dart2js -by creating a `build.yaml` file. - -```yaml -targets: - $default: - builders: - build_web_compilers:entrypoint: - options: - dart2js_args: - - --minify - - --fast-startup -``` - -## Troubleshooting - - - -### Diagnosing build times - -See -. - -### build_runner has no versions that match... - -1. Make sure you're using a 2.x SDK. - - ```sh - dart --version - ``` - -2. Check the versions of the packages that your app depends on. They should all - be compatible with a 2.x SDK. - -### Too many open files - -If you see a FileSystemException, saying the directory listing failed due to too -many open files, you might need to increase the OS limits. - -For details, see . diff --git a/docs/images/example_build.png b/docs/images/example_build.png deleted file mode 100644 index 5149bead5d..0000000000 Binary files a/docs/images/example_build.png and /dev/null differ diff --git a/docs/images/idea_schema_config.png b/docs/images/idea_schema_config.png deleted file mode 100644 index 0e90648d94..0000000000 Binary files a/docs/images/idea_schema_config.png and /dev/null differ diff --git a/docs/measuring_performance.md b/docs/measuring_performance.md deleted file mode 100644 index 00e6dc4921..0000000000 --- a/docs/measuring_performance.md +++ /dev/null @@ -1,99 +0,0 @@ -# Measuring performance with `build` and `build_runner` - -You can enable performance tracking by passing the `--log-performance ` -option (which will save logs to disk) or the `--track-performance` flag. Both of -these options will allow you to view the result in `serve` mode by navigating to -`/$perf`. On that page you will see a timeline something like this: - -![example build](images/example_build.png) - -If you are using the `--log-performance ` option that will save the logs to -disk so that you can attach them to bug reports. - -Note that for larger builds it may take a while to load the timeline. - -## Understanding the performance timeline - -Each row in the timeline corresponds to a single "action" in the build, which -is a single `Builder` being applied to a single primary input file. - -In the left hand column is the name of the builder, followed by a colon, and -then the `AssetId` for the primary input. - -The time for each action is split into at 3 primary pieces: - -- **Setup**: Primarily this is time spent checking content hashes of inputs to - see if the action needs to be reran. - - __Note__: This may also involve lazily building assets that were - optional, so seeing a long time here is not unexpected. -- **Build**: Time actually spent inside the `build` method of the `Builder`. -- **Finalize**: Time spent updating the asset graph for all outputs, and some - other cleanup. - -If the builder uses a `Resolver`, you will also see a breakdown of time spent -getting the resolver. This will appear below the `build` time since it overlaps -with it. - -### Slices - -A "slice" is a piece of actual synchronous work, up to the granularity of the -[slices resolution](#slices-resolution) configuration, visualized as a -horizontal bar in a row in the graph. It does not include asynchronous work -(unless it is less than the slice resolution). - -If [show async slices](#show-async-slices) is disabled, the granularity is -infinite, so a slice is just the entire time spent in a stage, including all -async work. - -### Stage time, slice time, user time, and real time - -Within each action (row), you can hover over one of the active time slices, -which will show several metrics, described as follows: - -**Stage time**: The start and end time (relative to the start of the build) for -the stage of the time slice you are currently hovering over. - -**Slice time**: The start and end time (relative to the start of the build) for -the slice. - -**Slice duration**: The total time spent on the current slice. - -**User time**: The actual time spent synchronously performing actions related to -the current stage. This does not count time spent waiting for asynchronous tasks. -This is generally the most relevant time, because asynchronous reads of assets -as an example may require actually building those files. - -**Real time**: The total time between the start and end time of this stage. This -includes time waiting for asynchronous tasks. - -## Configuring the timeline - -### Hide skipped actions - -Many actions in a build are "optional", which means they only run if some other -action asks to read their output. This setting toggles whether those rows -should be hidden or not. - -### Show async slices - -Off by default for performance reasons, this shows the actual synchronous points -in time during which the action was running, not including the time it spent -waiting for other async tasks. - -This shows you visually how much time it spent waiting for other async tasks, -typically other build actions or a resolver. - -### Slices resolution - -For performance reasons, when show async slices is enabled, the actual slices -are grouped together if they occur within a certain amount of time from each -other. This setting configures that granularity, and is measured in -milliseconds. - -Larger numbers will give less accurate but faster loads, smaller numbers are more -accurate but will slow down the performance of the visualization. - -### Filter - -This filters rows from the visualization based on a regex of the value in the -first column. This allows you to filter based on builder name or primary input. diff --git a/docs/partial_builds.md b/docs/partial_builds.md deleted file mode 100644 index edc281e289..0000000000 --- a/docs/partial_builds.md +++ /dev/null @@ -1,81 +0,0 @@ -# Build Filters - -Build filters allow you to choose explicitly which files to build instead of -building entire directories or projects. - -A build filter is a combination of a package and a path, with glob syntax -supported for each. - -Whenever a build filter is provided, only required outputs matching one of the -build filters will be built, in addition to any required files for those outputs. - -## Command Line Usage - -Build filters are supplied using the `--build-filter` option, which accepts -relative paths under the root package as well as `package:` uris. - -Glob syntax is allowed in both package names and paths. - -**Example**: The following would build and serve the JS output for an -application, as well as copy over the required SDK resources for that app: - -``` -dart run build_runner serve \ - --build-filter="web/main.dart.js" \ - --build-filter="package:build_web_compilers/**/*.js" -``` - -## Common Use Cases - -**Note**: For all the listed use cases it is up to the user or tool the user is -using to request all the required files for the desired task. This package only -provides the core building blocks for these use cases. - -### Testing - -If you have a large number of tests but only want to run a single one you can -now build just that test instead of all tests under the `test` directory. - -This can greatly speed up iteration times in packages with lots of tests. - -**Example**: This will build a single web test and run it: - -``` -dart run build_runner test \ - --build-filter="test/my_test.dart.browser_test.dart.js" \ - --build-filter="package:build_web_compilers/**/*.js" \ - -- -p chrome test/my_test.dart -``` - -**Note**: If your test requires any other generated files (css, etc) you will -need to add additional filters. - -### Applications - -This feature works as expected with the `--output ` and the `serve` -command. This means you can create an output directory for a single -application in your package instead of all applications under the same -directory. - -The serve command also uses the build filters to restrict what files are -available, which means it ensures if something works in serve mode it will -also work when you create an output directory. - -**Example**: This will build a single web app and serve it: - -``` -dart run build_runner serve \ - --build-filter="web/my_app.dart.js" \ - --build-filter="package:build_web_compilers/**/*.js" -``` - -**Note**: Your app may rely on additional code generated files (such as css -files), which you will need to list as additional filters. - -### Build Daemon Usage - -The build daemon accepts build filters when registering a build target. If no -filters are supplied these default filters are used: - -- `/**` -- `package:*/**` diff --git a/docs/windows_faq.md b/docs/windows_faq.md deleted file mode 100644 index b7b31d8607..0000000000 --- a/docs/windows_faq.md +++ /dev/null @@ -1,34 +0,0 @@ -When running on windows there are a few quick tips that can help your file I/O -performance. - -## Exclude your package directory from Windows Defender - -You can exclude your package entirely from windows defender by following -[these instructions](https://support.microsoft.com/en-us/help/4028485/windows-10-add-an-exclusion-to-windows-defender-antivirus). - -This gives significant speed improvements (~2x in some light testing), but also -comes with some obvious security considerations. - -### Security Considerations - -By opting into this you are potentially opening yourself up to vulnerabilities. -Any `Builder` which is being applied to your package will no longer have its -outputs checked by Windows Defender. If you don't trust your (even transitive!) -dependencies then you should be wary of this option. - -Note that the original package dependency sources are not in your package -directory but instead in your pub cache, so they will still be processed by -windows defender. - -## Hide your output directories and the `.dart_tool` directory from your IDE - -It is common for IDEs to run file watchers on your entire package in order to -update the file tree view among other things. Explicitly telling them to ignore -the `.dart_tool` directory and any directories that you commonly use with `-o` -can give a decent performance improvement. - -In `VsCode` you can do this using the `files.exclude` option in your -preferences. - -In `Intellij` you can right click on a directory, select `Mark Directory as`, -and choose `Excluded`. diff --git a/docs/writing_a_builder.md b/docs/writing_a_builder.md deleted file mode 100644 index 877b744ba8..0000000000 --- a/docs/writing_a_builder.md +++ /dev/null @@ -1,141 +0,0 @@ -# Output Restrictions - -A `Builder` must have outputs that are known statically based on input paths and -file extensions. For example an input named `foo.dart` always results in an -output named `foo.something.dart`. Builders are not allowed to read the input in -order to determine outputs. The only outputs that can be produced are those -which share a file base name with an input. - -Having a predictable set of outputs allows: - -- Ability to reason about builds. We've found that arbitrarily complex builds - tend to become difficult to debug. -- Compatibility with a wide set of build systems. A Builder that follows these - restrictions can run within `build_runner`, or in a build system like - [bazel](https://bazel.build). - -## Configuring outputs - -Each `Builder` implements a property `buildExtensions` which is a `Map>` to configure what outputs are created for 1 or more input -extensions. - -Keys in `buildExtensions` match a suffix in the path of potential inputs. That -is, a builder will run when an input ends with its input extension. -Valid outputs are formed by replacing the matched suffix with values in that -map. For instance, `{'.dart': ['.g.dart']}` matches all files ending with -`.dart` and allows the builder to write a file with the same name but with a -`.g.dart` extension instead. -A primary input `some_library.dart` would match the `.dart` suffix and expect -an output `some_library.g.dart`. - -The input extensions (keys in the `buildExtensions` map) may also start with a -`^`. In this case, the input extension changes from a suffix match to an exact -path match. For example an input extension of `^pubspec.yaml` would only match -the root pubspec file, and no other nested pubspecs. - -If a `Builder` has an empty string key in `buildExtensions` then every input -will trigger a build step, and the expected output will have the extension -appended. For example with the configuration `{'': ['.foo', '.bar']}` all files -will be passed as a primary input, and each build step may produce two assets. -For the primary input `some_file.txt` the allowed outputs are -`some_file.txt.foo` and `some_file.txt.bar`. Note that you will also see the -synthetic inputs when using this approach (`$package$` etc). You can detect -synthetic inputs by doing a `buildStep.canRead(buildStep.inputId)` check, if -you can't read it then you know it is not a normal file. - -### Capture groups - -Builders can declare more complex inputs and outputs by using a capture group -in their build input. Capture groups can be used to write outputs in a -different directory than the primary input. -For instance, consider a builder that generates Dart code for -[Protocol Buffers][protobuf]. Let's assume that proto definitions are stored in -a top-level `proto/` folder, and that generated files should go to -`lib/src/proto/`. This cannot be expressed with simple build extensions that -may replace a suffix in the asset's path only. -Using `{'proto/{{}}.proto': ['lib/src/proto/{{}}.dart']}` as a build extension -lets the builder read files in `proto/` and emit Dart files in the desired -location. Here, the __`{{}}`__ is called a _capture group_. Capture groups have -the following noteworthy properties: - -- Capture groups match at least one character in the input path, but may match - arbitrarily many characters. -- When the input uses a capture group, every output must reference that capture - group as well. -- Capture groups match as many characters as possible. In the proto example, - the asset `proto/nested/proto/test.proto` would match with `{{}}` capturing - `nested/proto/test`. The shorter suffix match with `{{}}` capturing just - `test` is not considered. -- General rules about build extensions matching suffixes still apply. When - using capture groups, the suffix typically spans across multiple path - components which is what enables directory moves. - In the proto example, the input extension might match an entire file - `proto/services/auth.proto`. With `{{}}` bound to `services/auth`, the - expected output is `lib/src/proto/services/auth.dart`. -- Build extensions using capture groups can start with `^` to enforce matches - over the entire input (which is still technically a suffix). - In the example above, the builder would also run on - `lib/src/proto/test.proto` (outputting `lib/src/lib/src/proto/test.dart`). - If the builder had used `^proto/{{}}.proto` as an input, it would not have - run on strict suffix matches. - -#### Using multiple capture groups - -A builder may use multiple capture groups in an input. Groups must be given a -name to distinguish them. For instance, `{{foo}}` declares a capture group -named `foo`. Names may consist of alphanumeric characters only. When using -multiple capture groups, they must all have unique names. And once again, every -output must refer to every capture group used in the input. - -Multiple groups come in handy when a builder needs to distinguish an asset's -directory and its file name. Consider a builder running on `.dart` files and -emitting files in a subdirectory next to the input. The following diagram -highlights the desired structure: - -``` -lib/src/ -├── generated/ -│ ├── service.api.dart (generated) -│ └── service.impl.dart (generated) -└── service.dart -``` - -This structure cannot be expressed with a single capture group. One might use -`{{}}.dart` as an input, but as `{{}}` matches the whole path there's no way to -introduce a `generated/` in the middle of the output path. - -With two capture groups, `{{dir}}/{{file}}.dart` can be used as an input. As -input extensions match suffixes, `{{file}}.dart` matches Dart files and assigns -everything between the last slash and the `.dart` extension to a capture named -`file`. Finally, `{{dir}}` captures the directory of the input. -By using `{{dir}}/generated/{{file}}.api.dart` and -`{{dir}}/generated/{{file}}.impl.dart` as output extensions, the builder may -emit files in the desired directory. - -## Working around the restrictions - -### Builders which produce an unknown number of outputs - -Often codegen is used to build Dart code - in these cases we can usually -generate all of the code in a single file. Dart does not have restrictions like -Java does - we don't need a 1:1 mapping between files and classes. - -If it's not possible to generate the output into a set of files known ahead of -time it's possible to write them to a single archive. - -### Builders which are based on an unknown number of inputs - -In Barback it was possible to write an `AggregateTransformer`, but there is no -matching concept as `Builder` - outputs must be known statically and must share -a `basename` with their input. - -In most cases the expected file is known statically, so the package can have a -placeholder (with a different extension) where that file should be created. For -example: if a Builder is meant to produce the file `lib/foo.dart` and use as -inputs existing Dart files in `lib/`. - Write an empty file to -`lib/foo.placeholder` - Use the extension config `{'.placeholder': ['.dart']}` - -Ignore the `buildStep.inputId` and find the real inputs with -`buildStep.findAssets(new Glob('lib/*dart')` - -[protobuf]: https://developers.google.com/protocol-buffers diff --git a/docs/writing_an_aggregate_builder.md b/docs/writing_an_aggregate_builder.md deleted file mode 100644 index 52517ff27b..0000000000 --- a/docs/writing_an_aggregate_builder.md +++ /dev/null @@ -1,226 +0,0 @@ -Most examples (and use-cases) for builders and generation are around reading a -single file, and outputting another file as a result. For example, -`json_serializable` emits a `.g.dart` per a `.dart` to encode/decode -to JSON, and `sass_builder` emits a `.css` per a `.scss`. - -However, sometimes you want to output one or more files based on the inputs of -_many_ files, perhaps even all of them. We call this abstractly, an _aggregate -builder_, or a builder with many inputs and one (or less) outputs. - -> **WARNING**: This pattern could have negative effects on your development -> cycle and incremental build, as it invalidates frequently (if _any_ of the -> read files change). - -## Defining your `Builder` - -Like usual, you'll implement the `Builder` class from `package:build`. Lets -write a simple builder that writes a text file called `all_files.txt` to your -`lib/` folder, which contains a listing of all the files found in `lib/**`. - -Obviously this builder isn't too useful, it's just an example! - -```dart -import 'package:build/build.dart'; - -class ListAllFilesBuilder implements Builder { - // TODO: Implement. -} -``` - -Every `Builder` needs a method, `build` implemented, and a field or getter, -`buildExtensions`. While they work the same here as any normal builder, they are -slightly more involved. Lets look at `buildExtensions` first. - -Normally to write, "generate `{file}.g.dart` for `{file.dart}`", you'd write: - -```dart -Map> get buildExtensions { - return const { - '.dart': const ['.g.dart'], - }; -} -``` - -However, we only want a single output file in (this) aggregate builder. So, -instead we will build on a _synthetic_ input - a file that does not actually -exist on disk, but rather is used as an identifier for build extensions. We -currently support the following synthetic files for this purpose: - -* `lib/$lib$` -* `$package$` - -When choosing whether to use `$package$` or `lib/$lib$`, there are two primary -considerations. - -- _where_ do you want to output your files (which directory should they be - written to). - - If you want to output to directories other than `lib`, you should use - `$package$`. - - If you want to output files only under `lib`, then use `lib/$lib$`. -- _which_ packages will this builder run on (only the root package or any - package in the dependency tree). - - If want to run on any package other than the root, you _must_ use - `lib/$lib$` since only files under `lib` are accessible from - dependencies - even synthetic files. - -## Writing the `Builder` using a synthetic input - -Each of these synthetic inputs exist if the folder exists (and is available to -the build), but they cannot be read. So, for this example, lets write one based -on `lib/$lib$`, and say that we will always emit the file `lib/all_files.txt`. - -Since out files are declared by simply replacing the declared input extension -with the declared output extensions, we can use `$lib$` as the input extension, -and `all_files.txt` as the output extension, which will declare an output at -`lib/all_files.txt`. - -**Note:** If using `$package$` as an input extension you need to declare the -full output path from the root of the package, since it lives at the root of the -package. - -```dart -import 'package:build/build.dart'; - -class ListAllFilesBuilder implements Builder { - @override - Map> get buildExtensions { - return const { - // Using r'...' is a "raw" string, so we don't interpret $lib$ as a field. - // An alternative is escaping manually, or '\$lib\$'. - r'$lib$': const ['all_files.txt'], - }; - } -} -``` - -Great! Now, to write the `build` method. Normally for a `build` method you'd -read an input, and write based on that. Again, aggregate builders work a little -differently, there is no "input" (you need to find inputs manually): - -```dart -import 'package:build/build.dart'; - -class ListAllFilesBuilder implements Builder { - @override - Future build(BuildStep buildStep) async { - // Will throw for aggregate builders, because '$lib$' isn't a real input! - buildStep.readAsString(buildStep.inputId); - } -} -``` - -Instead, we can use the `findAssets` API to find the inputs we want to process, -and create a new `AssetId` based off the current package we are processing. - -```dart -import 'package:build/build.dart'; -import 'package:glob/glob.dart'; -import 'package:path/path.dart' as p; - -class ListAllFilesBuilder implements Builder { - static final _allFilesInLib = new Glob('lib/**'); - - static AssetId _allFileOutput(BuildStep buildStep) { - return AssetId( - buildStep.inputId.package, - p.join('lib', 'all_files.txt'), - ); - } - - @override - Map> get buildExtensions { - return const { - r'$lib$': ['all_files.txt'], - }; - } - - @override - Future build(BuildStep buildStep) async { - final files = []; - await for (final input in buildStep.findAssets(_allFilesInLib)) { - files.add(input.path); - } - final output = _allFileOutput(buildStep); - return buildStep.writeAsString(output, files.join('\n')); - } -} -``` - -## Using a `Resolver` - -Since the input of aggregate builders isn't a real asset that could be read, we -also can't use `buildStep.inputLibrary` to resolve it. However some methods, -such as `libraryFor`, allow resolving any asset the builder can read. - -For instance, we could adapt the `ListAllFilesBuilder` from before to instead -list the names of all classes defined in `lib/`: - -```dart -import 'package:build/build.dart'; -import 'package:glob/glob.dart'; -import 'package:source_gen/source_gen.dart'; -import 'package:path/path.dart' as p; - -class ListAllClassesBuilder implements Builder { - @override - Map> get buildExtensions { - return const {r'$lib$': ['all_classes.txt']}; - } - - static AssetId _allFileOutput(BuildStep buildStep) { - return AssetId( - buildStep.inputId.package, - p.join('lib', 'all_classes.txt'), - ); - } - - @override - Future build(BuildStep buildStep) async { - final classNames = []; - - await for (final input in buildStep.findAssets(Glob('lib/**'))) { - final library = await buildStep.resolver.libraryFor(input); - final classesInLibrary = LibraryReader(library).classes; - - classNames.addAll(classesInLibrary.map((c) => c.name)); - } - - await buildStep.writeAsString( - _allFileOutput(buildStep), classNames.join('\n')); - } -} -``` - -As the resolver has no single entry point in aggregate builders, be aware that -[`findLibraryByName`][findLibraryByName] and [`libraries`][libraries] can only -find libraries that have been discovered through `libraryFor` or `isLibrary`. - -### Improving invalidation - -If the builder uses a `Resolver` the output will be invalidated, and the builder -will be rerun, any time there is a change in any resolved library _or any of -it's transitive imports_. If the builder output only depends on limited -information from the resolved libraries, it may be possible to invalidate the -output only when a library changes in a way that is meaningful to the builder. - -Split the process across two builders: - -1. A `Builder` with `buildExtensions` of `{'.dart': ['.some_name.info']}`. Use - the `Resolver` to find the information about the code that will be necessary - later. Serialize this to json or similar and write it as an intermediate - file. This should always be `build_to: cache`. -2. A `Builder` with `buildExtensiosn` of `{r'$lib$': ['final_output_name']}`. - Use the glob APIs to read and deserialize the outputs from the previous - step, then generate the final content. - -Each of these steps must be a separate `Builder` instance in Dart code. They can -be in the same builder definition in `build.yaml` only if they are both output -to cache. If the final result should be built to source they must be separate -builder definitions. In the single builder definition case ordering is handled -by the order of the `builder_factories` in the config. In the separate builder -definition case ordering should be handled by configuring the second step to -have a `required_inputs: ['.some_name.info']` based on the build extensions of -the first step. - -[findLibraryByName]: https://pub.dev/documentation/build/latest/build/Resolver/findLibraryByName.html -[libraries]: https://pub.dev/documentation/build/latest/build/Resolver/libraries.html