From f482d46dda150952fd67c208b97611d7c403a0ed Mon Sep 17 00:00:00 2001 From: Googler Date: Tue, 12 Dec 2023 10:09:15 -0800 Subject: [PATCH] New docs for labels, repos, etc * New entries in glossary.md * Clarify repo names in labels.md * Docs for REPO.bazel, `use_repo_rule` Fixes https://github.com/bazelbuild/bazel/issues/15821. PiperOrigin-RevId: 590246296 Change-Id: Idba4c35fdb068f9befba1122e3724880ba8fd1c8 --- site/en/concepts/labels.md | 43 +++++-- site/en/external/extension.md | 4 +- site/en/external/migration.md | 25 +++- site/en/external/module.md | 15 +++ site/en/external/overview.md | 33 ++++- site/en/reference/glossary.md | 119 +++++++++++++----- .../build/docgen/templates/be/functions.vm | 4 + 7 files changed, 193 insertions(+), 50 deletions(-) diff --git a/site/en/concepts/labels.md b/site/en/concepts/labels.md index a7a28b9b0af96b..adaf07f96b2e4e 100644 --- a/site/en/concepts/labels.md +++ b/site/en/concepts/labels.md @@ -5,18 +5,35 @@ Book: /_book.yaml {% include "_buttons.html" %} -All targets belong to exactly one package. The name of a target is -called its _label_. Every label uniquely identifies a target. A -typical label in canonical form looks like: +A **label** is an identifier for a target. A typical label in its full canonical +form looks like: + +```none +@@myrepo//my/app/main:app_binary +``` + +The first part of the label is the repository name, `@@myrepo`. The double-`@` +syntax signifies that this is a [*canonical* repo +name](/external/overview#canonical-repo-name), which is unique within +the workspace. Labels with canonical repo names unambiguously identify a target +no matter which context they appear in. + +Often the canonical repo name is an arcane string that looks like +`@@rules_java~7.1.0~toolchains~local_jdk`. What is much more commonly seen is +labels with an [*apparent* repo name](/external/overview#apparent-repo-name), +which looks like: ``` @myrepo//my/app/main:app_binary ``` -The first part of the label is the repository name, `@myrepo//`. +The only difference is the repo name being prefixed with one `@` instead of two. +This refers to a repo with the apparent name `myrepo`, which could be different +based on the context this label appears in. + In the typical case that a label refers to the same repository from which -it is used, the repository identifier may be abbreviated as `//`. -So, inside `@myrepo` this label is usually written as +it is used, the repo name part may be omitted. So, inside `@@myrepo` the first +label is usually written as ``` //my/app/main:app_binary @@ -26,9 +43,9 @@ The second part of the label is the un-qualified package name `my/app/main`, the path to the package relative to the repository root. Together, the repository name and the un-qualified package name form the fully-qualified package name -`@myrepo//my/app/main`. When the label refers to the same +`@@myrepo//my/app/main`. When the label refers to the same package it is used in, the package name (and optionally, the colon) -may be omitted. So, inside `@myrepo//my/app/main`, +may be omitted. So, inside `@@myrepo//my/app/main`, this label may be written either of the following ways: ``` @@ -56,14 +73,14 @@ this file is in the `my/app/main/testdata` subdirectory of the repository: //my/app/main:testdata/input.txt ``` -Strings like `//my/app` and `@some_repo//my/app` have two meanings depending on +Strings like `//my/app` and `@@some_repo//my/app` have two meanings depending on the context in which they are used: when Bazel expects a label, they mean -`//my/app:app` and `@some_repo//my/app:app`, respectively. But, when Bazel +`//my/app:app` and `@@some_repo//my/app:app`, respectively. But, when Bazel expects a package (e.g. in `package_group` specifications), they reference the package that contains that label. A common mistake in `BUILD` files is using `//my/app` to refer to a package, or -to *all* the targets in a package--it does not. Remember, it is +to *all* targets in a package--it does not. Remember, it is equivalent to `//my/app:app`, so it names the `app` target in the `my/app` package of the current repository. @@ -89,9 +106,9 @@ are two ways (one wrong, one correct) to refer to this file within -Labels starting with `@//` are references to the main +Labels starting with `@@//` are references to the main repository, which will still work even from external repositories. -Therefore `@//a/b/c` is different from +Therefore `@@//a/b/c` is different from `//a/b/c` when referenced from an external repository. The former refers back to the main repository, while the latter looks for `//a/b/c` in the external repository itself. diff --git a/site/en/external/extension.md b/site/en/external/extension.md index 7c1fb8a4dd1539..6c6aec83ebca98 100644 --- a/site/en/external/extension.md +++ b/site/en/external/extension.md @@ -179,7 +179,7 @@ incompatible change for your users. ### Specify the operating system and architecture If your extension relies on the operating system or its architecture type, -ensure to indicate this in the extension definition using the "os_dependent" -and "arch_dependent" boolean attributes. This ensures that Bazel recognizes the +ensure to indicate this in the extension definition using the `os_dependent` +and `arch_dependent` boolean attributes. This ensures that Bazel recognizes the need for re-evaluation if there are changes to either of them. diff --git a/site/en/external/migration.md b/site/en/external/migration.md index c4eb0c4050dbab..d860e1f894cd97 100644 --- a/site/en/external/migration.md +++ b/site/en/external/migration.md @@ -152,7 +152,9 @@ repository. ### Fetch external dependencies with module extensions{:#fetch-deps-module-extensions} If your dependency is not a Bazel project or not yet available in any Bazel -registry, you can introduce it using [module extensions](/external/extension). +registry, you can introduce it using +[`use_repo_rule`](/external/module#use_repo_rule) or [module +extensions](/external/extension). * **WORKSPACE** @@ -172,9 +174,24 @@ registry, you can introduce it using [module extensions](/external/extension). * **Bzlmod** - With Bzlmod, you have to move the definition into a `.bzl` file, which also - lets you share the definition between WORKSPACE and Bzlmod during the - migration period. + With Bzlmod, you can use the `use_repo_rule` directive in your MODULE.bazel + file to directly instantiate repos: + + ```python + ## MODULE.bazel + http_file = use_repo_rule("@bazel_tools//tools/build_defs/repo:http.bzl", "http_file") + http_file( + name = "data_file", + url = "http://example.com/file", + sha256 = "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + ) + ``` + + Under the hood, this is implemented using a module extension. If you need to + perform more complex logic than simply invoking a repo rule, you could also + implement a module extension yourself. You'll need to move the definition + into a `.bzl` file, which also lets you share the definition between + WORKSPACE and Bzlmod during the migration period. ```python ## repositories.bzl diff --git a/site/en/external/module.md b/site/en/external/module.md index b4210e1e007942..5baeecc0843e74 100644 --- a/site/en/external/module.md +++ b/site/en/external/module.md @@ -173,6 +173,21 @@ Bazel supports the following non-registry overrides: * [`git_override`](/rules/lib/globals/module#git_override) * [`local_path_override`](/rules/lib/globals/module#local_path_override) +## Define repos that don't represent Bazel modules {:#use_repo_rule} + +With `bazel_dep`, you can define repos that represent other Bazel modules. +Sometimes there is a need to define a repo that does _not_ represent a Bazel +module; for example, one that contains a plain JSON file to be read as data. + +In this case, you could use the [`use_repo_rule` +directive](/rules/lib/globals/module#use_repo_rule) to directly define a repo +by invoking a repo rule. This repo will only be visible to the module it's +defined in. + +Under the hood, this is implemented using the same mechanism as [module +extensions](/external/extension), which lets you define repos with more +flexibility. + ## Repository names and strict deps The [canonical name](/external/overview#canonical-repo-name) of a repo backing a diff --git a/site/en/external/overview.md b/site/en/external/overview.md index 97714878a0f46e..bdda1cf5f9bddc 100644 --- a/site/en/external/overview.md +++ b/site/en/external/overview.md @@ -25,8 +25,14 @@ in Bazel, before going into a bit more detail about the two systems in order. ### Repository {:#repository} -A directory with a `WORKSPACE` or `WORKSPACE.bazel` file, containing source -files to be used in a Bazel build. Often shortened to just **repo**. +A directory tree with a boundary marker file at its root, containing source +files that can be used in a Bazel build. Often shortened to just **repo**. + +A repo boundary marker file can be `MODULE.bazel` (signaling that this repo +represents a Bazel module), `REPO.bazel` (see [below](#repo.bazel)), or in +legacy contexts, `WORKSPACE` or `WORKSPACE.bazel`. Any repo boundary marker file +will signify the boundary of a repo; multiple such files can coexist in a +directory. ### Main repository {:#main-repository} @@ -100,6 +106,29 @@ canonical name `canonical_name`: ls $(bazel info output_base)/external/{{ '' }} canonical_name {{ '' }} ``` +### REPO.bazel file {:#repo.bazel} + +The `REPO.bazel` file is used to mark the topmost boundary of the directory tree +that constitutes a repo. It doesn't need to contain anything to serve as a repo +boundary file; however, it can also be used to specify some common attributes +for all build targets inside the repo. + +The syntax of a `REPO.bazel` file is similar to `BUILD` files, except that no +`load` statements are supported, and only a single function, `repo()`, is +available. `repo()` takes the same arguments as the [`package()` +function](/reference/be/functions#package) in `BUILD` files; whereas `package()` +specifies common attributes for all build targets inside the package, `repo()` +analogously does so for all build targets inside the repo. + +For example, you can specify a common license for all targets in your repo by +having the following `REPO.bazel` file: + +```python +repo( + default_package_metadata = ["//:my_license"], +) +``` + ## Manage external dependencies with Bzlmod {:#bzlmod} Bzlmod, the new external dependency subsystem, does not directly work with repo diff --git a/site/en/reference/glossary.md b/site/en/reference/glossary.md index a63bd5176d3ca1..09d31265506406 100644 --- a/site/en/reference/glossary.md +++ b/site/en/reference/glossary.md @@ -232,6 +232,9 @@ reference to `//:bar`. `//:foo` has an *action dependency* on `//:bar` if an action in `//:foo` depends on an input [artifact](#artifact) created by an action in `//:bar`. +In certain contexts, it could also refer to an _external dependency_; see +[modules](#module). + ### Depset {:#depset} A data structure for collecting data on transitive dependencies. Optimized so @@ -308,28 +311,30 @@ build. ### Label {:#label} -An identifier for a [target](#target). A fully-qualified label such as -`//path/to/package:target` consists of `//` to mark the workspace root -directory, `path/to/package` as the directory that contains the [`BUILD` -file](#build-file) declaring the target, and `:target` as the name of the target -declared in the aforementioned `BUILD` file. May also be prefixed with -`@my_repository//<..>` to indicate that the target is declared in an [external -repository](/docs/external) named `my_repository`. +An identifier for a [target](#target). Generally has the form +`@repo//path/to/package:target`, where `repo` is the (apparent) name of the +[repository](#repository) containing the target, `path/to/package` is the path +to the directory that contains the [`BUILD` file](#build-file) declaring the +target (this directory is also known as the [package](#package)), and `target` +is the name of the target itself. Depending on the situation, parts of this +syntax may be omitted. + +**See also**: [Labels](/concepts/labels) ### Loading phase {:#loading-phase} -The first phase of a build where Bazel parses `WORKSPACE`, `BUILD`, and [`.bzl` -files](#bzl-file) to create [packages](#package). [Macros](#macro) and certain -functions like `glob()` are evaluated in this phase. Interleaved with the second -phase of the build, the [analysis phase](#analysis-phase), to build up a [target +The first phase of a build where Bazel executes [`BUILD` files](#build-file) to +create [packages](#package). [Macros](#macro) and certain functions like +`glob()` are evaluated in this phase. Interleaved with the second phase of the +build, the [analysis phase](#analysis-phase), to build up a [target graph](#target-graph). ### Macro {:#macro} A mechanism to compose multiple [rule](#rule) target declarations together under a single [Starlark](#starlark) function. Enables reusing common rule declaration -patterns across `BUILD` files. Expanded to the underlying rule target declarations -during the [loading phase](#loading-phase). +patterns across `BUILD` files. Expanded to the underlying rule target +declarations during the [loading phase](#loading-phase). **See also:** [Macro documentation](/extending/macros) @@ -341,6 +346,32 @@ identifiers for *spawn strategy* selections. Some examples of action mnemonics are `Javac` from Java rules, `CppCompile` from C++ rules, and `AndroidManifestMerger` from Android rules. +### Module {:#module} + +A Bazel project that can have multiple versions, each of which can have +dependencies on other modules. This is analogous to familiar concepts in other +dependency management systems, such as a Maven _artifact_, an npm _package_, a +Go _module_, or a Cargo _crate_. Modules form the backbone of Bazel's external +dependency management system. + +Each module is backed by a [repo](#repository) with a `MODULE.bazel` file at its +root. This file contains metadata about the module itself (such as its name and +version), its direct dependencies, and various other data including toolchain +registrations and [module extension](#module-extension) input. + +Module metadata is hosted in Bazel registries. + +**See also:** [Bazel modules](/external/module) + +### Module Extension {:#module-extension} + +A piece of logic that can be run to generate [repos](#repository) by reading +inputs from across the [module](#module) dependency graph and invoking [repo +rules](#repository-rule). Module extensions have capabilities similar to repo +rules, allowing them to access the internet, perform file I/O, and so on. + +**See also:** [Module extensions](/external/extension) + ### Native rules {:#native-rules} [Rules](#rule) that are built into Bazel and implemented in Java. Such rules @@ -351,8 +382,8 @@ example, `native.cc_library` or `native.java_library`). User-defined rules ### Output base {:#output-base} A [workspace](#workspace)-specific directory to store Bazel output files. Used -to separate outputs from the *workspace*'s source tree. Located in the [output -user root](#output-user-root). +to separate outputs from the *workspace*'s source tree (the [main +repo](#repository)). Located in the [output user root](#output-user-root). ### Output groups {:#output-groups} @@ -376,9 +407,9 @@ also known as [output bases](#output-base). ### Package {:#package} The set of [targets](#target) defined by a [`BUILD` file](#build-file). A -package's name is the `BUILD` file's path relative to the workspace root. A -package can contain subpackages, or subdirectories containing `BUILD` files, -thus forming a package hierarchy. +package's name is the `BUILD` file's path relative to the [repo](#repository) +root. A package can contain subpackages, or subdirectories containing `BUILD` +files, thus forming a package hierarchy. ### Package group {:#package-group} @@ -422,16 +453,48 @@ but can't analyze the effects of `select()`, [build flags](#command-flags), **See also:** [Query how-to](/query/guide), [Query reference](/query/language) +### Repository {:#repository} + +A directory tree with a boundary marker file at its root, containing source +files that can be used in a Bazel build. Often shortened to just **repo**. + +A repo boundary marker file can be `MODULE.bazel` (signaling that this repo +represents a Bazel module), `REPO.bazel`, or in legacy contexts, `WORKSPACE` or +`WORKSPACE.bazel`. Any repo boundary marker file will signify the boundary of a +repo; multiple such files can coexist in a directory. + +The *main repo* is the repo in which the current Bazel command is being run. + +*External repos* are defined by specifying [modules](#module) in `MODULE.bazel` +files, or invoking [repo rules](#repository-rule) in [module +extensions](#module-extension). They can be fetched on demand to a predetermined +"magical" location on disk. + +Each repo has a unique, constant *canonical* name, and potentially different +*apparent* names when viewed from other repos. + +**See also**: [External dependencies overview](/external/overview) + ### Repository cache {:#repo-cache} A shared content-addressable cache of files downloaded by Bazel for builds, shareable across [workspaces](#workspace). Enables offline builds after the -initial download. Commonly used to cache files downloaded through repository -rules like `http_archive` and repository rule APIs like +initial download. Commonly used to cache files downloaded through [repository +rules](#repository-rule) like `http_archive` and repository rule APIs like `repository_ctx.download`. Files are cached only if their SHA-256 checksums are specified for the download. - +### Repository rule {:#repository-rule} + +A schema for repository definitions that tells Bazel how to materialize (or +"fetch") a [repository](#repository). Often shortened to just **repo rule**. +Repo rules are invoked by Bazel internally to define repos backed by +[modules](#module), or can be invoked by [module extensions](#module-extension). +Repo rules can access the internet or perform file I/O; the most common repo +rule is `http_archive` to download an archive containing source files from the +internet. + +**See also:** [Repo rule documentation](/extending/repo) ### Reproducibility {:#reproducibility} @@ -618,12 +681,10 @@ or `.bzl` file may load a given `.bzl` file. Without context, usually ### Workspace {:#workspace} -A directory containing a `WORKSPACE` file and source code for the software you -want to build. Labels that start with `//` are relative to the workspace -directory. - -### WORKSPACE file {:#workspace-file} +The environment shared by all Bazel commands run from the same [main +repository](#repository). -Defines a directory to be a [workspace](#workspace). The file can be empty, -although it usually contains external repository declarations to fetch -additional dependencies from the network or local filesystem. +Note that historically the concepts of "repository" and "workspace" have been +conflated; the term "workspace" has often been used to refer to the main +repository, and sometimes even used as a synonym of "repository". Such usage +should be avoided for clarity. diff --git a/src/main/java/com/google/devtools/build/docgen/templates/be/functions.vm b/src/main/java/com/google/devtools/build/docgen/templates/be/functions.vm index 5e92910e5f6fd5..80c354a9fc9381 100644 --- a/src/main/java/com/google/devtools/build/docgen/templates/be/functions.vm +++ b/src/main/java/com/google/devtools/build/docgen/templates/be/functions.vm @@ -39,6 +39,10 @@ package(default_deprecation, default_package_metadata, default_testonly, default

This function declares metadata that applies to every rule in the package. It is used at most once within a package (BUILD file).

+

For the counterpart that declares metadata applying to every rule in the whole +repository, use the repo() function in the +REPO.bazel file at the root of your repo. +The repo() function takes exactly the same arguments as package().

The package() function should be called right after all the load() statements at the top of the file, before any rule.