Skip to content

Commit

Permalink
[7.1.0] New docs for labels, repos, etc (#20978)
Browse files Browse the repository at this point in the history
* New entries in glossary.md
* Clarify repo names in labels.md
* Docs for REPO.bazel, `use_repo_rule`

Fixes #15821.

PiperOrigin-RevId: 590246296
Change-Id: Idba4c35fdb068f9befba1122e3724880ba8fd1c8
  • Loading branch information
Wyverald committed Jan 23, 2024
1 parent e0180e1 commit 4e51534
Show file tree
Hide file tree
Showing 7 changed files with 200 additions and 49 deletions.
43 changes: 30 additions & 13 deletions site/en/concepts/labels.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,18 +5,35 @@ Book: /_book.yaml

{% include "_buttons.html" %}

All targets belong to exactly one package. The name of a target is
called its _label_. Every label uniquely identifies a target. A
typical label in canonical form looks like:
A **label** is an identifier for a target. A typical label in its full canonical
form looks like:

```none
@@myrepo//my/app/main:app_binary
```

The first part of the label is the repository name, `@@myrepo`. The double-`@`
syntax signifies that this is a [*canonical* repo
name](/external/overview#canonical-repo-name), which is unique within
the workspace. Labels with canonical repo names unambiguously identify a target
no matter which context they appear in.

Often the canonical repo name is an arcane string that looks like
`@@rules_java~7.1.0~toolchains~local_jdk`. What is much more commonly seen is
labels with an [*apparent* repo name](/external/overview#apparent-repo-name),
which looks like:

```
@myrepo//my/app/main:app_binary
```

The first part of the label is the repository name, `@myrepo//`.
The only difference is the repo name being prefixed with one `@` instead of two.
This refers to a repo with the apparent name `myrepo`, which could be different
based on the context this label appears in.

In the typical case that a label refers to the same repository from which
it is used, the repository identifier may be abbreviated as `//`.
So, inside `@myrepo` this label is usually written as
it is used, the repo name part may be omitted. So, inside `@@myrepo` the first
label is usually written as

```
//my/app/main:app_binary
Expand All @@ -26,9 +43,9 @@ The second part of the label is the un-qualified package name
`my/app/main`, the path to the package
relative to the repository root. Together, the repository name and the
un-qualified package name form the fully-qualified package name
`@myrepo//my/app/main`. When the label refers to the same
`@@myrepo//my/app/main`. When the label refers to the same
package it is used in, the package name (and optionally, the colon)
may be omitted. So, inside `@myrepo//my/app/main`,
may be omitted. So, inside `@@myrepo//my/app/main`,
this label may be written either of the following ways:

```
Expand Down Expand Up @@ -56,14 +73,14 @@ this file is in the `my/app/main/testdata` subdirectory of the repository:
//my/app/main:testdata/input.txt
```

Strings like `//my/app` and `@some_repo//my/app` have two meanings depending on
Strings like `//my/app` and `@@some_repo//my/app` have two meanings depending on
the context in which they are used: when Bazel expects a label, they mean
`//my/app:app` and `@some_repo//my/app:app`, respectively. But, when Bazel
`//my/app:app` and `@@some_repo//my/app:app`, respectively. But, when Bazel
expects a package (e.g. in `package_group` specifications), they reference the
package that contains that label.

A common mistake in `BUILD` files is using `//my/app` to refer to a package, or
to *all* the targets in a package--it does not. Remember, it is
to *all* targets in a package--it does not. Remember, it is
equivalent to `//my/app:app`, so it names the `app` target in the `my/app`
package of the current repository.

Expand All @@ -89,9 +106,9 @@ are two ways (one wrong, one correct) to refer to this file within



Labels starting with `@//` are references to the main
Labels starting with `@@//` are references to the main
repository, which will still work even from external repositories.
Therefore `@//a/b/c` is different from
Therefore `@@//a/b/c` is different from
`//a/b/c` when referenced from an external repository.
The former refers back to the main repository, while the latter
looks for `//a/b/c` in the external repository itself.
Expand Down
10 changes: 9 additions & 1 deletion site/en/external/extension.md
Original file line number Diff line number Diff line change
Expand Up @@ -174,4 +174,12 @@ repositories generated by another extension. Even if you don't use this
functionality, it's best to put them in separate files in case you need it
later. This is because the extension's identify is based on its file, so moving
the extension into another file later changes your public API and is a backwards
incompatible change for your users.
incompatible change for your users.

### Specify the operating system and architecture

If your extension relies on the operating system or its architecture type,
ensure to indicate this in the extension definition using the `os_dependent`
and `arch_dependent` boolean attributes. This ensures that Bazel recognizes the
need for re-evaluation if there are changes to either of them.

25 changes: 21 additions & 4 deletions site/en/external/migration.md
Original file line number Diff line number Diff line change
Expand Up @@ -152,7 +152,9 @@ repository.
### Fetch external dependencies with module extensions{:#fetch-deps-module-extensions}
If your dependency is not a Bazel project or not yet available in any Bazel
registry, you can introduce it using [module extensions](/external/extension).
registry, you can introduce it using
[`use_repo_rule`](/external/module#use_repo_rule) or [module
extensions](/external/extension).
* **WORKSPACE**
Expand All @@ -172,9 +174,24 @@ registry, you can introduce it using [module extensions](/external/extension).
* **Bzlmod**
With Bzlmod, you have to move the definition into a `.bzl` file, which also
lets you share the definition between WORKSPACE and Bzlmod during the
migration period.
With Bzlmod, you can use the `use_repo_rule` directive in your MODULE.bazel
file to directly instantiate repos:
```python
## MODULE.bazel
http_file = use_repo_rule("@bazel_tools//tools/build_defs/repo:http.bzl", "http_file")
http_file(
name = "data_file",
url = "http://example.com/file",
sha256 = "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
)
```
Under the hood, this is implemented using a module extension. If you need to
perform more complex logic than simply invoking a repo rule, you could also
implement a module extension yourself. You'll need to move the definition
into a `.bzl` file, which also lets you share the definition between
WORKSPACE and Bzlmod during the migration period.
```python
## repositories.bzl
Expand Down
15 changes: 15 additions & 0 deletions site/en/external/module.md
Original file line number Diff line number Diff line change
Expand Up @@ -173,6 +173,21 @@ Bazel supports the following non-registry overrides:
* [`git_override`](/rules/lib/globals/module#git_override)
* [`local_path_override`](/rules/lib/globals/module#local_path_override)

## Define repos that don't represent Bazel modules {:#use_repo_rule}

With `bazel_dep`, you can define repos that represent other Bazel modules.
Sometimes there is a need to define a repo that does _not_ represent a Bazel
module; for example, one that contains a plain JSON file to be read as data.

In this case, you could use the [`use_repo_rule`
directive](/rules/lib/globals/module#use_repo_rule) to directly define a repo
by invoking a repo rule. This repo will only be visible to the module it's
defined in.

Under the hood, this is implemented using the same mechanism as [module
extensions](/external/extension), which lets you define repos with more
flexibility.

## Repository names and strict deps

The [canonical name](/external/overview#canonical-repo-name) of a repo backing a
Expand Down
33 changes: 31 additions & 2 deletions site/en/external/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,8 +25,14 @@ in Bazel, before going into a bit more detail about the two systems in order.

### Repository {:#repository}

A directory with a `WORKSPACE` or `WORKSPACE.bazel` file, containing source
files to be used in a Bazel build. Often shortened to just **repo**.
A directory tree with a boundary marker file at its root, containing source
files that can be used in a Bazel build. Often shortened to just **repo**.

A repo boundary marker file can be `MODULE.bazel` (signaling that this repo
represents a Bazel module), `REPO.bazel` (see [below](#repo.bazel)), or in
legacy contexts, `WORKSPACE` or `WORKSPACE.bazel`. Any repo boundary marker file
will signify the boundary of a repo; multiple such files can coexist in a
directory.

### Main repository {:#main-repository}

Expand Down Expand Up @@ -100,6 +106,29 @@ canonical name `canonical_name`:
ls $(bazel info output_base)/external/{{ '<var>' }} canonical_name {{ '</var>' }}
```

### REPO.bazel file {:#repo.bazel}

The `REPO.bazel` file is used to mark the topmost boundary of the directory tree
that constitutes a repo. It doesn't need to contain anything to serve as a repo
boundary file; however, it can also be used to specify some common attributes
for all build targets inside the repo.

The syntax of a `REPO.bazel` file is similar to `BUILD` files, except that no
`load` statements are supported, and only a single function, `repo()`, is
available. `repo()` takes the same arguments as the [`package()`
function](/reference/be/functions#package) in `BUILD` files; whereas `package()`
specifies common attributes for all build targets inside the package, `repo()`
analogously does so for all build targets inside the repo.

For example, you can specify a common license for all targets in your repo by
having the following `REPO.bazel` file:

```python
repo(
default_package_metadata = ["//:my_license"],
)
```

## Manage external dependencies with Bzlmod {:#bzlmod}

Bzlmod, the new external dependency subsystem, does not directly work with repo
Expand Down
119 changes: 90 additions & 29 deletions site/en/reference/glossary.md
Original file line number Diff line number Diff line change
Expand Up @@ -232,6 +232,9 @@ reference to `//:bar`. `//:foo` has an *action dependency* on `//:bar` if an
action in `//:foo` depends on an input [artifact](#artifact) created by an
action in `//:bar`.

In certain contexts, it could also refer to an _external dependency_; see
[modules](#module).

### Depset {:#depset}

A data structure for collecting data on transitive dependencies. Optimized so
Expand Down Expand Up @@ -308,28 +311,30 @@ build.

### Label {:#label}

An identifier for a [target](#target). A fully-qualified label such as
`//path/to/package:target` consists of `//` to mark the workspace root
directory, `path/to/package` as the directory that contains the [`BUILD`
file](#build-file) declaring the target, and `:target` as the name of the target
declared in the aforementioned `BUILD` file. May also be prefixed with
`@my_repository//<..>` to indicate that the target is declared in an [external
repository](/docs/external) named `my_repository`.
An identifier for a [target](#target). Generally has the form
`@repo//path/to/package:target`, where `repo` is the (apparent) name of the
[repository](#repository) containing the target, `path/to/package` is the path
to the directory that contains the [`BUILD` file](#build-file) declaring the
target (this directory is also known as the [package](#package)), and `target`
is the name of the target itself. Depending on the situation, parts of this
syntax may be omitted.

**See also**: [Labels](/concepts/labels)

### Loading phase {:#loading-phase}

The first phase of a build where Bazel parses `WORKSPACE`, `BUILD`, and [`.bzl`
files](#bzl-file) to create [packages](#package). [Macros](#macro) and certain
functions like `glob()` are evaluated in this phase. Interleaved with the second
phase of the build, the [analysis phase](#analysis-phase), to build up a [target
The first phase of a build where Bazel executes [`BUILD` files](#build-file) to
create [packages](#package). [Macros](#macro) and certain functions like
`glob()` are evaluated in this phase. Interleaved with the second phase of the
build, the [analysis phase](#analysis-phase), to build up a [target
graph](#target-graph).

### Macro {:#macro}

A mechanism to compose multiple [rule](#rule) target declarations together under
a single [Starlark](#starlark) function. Enables reusing common rule declaration
patterns across `BUILD` files. Expanded to the underlying rule target declarations
during the [loading phase](#loading-phase).
patterns across `BUILD` files. Expanded to the underlying rule target
declarations during the [loading phase](#loading-phase).

**See also:** [Macro documentation](/extending/macros)

Expand All @@ -341,6 +346,32 @@ identifiers for *spawn strategy* selections. Some examples of action mnemonics
are `Javac` from Java rules, `CppCompile` from C++ rules, and
`AndroidManifestMerger` from Android rules.

### Module {:#module}

A Bazel project that can have multiple versions, each of which can have
dependencies on other modules. This is analogous to familiar concepts in other
dependency management systems, such as a Maven _artifact_, an npm _package_, a
Go _module_, or a Cargo _crate_. Modules form the backbone of Bazel's external
dependency management system.

Each module is backed by a [repo](#repository) with a `MODULE.bazel` file at its
root. This file contains metadata about the module itself (such as its name and
version), its direct dependencies, and various other data including toolchain
registrations and [module extension](#module-extension) input.

Module metadata is hosted in Bazel registries.

**See also:** [Bazel modules](/external/module)

### Module Extension {:#module-extension}

A piece of logic that can be run to generate [repos](#repository) by reading
inputs from across the [module](#module) dependency graph and invoking [repo
rules](#repository-rule). Module extensions have capabilities similar to repo
rules, allowing them to access the internet, perform file I/O, and so on.

**See also:** [Module extensions](/external/extension)

### Native rules {:#native-rules}

[Rules](#rule) that are built into Bazel and implemented in Java. Such rules
Expand All @@ -351,8 +382,8 @@ example, `native.cc_library` or `native.java_library`). User-defined rules
### Output base {:#output-base}

A [workspace](#workspace)-specific directory to store Bazel output files. Used
to separate outputs from the *workspace*'s source tree. Located in the [output
user root](#output-user-root).
to separate outputs from the *workspace*'s source tree (the [main
repo](#repository)). Located in the [output user root](#output-user-root).

### Output groups {:#output-groups}

Expand All @@ -376,9 +407,9 @@ also known as [output bases](#output-base).
### Package {:#package}

The set of [targets](#target) defined by a [`BUILD` file](#build-file). A
package's name is the `BUILD` file's path relative to the workspace root. A
package can contain subpackages, or subdirectories containing `BUILD` files,
thus forming a package hierarchy.
package's name is the `BUILD` file's path relative to the [repo](#repository)
root. A package can contain subpackages, or subdirectories containing `BUILD`
files, thus forming a package hierarchy.

### Package group {:#package-group}

Expand Down Expand Up @@ -422,16 +453,48 @@ but can't analyze the effects of `select()`, [build flags](#command-flags),

**See also:** [Query how-to](/query/guide), [Query reference](/query/language)

### Repository {:#repository}

A directory tree with a boundary marker file at its root, containing source
files that can be used in a Bazel build. Often shortened to just **repo**.

A repo boundary marker file can be `MODULE.bazel` (signaling that this repo
represents a Bazel module), `REPO.bazel`, or in legacy contexts, `WORKSPACE` or
`WORKSPACE.bazel`. Any repo boundary marker file will signify the boundary of a
repo; multiple such files can coexist in a directory.

The *main repo* is the repo in which the current Bazel command is being run.

*External repos* are defined by specifying [modules](#module) in `MODULE.bazel`
files, or invoking [repo rules](#repository-rule) in [module
extensions](#module-extension). They can be fetched on demand to a predetermined
"magical" location on disk.

Each repo has a unique, constant *canonical* name, and potentially different
*apparent* names when viewed from other repos.

**See also**: [External dependencies overview](/external/overview)

### Repository cache {:#repo-cache}

A shared content-addressable cache of files downloaded by Bazel for builds,
shareable across [workspaces](#workspace). Enables offline builds after the
initial download. Commonly used to cache files downloaded through repository
rules like `http_archive` and repository rule APIs like
initial download. Commonly used to cache files downloaded through [repository
rules](#repository-rule) like `http_archive` and repository rule APIs like
`repository_ctx.download`. Files are cached only if their SHA-256 checksums are
specified for the download.

<!-- TODO: ### Repository rule -->
### Repository rule {:#repository-rule}

A schema for repository definitions that tells Bazel how to materialize (or
"fetch") a [repository](#repository). Often shortened to just **repo rule**.
Repo rules are invoked by Bazel internally to define repos backed by
[modules](#module), or can be invoked by [module extensions](#module-extension).
Repo rules can access the internet or perform file I/O; the most common repo
rule is `http_archive` to download an archive containing source files from the
internet.

**See also:** [Repo rule documentation](/extending/repo)

### Reproducibility {:#reproducibility}

Expand Down Expand Up @@ -618,12 +681,10 @@ or `.bzl` file may load a given `.bzl` file. Without context, usually

### Workspace {:#workspace}

A directory containing a `WORKSPACE` file and source code for the software you
want to build. Labels that start with `//` are relative to the workspace
directory.

### WORKSPACE file {:#workspace-file}
The environment shared by all Bazel commands run from the same [main
repository](#repository).

Defines a directory to be a [workspace](#workspace). The file can be empty,
although it usually contains external repository declarations to fetch
additional dependencies from the network or local filesystem.
Note that historically the concepts of "repository" and "workspace" have been
conflated; the term "workspace" has often been used to refer to the main
repository, and sometimes even used as a synonym of "repository". Such usage
should be avoided for clarity.
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,10 @@ package(default_deprecation, default_package_metadata, default_testonly, default
<p>This function declares metadata that applies to every rule in the
package. It is used at most once within a package (BUILD file).</p>

<p>For the counterpart that declares metadata applying to every rule in the whole
<em>repository</em>, use the <code>repo()</code> function in the
<a href="/external/overview#repo.bazel"><code>REPO.bazel</code> file</a> at the root of your repo.
The <code>repo()</code> function takes exactly the same arguments as <code>package()</code>.</p>
<p>The package() function should be called right after all the load() statements at the top of the
file, before any rule.</p>

Expand Down

0 comments on commit 4e51534

Please sign in to comment.