Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 9 additions & 5 deletions docs/build/backends.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,22 +2,26 @@ To decouple the building of a conda package from Pixi we provide something what
These are essentially executables following a specific protocol that is implemented for both Pixi and the build backend.
This also allows for decoupling of the build backend from Pixi and it's manifest specification.

The backends we are currently developing are available in the following [conda channel](https://prefix.dev/channels/pixi-build-backends).
And are being developed in the [pixi-build-backends](https://github.com/prefix-dev/pixi-build-backends) repository.
The Prefix.dev managed backends are being developed in the [pixi-build-backends](https://github.com/prefix-dev/pixi-build-backends) repository, and have their own
[documentation](https://prefix-dev.github.io/pixi-build-backends/).

### Installation

When looking at the following build-section:
Install a certain build backend by adding it to the `package.build` section of the manifest file.:

```toml
--8<-- "docs/source_files/pixi_tomls/simple_pixi_build.toml:build-system"
```

5. This will allow Pixi to install desired backends from the `pixi-build-backends` channel, and any requirements from `conda-forge`. Backends are installed into isolated environments, and will be shared across Pixi workspaces.
For custom backend channels, you can add the channel to the `channels` section of the manifest file:
```toml
--8<-- "docs/source_files/pixi_tomls/pixi-build-backend-channel.toml:build"
```


### Overriding the Build Backend

Sometimes you want to override the build backend that is used by pixi. Meaning overriding the backend that is specified in the [`[package.build]`](../reference/pixi_manifest.md#the-build-system). We currently have two environment variables that allow for this:
Sometimes you want to override the build backend that is used by pixi. Meaning overriding the backend that is specified in the [`[package.build]`](../reference/pixi_manifest.md#build-table). We currently have two environment variables that allow for this:

1. `PIXI_BUILD_BACKEND_OVERRIDE`: This environment variable allows for overriding of one or multiple backends. Use `{name}={path}` to specify a backend name mapped to a path and `,` to separate multiple backends.
For example: `pixi-build-cmake=/path/to/bin,pixi-build-python` will:
Expand Down
5 changes: 1 addition & 4 deletions docs/build/workspace.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,10 +48,7 @@ Therefore, we can remove the workspace section in the manifest of `cpp_math`.

```diff title="packages/cpp_math/pixi.toml"
-[workspace]
-channels = [
- "https://prefix.dev/pixi-build-backends",
- "https://prefix.dev/conda-forge",
-]
-channels = ["https://prefix.dev/conda-forge"]
-platforms = ["osx-arm64", "osx-64", "linux-64", "win-64"]
-preview = ["pixi-build"]
-
Expand Down
153 changes: 101 additions & 52 deletions docs/reference/pixi_manifest.md
Original file line number Diff line number Diff line change
Expand Up @@ -501,29 +501,6 @@ rust = "==1.72"
pytorch-cpu = { version = "~=1.1", channel = "pytorch" }
```


### `host-dependencies`

```toml
[host-dependencies]
python = "~=3.10.3"
```
Typical examples of host dependencies are:

- Base interpreters: a Python package would list `python` here and an R package would list `mro-base` or `r-base`.
- Libraries your workspace links against during compilation like `openssl`, `rapidjson`, or `xtensor`.

### `build-dependencies`

This table contains dependencies that are needed to build the workspace.
Different from `dependencies` and `host-dependencies` these packages are installed for the architecture of the _build_ machine.
This enables cross-compiling from one machine architecture to another.

```toml
[build-dependencies]
cmake = "~=3.24"
```

### `pypi-dependencies`

??? info "Details regarding the PyPI integration"
Expand Down Expand Up @@ -915,55 +892,127 @@ An example of a preview feature in the manifest:

Preview features in the documentation will be marked as such on the relevant pages.

## Pixi Build

Pixi build is an experimental feature that requires `preview = ["pixi-build"]` to be set in `[workspace]`
## The `package` section

### Workspace section

Currently, `workspace` is an alias for `project` and we recommend using `workspace` instead of `project`,
when making use of the `pixi-build` preview feature.
To use this keyword the preview feature *does not* need to be enabled, but for now we do recommend it for that use-case solely.

### Package section
!!! warning "Important note"
`pixi-build` is a [preview feature](#preview-features), and will change until it is stabilized.
Please keep that in mind when you use it for your workspaces.
```toml
--8<-- "docs/source_files/pixi_tomls/simple_pixi_build.toml:preview"
```

The package section is used to define the package that is built by the project.
Pixi only allows this table if `preview = ["pixi-build"]` is set in `[workspace]`.
The package section can be added
to a workspace manifest to define the package that is built by Pixi.

```toml
--8<-- "docs/source_files/pixi_tomls/simple_pixi_build.toml:package"
```
A package section needs to be inside a `workspace`,
either in the same manifest file as the `[workspace]` table or in a sub folder `pixi.toml`/`pyproject.toml` file.

### Host, Build, dependencies
These packages will be built into a conda package that can be installed into a conda environment.
The package section is defined using the following fields:

The package section re-uses the `host-dependencies` and `build-dependencies`,
which you can read about here: [host-build-dependencies](#host-dependencies) and [build-dependencies](#build-dependencies).
If you have the `preview = ["pixi-build"]` enabled these are interpreted as part of the package.
- `name`: The name of the package.
- `version`: The version of the package.
- `build`: The build system used to build the package.
- `build-dependencies`: The build dependencies of the package.
- `host-dependencies`: The host dependencies of the package.
- `run-dependencies`: The run dependencies of the package.
- `target`: The target table to configure target specific dependencies. (Similar to the [target](#the-target-table) table)

### Run dependencies
And to extend the basics, it can also contain the following fields:

Run dependencies are dependencies that are required at runtime by your package.
For Python packages, these are the most common dependency types.
For compiled languages, these are less common and would basically be dependencies that you do not need when compiling the package but are needed when running it.
- `description`: A short description of the package.
- `authors`: A list of authors of the package.
- `license`: The license of the package.
- `license-file`: The license file of the package.
- `readme`: The README file of the package.
- `homepage`: The homepage link of the package.
- `repository`: The repository link of the package.
- `documentation`: The documentation link of the package.

```toml
--8<-- "docs/source_files/pixi_tomls/simple_pixi_build.toml:run-dependencies"
--8<-- "docs/source_files/pixi_tomls/pixi-package-manifest.toml:package"
--8<-- "docs/source_files/pixi_tomls/pixi-package-manifest.toml:extra-fields"
```

### The `build-system`
!!! note "Workspace inheritance"
Most extra fields can be inherited from the workspace manifest.
This means that you can define the `description`, `authors`, `license` in the workspace manifest, and they will be inherited by the package manifest.
```toml
[workspace]
name = "my-workspace"

[package]
name = { workspace = true } # Inherit the name from the workspace
```

### `build` table

The build system specifies how the package can be built.
The build system is a table that can contain the following fields:

- `channels`: specifies the channels to get the build backend from.
- `build-backend`: specifies the build backend to use. This is a table that can contain the following fields:
- `backend`: specifies the build backend to use. This is a table that can contain the following fields:
- `name`: the name of the build backend to use. This will also be the executable name.
- `version`: the version of the build backend to use.
- `configuration`: a table that contains the configuration options for the build backend.
- `target`: a table that can contain target specific build configuration.

More documentation on the backends can be found in the [build backend documentation](../build/backends.md).

```toml
--8<-- "docs/source_files/pixi_tomls/simple_pixi_build.toml:build-system"
--8<-- "docs/source_files/pixi_tomls/pixi-package-manifest.toml:build-system"
```

!!! note
We are currently not publishing the backends on conda-forge, but will do so in the future.
For now the backends are published at [conda channel](https://prefix.dev/channels/pixi-build-backends).

### The `build` `host` and `run` dependencies tables
The dependencies of a package are split into three tables.
Each of these tables has a different purpose and is used to define the dependencies of the package.

- [`build-dependencies`](#build-dependencies): Dependencies that are required to build the package on the build platform.
- [`host-dependencies`](#host-dependencies): Dependencies that are required during the build process, to link against the package on the target platform.
- [`run-dependencies`](#run-dependencies): Dependencies that are required to run the package on the target platform.


### `build-dependencies`
Build dependencies are required in the build environment and contain all tools that are not needed on the host of the package.

Following packages are examples of typical build dependencies:

- compilers (`gcc`, `clang`, `gfortran`)
- build tools (`cmake`, `ninja`, `meson`)
- `make`
- `pkg-config`
- VSC packages (`git`, `svn`)

```toml
--8<-- "docs/source_files/pixi_tomls/pixi-package-manifest.toml:build-dependencies"
```

### `host-dependencies`

Host dependencies are required during build phase, but in contrast to build packages they have to be present on the host.

Following packages are typical examples for host dependencies:

- shared libraries (c/c++)
- python/r libraries that link against c libraries
- `python`, `r-base`
- `setuptools`, `pip`

```toml
--8<-- "docs/source_files/pixi_tomls/pixi-package-manifest.toml:host-dependencies"
```

### `run-dependencies`
The `run-dependencies` are the packages that will be installed in the environment when the package is run.

- Libraries
- Extra data file packages
- Python/R packages that are not needed during build time

```toml
--8<-- "docs/source_files/pixi_tomls/pixi-package-manifest.toml:run-dependencies"
```


1 change: 1 addition & 0 deletions docs/source_files/pixi_tomls/LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
DUMMY FOR PARSING
6 changes: 1 addition & 5 deletions docs/source_files/pixi_tomls/dependency_types.toml
Original file line number Diff line number Diff line change
Expand Up @@ -8,11 +8,7 @@ name = "simple_cpp"
version = "0.1.0"

[package.build]
backend = { name = "pixi-build-cmake", version = "0.1.*" }
channels = [
"https://prefix.dev/pixi-build-backends",
"https://prefix.dev/conda-forge",
]
backend = { name = "pixi-build-cmake", version = "0.3.*" }
# --8<-- [start:dependencies]
[package.build-dependencies]
cxx-compiler = "*"
Expand Down
16 changes: 16 additions & 0 deletions docs/source_files/pixi_tomls/pixi-build-backend-channel.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
[workspace]
channels = []
platforms = []
preview = ["pixi-build"]

# --8<-- [start:build]
[package.build]
backend = { name = "pixi-build-python", version = "==0.3.2" }
channels = [
"https://prefix.dev/pixi-build-backends",
"https://prefix.dev/conda-forge",
]
# --8<-- [end:build]
[package]
name = "pixi-build-backend-channel"
version = "0.1.0"
47 changes: 47 additions & 0 deletions docs/source_files/pixi_tomls/pixi-package-manifest.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
# --8<-- [start:package]
[package]
name = "package"
version = "0.1.0"
# --8<-- [end:package]
# --8<-- [start:extra-fields]
authors = ["Author Name <author.name@email.org>"]
description = "A package description"
documentation = "https://docs.example.com"
homepage = "https://example.com"
license = "MIT"
license-file = "LICENSE"
readme = "README.md"
repository = "https://github.com/user/repo"
# --8<-- [end:extra-fields]

# --8<-- [start:build-system]
[package.build]
backend = { name = "pixi-build-cmake", version = "0.3.*" }
# not required:
channels = ["https://prefix.dev/conda-forge"]
configuration = { key = "value" } # Optional configuration, specific to the build backend
# --8<-- [end:build-system]


# --8<-- [start:build-dependencies]
[package.build-dependencies]
cmake = "*"
git = "*"

# --8<-- [end:build-dependencies]

# --8<-- [start:host-dependencies]
[package.host-dependencies]
python = "*"
# --8<-- [end:host-dependencies]

# --8<-- [start:run-dependencies]
[package.run-dependencies]
rich = ">=13.9.4,<14"
# --8<-- [end:run-dependencies]


[workspace]
channels = ["https://prefix.dev/conda-forge"]
platforms = ["osx-arm64", "osx-64", "linux-64", "win-64"]
preview = ["pixi-build"]
6 changes: 1 addition & 5 deletions docs/source_files/pixi_tomls/simple_pixi_build.toml
Original file line number Diff line number Diff line change
Expand Up @@ -19,11 +19,7 @@ version = "0.1.0"
# We are using `pixi-build-python` in order to build a Python package
# --8<-- [start:build-system]
[package.build]
backend = { name = "pixi-build-python", version = "0.1.*" }
channels = [
"https://prefix.dev/pixi-build-backends",
"https://prefix.dev/conda-forge",
]
backend = { name = "pixi-build-python", version = "==0.3.2" }
# --8<-- [end:build-system]


Expand Down
Loading
Loading