Skip to content

feat: add pixi publish command for building and uploading packages#5678

Merged
ruben-arts merged 22 commits intomainfrom
claude/add-pixi-publish-3FWWs
Mar 26, 2026
Merged

feat: add pixi publish command for building and uploading packages#5678
ruben-arts merged 22 commits intomainfrom
claude/add-pixi-publish-3FWWs

Conversation

@wolfv
Copy link
Copy Markdown
Member

@wolfv wolfv commented Mar 15, 2026

Description

This PR introduces a new pixi publish command that combines building and uploading conda packages in a single step. It's a convenience wrapper around pixi build and pixi upload functionality. This is ported from rattler-build.

Key Features:

  • Builds conda packages from a workspace manifest
  • Automatically uploads to multiple backend types:
    • prefix.dev (with optional sigstore attestation support)
    • Anaconda.org
    • S3-compatible storage (with automatic channel initialization and indexing)
    • Local filesystem (with automatic channel initialization and indexing)
    • Quetz servers
    • JFrog Artifactory
  • Supports all standard build options: --target-platform, --build-platform, --build-dir, --clean, --path
  • Supports upload options: --force (overwrite existing packages), --generate-attestation (prefix.dev only)
  • Reuses existing authentication infrastructure via pixi auth and environment variables
  • Automatically detects upload backend from URL scheme or host

Implementation Details:

  • Phase 1: Reuses the build logic from pixi build command to compile packages
  • Phase 2: Routes to appropriate upload backend based on URL parsing
  • Handles platform-specific subdirectories for S3 and local filesystem uploads
  • Automatically initializes and indexes channels for S3 and filesystem backends

Documentation:

  • Added comprehensive CLI reference documentation with examples for each backend
  • Updated prefix.dev deployment guide with pixi publish examples and trusted publishing information
  • Included authentication guidance and comparison with separate pixi build/pixi upload workflow

How Has This Been Tested?

The implementation leverages existing, well-tested components:

  • Build logic is identical to pixi build command
  • Upload backends use rattler_upload and rattler_index libraries (existing dependencies)
  • URL parsing and backend detection are straightforward and covered by the routing logic
  • Authentication uses the existing pixi_auth infrastructure

Manual testing would involve:

  1. Building and publishing to prefix.dev with API key authentication
  2. Publishing to local filesystem and verifying channel indexing
  3. Testing URL parsing for various backend formats
  4. Verifying --force flag behavior with existing packages

Checklist:

  • I have performed a self-review of my own code
  • I have made corresponding changes to the documentation
  • Changes follow the existing code patterns in pixi_cli (similar to build.rs)

https://claude.ai/code/session_016n3z1PSG2UYV622ZDxwXD6

claude added 4 commits March 15, 2026 11:17
Add a new `pixi publish` command that combines `pixi build` and
`pixi upload` into a single workflow, similar to `rattler-build publish`.

The command builds conda packages from a manifest and uploads them to a
target channel specified via the `--to` flag. Supported backends:
- prefix.dev (https://prefix.dev/<channel>)
- anaconda.org (https://anaconda.org/<owner>/<label>)
- S3 (s3://bucket-name) with automatic channel initialization and indexing
- Local filesystem (file:///path/to/channel) with automatic indexing
- Quetz (quetz://server/<channel>)
- Artifactory (artifactory://server/<channel>)

Upload routing is adapted from rattler_build_core::publish.

https://claude.ai/code/session_016n3z1PSG2UYV622ZDxwXD6
- Add auto-generated CLI reference page for `pixi publish`
- Add `publish_extender` with detailed examples for all backends
  (prefix.dev, anaconda.org, S3, local filesystem, quetz, artifactory)
- Update prefix.dev deployment guide to showcase `pixi publish` as the
  recommended workflow for building and publishing in one step
- Update trusted publishing CI example to use `pixi publish`

https://claude.ai/code/session_016n3z1PSG2UYV622ZDxwXD6
- Fix rustdoc bare URL and unclosed HTML tag warnings by wrapping URLs
  and angle-bracket placeholders in backticks/autolinks
- Fix rustfmt formatting issues in upload_packages and
  determine_package_subdir
- Add `pixi publish` to test_frozen_no_install_invariant test coverage
- Regenerate CLI docs to reflect rustdoc changes

https://claude.ai/code/session_016n3z1PSG2UYV622ZDxwXD6
pixi publish uses --path instead of --manifest-path (like pixi build),
so it needs a special case in the test. Expect failure since publish
will fail at the upload step (no auth/package) but the invariant check
on conda-meta is what matters.

https://claude.ai/code/session_016n3z1PSG2UYV622ZDxwXD6
@wolfv wolfv changed the title Add pixi publish command for building and uploading packages feat: add pixi publish command for building and uploading packages Mar 16, 2026
@ruben-arts ruben-arts marked this pull request as draft March 16, 2026 13:43
@wolfv
Copy link
Copy Markdown
Member Author

wolfv commented Mar 16, 2026

Test protocol:

  • pixi auth login prefix.dev --token $PREFIX_API_KEY
  • pixi publish --to https://prefix.dev/wolf-private-test succesfully published sdl_example to the channel (for osx-arm64).

Local file channel works and creates local index:

╭─wolfv@pixi ~/Programs/pixi/examples/pixi-build/cpp-sdl ‹claude/add-pixi-publish-3FWWs●›
╰─$ cargo r -- publish --to file:///tmp/channel                                                                                                                                                                                                            
✔ Successfully built 'sdl_example-0.1.0-h60d57d3_0.conda'
📦 Publishing 1 package(s) to file:///tmp/channel                                                                                                                                                                                                               
✔ Successfully published 1 package(s) to file:///tmp/channel
╭─wolfv@pixi ~/Programs/pixi/examples/pixi-build/cpp-sdl ‹claude/add-pixi-publish-3FWWs●›
╰─$ ll /tmp/channel
total 0
drwxr-xr-x 3 wolfv  96 Mar 16 15:39 noarch
drwxr-xr-x 7 wolfv 224 Mar 16 15:39 osx-arm64
╭─wolfv@pixi ~/Programs/pixi/examples/pixi-build/cpp-sdl ‹claude/add-pixi-publish-3FWWs●›
╰─$ ll /tmp/channel/osx-arm64
total 20K
-rw-r--r-- 1 wolfv  155 Mar 16 15:39 repodata_shards.msgpack.zst
-rw-r--r-- 1 wolfv  459 Mar 16 15:39 repodata.json
-rw-r--r-- 1 wolfv  311 Mar 16 15:39 repodata.json.zst
-rw------- 1 wolfv 5.5K Mar 16 15:18 sdl_example-0.1.0-h60d57d3_0.conda
drwxr-xr-x 3 wolfv   96 Mar 16 15:39 shards

@ruben-arts ruben-arts mentioned this pull request Mar 17, 2026
@wolfv wolfv marked this pull request as ready for review March 19, 2026 17:32
@wolfv
Copy link
Copy Markdown
Member Author

wolfv commented Mar 19, 2026

I also tested with S3 and Anaconda and after the fixes in rattler (merged!) this is good :)

Copy link
Copy Markdown
Contributor

@ruben-arts ruben-arts left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some prelimenary reviews

Comment thread crates/pixi_cli/src/publish.rs
Comment thread crates/pixi_cli/src/publish.rs
Comment thread crates/pixi_cli/src/publish.rs
Comment thread crates/pixi_cli/src/publish.rs
Comment thread crates/pixi_cli/src/publish.rs
claude and others added 2 commits March 24, 2026 10:06
- Add --skip-existing flag (default true) to skip uploading packages
  that already exist on the target channel, avoiding confusing duplicate
  upload messages and HTTP 409 errors
- Show initial build summary listing all packages to be built with
  name, version, build string, and platform before starting builds
- Improve build result output to include package file size and full
  path to the built artifact
- Clear progress bars after upload completes to avoid leftover
  "Uploading" progress bar text in the terminal

https://claude.ai/code/session_016CakNMdgLnjAjDCGnEKxmy
Comment thread crates/pixi_cli/src/publish.rs Outdated
claude and others added 10 commits March 24, 2026 10:16
Break the build result into a structured multi-line format showing
name, size, and path on separate lines instead of a single long line.
Use indicatif::HumanBytes (already used elsewhere in pixi) instead of
a custom format_size helper.

https://claude.ai/code/session_016CakNMdgLnjAjDCGnEKxmy
Clear the progress bar lines left behind by rattler_upload after upload
completes using console::Term::clear_last_lines(). Also improve the
success summary to list each published package by name.

https://claude.ai/code/session_016CakNMdgLnjAjDCGnEKxmy
The clear_last_lines hack exists because rattler_upload uses
progress_bar.finish() instead of finish_and_clear(), leaving
rendered bars on stderr. Added a clear comment explaining the
upstream fix needed (3 lines in rattler_upload).

https://claude.ai/code/session_016CakNMdgLnjAjDCGnEKxmy
The finish vs finish_and_clear fix should be done in rattler_upload
upstream, not worked around here.

https://claude.ai/code/session_016CakNMdgLnjAjDCGnEKxmy
Wrap the upload in await_in_progress so there's a pixi-managed
spinner during package uploads. The spinner is properly cleared
via finish_and_clear when the upload completes.

https://claude.ai/code/session_016CakNMdgLnjAjDCGnEKxmy
Move per-build "Successfully built" messages out of the build loop
and print a single consolidated summary after all builds complete,
right before the publish phase. This gives a cleaner output flow:
build plan -> build output -> build summary -> publish -> publish summary.

https://claude.ai/code/session_016CakNMdgLnjAjDCGnEKxmy
@dhirschfeld
Copy link
Copy Markdown
Contributor

Automatically uploads to multiple backend types:

  • prefix.dev (with optional sigstore attestation support)
  • Anaconda.org
  • S3-compatible storage (with automatic channel initialization and indexing)
  • Local filesystem (with automatic channel initialization and indexing)
  • Quetz servers
  • JFrog Artifactory

Cloudsmith?

Copy link
Copy Markdown
Contributor

@ruben-arts ruben-arts left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving this as a good first version! Looking forward to making this subcommand even better in the future!

@ruben-arts ruben-arts enabled auto-merge (squash) March 26, 2026 16:23
@ruben-arts ruben-arts merged commit 36752a5 into main Mar 26, 2026
41 checks passed
@ruben-arts ruben-arts deleted the claude/add-pixi-publish-3FWWs branch March 26, 2026 16:25
HARSHVARANDANI pushed a commit to HARSHVARANDANI/pixi that referenced this pull request Mar 27, 2026
…refix-dev#5678)

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Ruben Arts <ruben.arts@pm.me>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants