Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
6f9da1b
Ship mstar_rust in the Linux wheels on PyPI
dghosef Oct 26, 2026
4bbc490
release: build the Linux wheels for CPython 3.14 too
dghosef Oct 27, 2026
c920649
rust: pin the compiler to 1.99.1 in rust-toolchain.toml
dghosef Oct 27, 2026
ca89297
release: ship the mstar-server frontend binary in the Linux wheels
dghosef Oct 27, 2026
9616a63
release: collect the Rust third-party licenses into the wheels
dghosef Oct 27, 2026
4d9ed9c
release: check each built wheel in a clean venv
dghosef Oct 27, 2026
b00d056
docs: describe the Rust wheels in PACKAGING.md
dghosef Oct 27, 2026
a7284db
Merge remote-tracking branch 'origin/main' into review/pr883-fixes
dghosef Oct 28, 2026
53e3b4e
release: publish the Rust wheels in their own job, after the pure dists
dghosef Oct 28, 2026
65292a1
api_server: look for mstar-server in the running env before $PATH
dghosef Oct 28, 2026
8603aa0
communication: say which mstar_rust loaded, and how to fix a stale one
dghosef Oct 28, 2026
d9ff325
rust: put pyo3 behind a `python` feature so mstar-server links no Python
dghosef Oct 28, 2026
e61580e
release: build and check the wheels on PRs and by hand, publish only …
dghosef Oct 28, 2026
50680c1
release: run test/rust against each built wheel
dghosef Oct 28, 2026
4df28ae
release: check each wheel's mstar_rust is the version mstar expects
dghosef Oct 28, 2026
701beab
setup.py: build the wheels' Rust with --locked and strip them
dghosef Oct 28, 2026
0c14c50
rust-toolchain: pin rustfmt and clippy too, and bake the pin into the…
dghosef Oct 28, 2026
05de822
pyproject: list CPython 3.13 and 3.14 in the classifiers
dghosef Oct 28, 2026
50a72ff
rust: mark mstar-rust and mstar-server publish = false
dghosef Oct 28, 2026
a1cd6e8
release, about.toml: tidy two comments
dghosef Oct 28, 2026
1e64509
release: collect license notices for the extension as well as the fro…
dghosef Oct 28, 2026
32fe0f1
release: on PR and manual runs, tolerate only libzmq's license
dghosef Oct 28, 2026
8a31166
release: build one CPython per arch off a release, and cancel superse…
dghosef Oct 28, 2026
91b7d8a
docker: make the pinned toolchain the XPU image's default
dghosef Oct 28, 2026
5c1cf53
docs: say who compiles rust/ and what the wheel checks cover
dghosef Oct 28, 2026
2393775
rust: move the PyO3 bindings into bindings.rs behind one cfg
dghosef Oct 28, 2026
78e45bf
MANIFEST.in: ship rust/ and rust-toolchain.toml in the sdist
dghosef Oct 28, 2026
d34e178
release: read-only token by default, and no credentials left in check…
dghosef Oct 28, 2026
12d03d4
release: key the concurrency group on the event, and fix two comments
dghosef Oct 28, 2026
5977093
setup.py, MANIFEST.in: say what gates the Rust build and what an sdis…
dghosef Oct 28, 2026
df43342
Merge remote-tracking branch 'origin/main' into review/pr883-fixes
dghosef Oct 28, 2026
371f541
Build the vendored libzmq with posix_memalign
dghosef Oct 28, 2026
b299afc
Accept libzmq's LGPL so the Linux wheels publish
dghosef Oct 28, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 33 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,13 +12,41 @@ name: Publish to PyPI
# one release run at a time: publish with the engine's pending publisher,
# add the next alias's pending publisher, re-run the failed job, repeat.
# Uploads skip files that already exist, so re-runs are safe.
#
# mstar-ai ships an sdist, a pure-Python wheel, and Linux wheels per CPython
# version and architecture that also carry the mstar_rust extension (setup.py
# compiles rust/ only under cibuildwheel). pip prefers the Linux wheel where
# one matches and falls back to the pure one elsewhere, which then runs on the
# Python transport and graph runtime.

on:
release:
types: [published]

jobs:
wheels:
strategy:
matrix:
os: [ubuntu-latest, ubuntu-24.04-arm]
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v4
- uses: pypa/cibuildwheel@v4.2.1
env:
# Version-specific rather than abi3, matching the extension that
# `maturin develop` builds and the tests exercise.
CIBW_BUILD: "cp312-manylinux_* cp313-manylinux_*"
CIBW_BEFORE_ALL_LINUX: curl -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal
CIBW_ENVIRONMENT_LINUX: PATH="$HOME/.cargo/bin:$PATH"
with:
output-dir: wheelhouse
- uses: actions/upload-artifact@v4
with:
name: wheels-${{ matrix.os }}
path: wheelhouse/*.whl

pypi:
needs: wheels
runs-on: ubuntu-latest
environment: pypi
permissions:
Expand All @@ -39,6 +67,11 @@ jobs:
- run: pip install build
- name: Build mstar-ai
run: python -m build
- uses: actions/download-artifact@v4
with:
pattern: wheels-*
path: dist
merge-multiple: true
- name: Build the alias packages
run: |
for alias in packaging/aliases/*/; do
Expand Down
5 changes: 3 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -166,8 +166,9 @@ The ZeroMQ control mesh can run over a Rust transport (vendored in [`rust/`](rus
pyzmq — wire-compatible, selectable per process with `MSTAR_RUST_ZMQ` (default `AUTO`).
Note what AUTO-by-default means: on any machine where the extension is built, the whole
mesh switches to the Rust transport with no configuration change — each process logs its
choice at startup (`control mesh transport: ...`), and `MSTAR_RUST_ZMQ=0` pins pyzmq. Build it with
[maturin](https://www.maturin.rs): `uv pip install maturin && maturin develop --release -m rust/Cargo.toml`.
choice at startup (`control mesh transport: ...`), and `MSTAR_RUST_ZMQ=0` pins pyzmq. The Linux
wheels on PyPI (x86_64 and aarch64, CPython 3.12 and 3.13) ship the extension; in a source checkout,
build it with [maturin](https://www.maturin.rs): `uv pip install maturin && maturin develop --release -m rust/Cargo.toml`.
See the [installation docs](https://m-star.org/mstar/installation.html) and
[environment variables](https://m-star.org/mstar/environment_variables.html).

Expand Down
7 changes: 5 additions & 2 deletions docs/installation.rst
Original file line number Diff line number Diff line change
Expand Up @@ -478,8 +478,11 @@ runs on the pure-Python paths as before. Migrated so far (see
wherever that holds, so on a machine with the extension built, installing
it is what moves a worker onto the Rust runtime.

Build the extension into your environment with `maturin
<https://www.maturin.rs>`_ (needs a Rust toolchain; ``rustup`` works):
The PyPI wheels for Linux (x86_64 and aarch64, CPython 3.12 and 3.13) already
carry the extension, so ``pip install mstar-ai`` there needs nothing more. Other
platforms get the pure-Python wheel. In a source checkout, build the extension into
your environment with `maturin <https://www.maturin.rs>`_ (needs a Rust toolchain;
``rustup`` works):

.. code-block:: bash

Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
[build-system]
requires = ["setuptools>=78.1.1", "wheel"]
requires = ["setuptools>=78.1.1", "setuptools-rust>=1.11", "wheel"]
build-backend = "setuptools.build_meta"

[project]
Expand Down
26 changes: 19 additions & 7 deletions setup.py
Original file line number Diff line number Diff line change
@@ -1,13 +1,14 @@
"""Build shim: copy the repo-root configs/ into mstar/default_configs/ so the
default deployment YAMLs ship inside the wheel under the mstar namespace.
default deployment YAMLs ship inside the wheel under the mstar namespace, and
compile the ``mstar_rust`` extension into the published Linux wheels.

Project metadata lives in pyproject.toml; this file exists only to populate
the packaged configs at build time. configs/ stays the single source of truth
(unchanged for checkouts and in-flight model PRs); nothing is duplicated in
git. The sdist carries configs/ via MANIFEST.in so a wheel built from the
sdist also picks them up.
Project metadata lives in pyproject.toml; this file exists only for those two
build steps. configs/ stays the single source of truth (unchanged for checkouts
and in-flight model PRs); nothing is duplicated in git. The sdist carries
configs/ via MANIFEST.in so a wheel built from the sdist also picks them up.
"""

import os
import shutil
from pathlib import Path

Expand All @@ -34,4 +35,15 @@ def run(self):
shutil.copy2(y, dst / y.name)


setup(cmdclass={"build_py": build_py_with_configs})
# Only the release's cibuildwheel jobs compile rust/; every other build is the
# pure-Python package, and a checkout builds the extension with maturin
# (docs/installation.rst). setuptools' editable finder cannot import a
# top-level extension built in place, and compiling it would slow every
# `pip install -e .`.
rust_extensions = []
if os.environ.get("CIBUILDWHEEL") == "1":
from setuptools_rust import Binding, RustExtension

rust_extensions.append(RustExtension("mstar_rust", path="rust/Cargo.toml", binding=Binding.PyO3))

setup(cmdclass={"build_py": build_py_with_configs}, rust_extensions=rust_extensions)
Loading