Add Buildkite CI pipeline (CPU jobs + manual-gated GPU suites) and remove PR test on GHA

[khluu]

Summary

Buildkite port of vime CI, mirroring the jobs in .github/workflows/pr-test.yml.j2. GitHub Actions keeps running in parallel and stays authoritative until Buildkite proves itself.

• .buildkite/pipeline.yml — always-on CPU steps (pre-commit, plugin-contracts, agent-adapter, unit), run on every PR and push to main.
• .buildkite/gpu_suites.py — the run-ci-* GPU suites (short, vllm-config, megatron, precision, ckpt) behind a manual block-step gate (PR labels can't trigger Buildkite), generating one K8s pod per test on the mithril-h100-pool H100 queue.
• .buildkite/README.md — one-time Buildkite setup and how to keep the test lists in sync with the GHA matrix.
• Drops 4 orphaned test_arguments.py unit tests left behind by the sync: complete slime #1920, #1967, #1985 #232 arguments refactor.

gsm8k_async_short (80 GB OOM) and parallel_check (CP-invariance) are marked soft_fail pending follow-up.

Also deleted PR workflows on GHA

https://claude.ai/code/session_01BSqHKH1FafdRobA7ZeRzBQ

[gemini-code-assist]

Code Review

This pull request introduces a Buildkite pipeline configuration and generator script to port always-on CPU CI jobs from GitHub Actions to Buildkite. The review feedback suggests making the docker_command helper more robust by handling trailing spaces in extra_args and centrally passing GIT_CONFIG_PARAMETERS to avoid Git ownership errors. Consequently, the manual git config workaround in pre_commit_step and the trailing space in unit_step can be cleaned up. Additionally, a minor typo in the README documentation should be corrected.

[gemini-code-assist]

The current implementation of docker_command is fragile because it directly concatenates extra_args without ensuring proper spacing. If a caller passes extra_args without a trailing space (e.g., "--network host"), it will result in a malformed command (e.g., --network host-v "$PWD:/workspace").

Additionally, running git commands inside containers (which run as root) on a workspace owned by the host's buildkite-agent user will trigger Git's "dubious ownership" error. Centrally passing GIT_CONFIG_PARAMETERS as an environment variable in docker_command solves this globally for all steps, eliminating the need for manual git config workarounds in individual steps.

def docker_command(image: str, script: str, extra_args: str = "") -> str:
    """`docker run` wrapper matching the GHA jobs' raw-docker style."""
    # Pass GIT_CONFIG_PARAMETERS to avoid "dubious ownership" errors since the container runs as root
    # while the workspace is owned by the host's buildkite-agent user.
    extra = f"{extra_args.strip()} " if extra_args.strip() else ""
    return (
        "docker run --rm --ipc=host --shm-size=4g "
        '-e GIT_CONFIG_PARAMETERS="\\'safe.directory=*\\'" '
        f"{extra}"
        f'-v "$PWD:{REPO_MOUNT}" -w {REPO_MOUNT} '
        f"{image} bash -c {shlex.quote(script)}"
    )

[gemini-code-assist]

Since safe.directory is now centrally handled via GIT_CONFIG_PARAMETERS in docker_command, we can clean up and simplify pre_commit_step by removing the manual git config workaround.

def pre_commit_step() -> dict: 
    script = "\n".join(
        [
            "set -euo pipefail",
            "pip install -q pre-commit",
            "pre-commit run --all-files --show-diff-on-failure --color=always",
        ]
    )
    return step(
        ":lint-roller: pre-commit",
        "pre-commit",
        CPU_QUEUE_SMALL,
        docker_command(PYTHON_IMAGE, script),
        timeout=15,
        gated=False,
    )

[gemini-code-assist]

With the updated docker_command handling trailing spaces robustly, we can remove the manual trailing space from the extra_args parameter here.

Suggested change

	return step(
	":pytest: unit & utils tests (in-image)",
	"unit",
	CPU_QUEUE_MEDIUM,
	docker_command(CI_IMAGE, script, extra_args="--network host "),
	timeout=45,
	)
	return step(
	":pytest: unit & utils tests (in-image)",
	"unit",
	CPU_QUEUE_MEDIUM,
	docker_command(CI_IMAGE, script, extra_args="--network host"),
	timeout=45,
	)

[gemini-code-assist]

There is a minor typo in the documentation. The Buildkite YAML key is depends_on (with an s), but the README refers to it as depend_on.

Suggested change

	All non-gate steps `depend_on` the pre-commit gate, matching the GHA
	All non-gate steps `depends_on` the pre-commit gate, matching the GHA

[read-the-docs-community]

Documentation build overview

📚 vime | 🛠️ Build #33160508 | 📁 Comparing 2f01009 against latest (fa0b6e9)

  🔍 Preview build  

26 files changed · ± 26 modified

± Modified

• genindex.html
• index.html
• search.html
• advanced/arch-support-beyond-megatron.html
• advanced/fault-tolerance.html
• advanced/megatron-config.html
• advanced/pd-disaggregation.html
• advanced/reproducibility.html
• advanced/speculative-decoding.html
• advanced/vllm-config.html
• and 16 more...

[aoshen02]

shrink the max_batched_tokens to 2048 as it will oom in h100, it's slime's problem and will optimized in the future