Skip to content

python3Packages.ruff: reuse already built ruff binary#412112

Merged
GaetanLepage merged 1 commit intoNixOS:masterfrom
bengsparks:ruff-no-maturin
May 30, 2025
Merged

python3Packages.ruff: reuse already built ruff binary#412112
GaetanLepage merged 1 commit intoNixOS:masterfrom
bengsparks:ruff-no-maturin

Conversation

@bengsparks
Copy link
Contributor

@bengsparks bengsparks commented May 29, 2025

The maturin build system cannot see that we already build the ruff binary in nixpkgs. This leads to the ruff binary being rebuilt and placed inside of the resulting Python package. Every nixpkgs-review that passes through the pkgs.ruff package therefore builds ruff 3 times per architecture. Said binary was never even used due to the patching that already occurs in find_ruff_bin.

Especially with how time-consuming release builds can be in Rust, there is no reason to keep this package in this state.

master: nix-build -A python3Packages.ruff - 0.44s user 0.28s system 0% cpu 5:09.64 total
branch: nix-build -A python3Packages.ruff - 0.36s user 0.12s system 9% cpu 4.856 total

Still requires a symlink at the appropriate place that points at the ruff binary; I'll get to it later today.

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • Nixpkgs 25.11 Release Notes (or backporting 24.11 and 25.05 Nixpkgs Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
  • NixOS 25.11 Release Notes (or backporting 24.11 and 25.05 NixOS Release notes)
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Add a 👍 reaction to pull requests you find important.

@github-actions github-actions bot added the 6.topic: python Python is a high-level, general-purpose programming language. label May 29, 2025
@github-actions github-actions bot added 10.rebuild-darwin: 11-100 This PR causes between 11 and 100 packages to rebuild on Darwin. 10.rebuild-linux: 11-100 This PR causes between 11 and 100 packages to rebuild on Linux. 11.by: package-maintainer This PR was created by a maintainer of all the package it changes. labels May 29, 2025
@nix-owners nix-owners bot requested review from GaetanLepage and figsoda May 29, 2025 15:29
@bengsparks bengsparks marked this pull request as draft May 29, 2025 16:29
Copy link
Contributor

@GaetanLepage GaetanLepage left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a great idea! It could be done for uv too.
Is the PR ready?

@bengsparks
Copy link
Contributor Author

bengsparks commented May 29, 2025

@GaetanLepage I haven't been home since I filed the PR, give me a few more hours. It's missing a symlink in the correct location.
But yes, I intend to replicate this for uv, that is the idea 😄 I want to submit that PR after this one gets approved.

The maturin build system cannot see that we already build the ruff binary in nixpkgs.
This leads to the ruff binary being rebuilt and placed inside of the resulting Python package.
Every nixpkgs-review that passes through the `pkgs.ruff` package therefore builds `ruff` 3 times per architecture.
Said binary was never even used due to the patching that already occurs in `find_ruff_bin`.

Especially with how time-consuming release builds can be in Rust, there is no reason to keep this package in this state.

master: `nix-build -A python3Packages.ruff` - `0.44s user 0.28s system 0% cpu 5:09.64 total`
branch: `nix-build -A python3Packages.ruff` - `0.36s user 0.12s system 9% cpu 4.856 total`
@bengsparks
Copy link
Contributor Author

nixpkgs-review result

Generated using nixpkgs-review.

Command: nixpkgs-review pr 412112


aarch64-darwin

✅ 80 packages built:
  • dmarc-metrics-exporter
  • dmarc-metrics-exporter.dist
  • litellm
  • litellm.dist
  • marimo (python312Packages.marimo)
  • marimo.dist (python312Packages.marimo.dist)
  • openapi-python-client
  • openapi-python-client.dist
  • oterm
  • oterm.dist
  • python312Packages.fastapi-mcp
  • python312Packages.fastapi-mcp.dist
  • python312Packages.fastmcp
  • python312Packages.fastmcp.dist
  • python312Packages.gradio
  • python312Packages.gradio.dist
  • python312Packages.mcp
  • python312Packages.mcp.dist
  • python312Packages.mcpadapt
  • python312Packages.mcpadapt.dist
  • python312Packages.mypy-boto3-builder
  • python312Packages.mypy-boto3-builder.dist
  • python312Packages.pyhanko
  • python312Packages.pyhanko.dist
  • python312Packages.pytest-examples
  • python312Packages.pytest-examples.dist
  • python312Packages.pytest-ruff
  • python312Packages.pytest-ruff.dist
  • python312Packages.python-lsp-ruff
  • python312Packages.python-lsp-ruff.dist
  • python312Packages.reflex
  • python312Packages.reflex-chakra
  • python312Packages.reflex-chakra.dist
  • python312Packages.reflex.dist
  • python312Packages.ruff
  • python312Packages.ruff.dist
  • python312Packages.smolagents
  • python312Packages.smolagents.dist
  • python312Packages.xhtml2pdf
  • python312Packages.xhtml2pdf.dist
  • python312Packages.xsdata
  • python312Packages.xsdata.dist
  • python313Packages.fastapi-mcp
  • python313Packages.fastapi-mcp.dist
  • python313Packages.fastmcp
  • python313Packages.fastmcp.dist
  • python313Packages.gradio
  • python313Packages.gradio.dist
  • python313Packages.marimo
  • python313Packages.marimo.dist
  • python313Packages.mcp
  • python313Packages.mcp.dist
  • python313Packages.mcpadapt
  • python313Packages.mcpadapt.dist
  • python313Packages.mypy-boto3-builder
  • python313Packages.mypy-boto3-builder.dist
  • python313Packages.pyhanko
  • python313Packages.pyhanko.dist
  • python313Packages.pytest-examples
  • python313Packages.pytest-examples.dist
  • python313Packages.pytest-ruff
  • python313Packages.pytest-ruff.dist
  • python313Packages.python-lsp-ruff
  • python313Packages.python-lsp-ruff.dist
  • python313Packages.reflex
  • python313Packages.reflex-chakra
  • python313Packages.reflex-chakra.dist
  • python313Packages.reflex.dist
  • python313Packages.ruff
  • python313Packages.ruff.dist
  • python313Packages.smolagents
  • python313Packages.smolagents.dist
  • python313Packages.xhtml2pdf
  • python313Packages.xhtml2pdf.dist
  • python313Packages.xsdata
  • python313Packages.xsdata.dist
  • vectorcode
  • vectorcode.dist
  • vunnel
  • vunnel.dist

@bengsparks bengsparks marked this pull request as ready for review May 30, 2025 00:05
Copy link
Member

@Prince213 Prince213 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Built on x86_64-linux and build logs LGTM

@Prince213 Prince213 added the 12.approvals: 1 This PR was reviewed and approved by one person. label May 30, 2025
@GaetanLepage GaetanLepage requested a review from kirillrdy May 30, 2025 07:08
Copy link
Contributor

@GaetanLepage GaetanLepage left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

@bengsparks bengsparks added 12.approvals: 3+ This PR was reviewed and approved by three or more persons. and removed 12.approvals: 1 This PR was reviewed and approved by one person. labels May 30, 2025
@GaetanLepage GaetanLepage merged commit f00c0c8 into NixOS:master May 30, 2025
28 of 29 checks passed
@bengsparks bengsparks deleted the ruff-no-maturin branch May 30, 2025 22:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

6.topic: python Python is a high-level, general-purpose programming language. 10.rebuild-darwin: 11-100 This PR causes between 11 and 100 packages to rebuild on Darwin. 10.rebuild-linux: 11-100 This PR causes between 11 and 100 packages to rebuild on Linux. 11.by: package-maintainer This PR was created by a maintainer of all the package it changes. 12.approvals: 3+ This PR was reviewed and approved by three or more persons.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants