Skip to content

python3Packages.amazon-ion: include c_ext module#461391

Merged
dotlambda merged 1 commit intoNixOS:masterfrom
ony:push-kvwspzupopzp
Dec 11, 2025
Merged

python3Packages.amazon-ion: include c_ext module#461391
dotlambda merged 1 commit intoNixOS:masterfrom
ony:push-kvwspzupopzp

Conversation

@ony
Copy link
Contributor

@ony ony commented Nov 13, 2025

Address problem of slow amazon-ion package due to miss of C extension module (link claiming 60x speedup vs pure implementation).

See #461284

Things done

  • Built on platform:
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • Tested, as applicable:
  • Ran nixpkgs-review on this PR. See nixpkgs-review usage ².
  • Tested basic functionality of all binary files, usually in ./result/bin/ ¹.
  • Nixpkgs Release Notes
    • Package update: when the change is major or breaking.
  • NixOS Release Notes
    • Module addition: when adding a new NixOS module.
    • Module update: when the change is significant.
  • Fits CONTRIBUTING.md, pkgs/README.md, maintainers/README.md and other READMEs.

¹ Tested effect with next commands

nix build --print-build-logs '.#python3Packages.amazon-ion'
nix develop '.#python3Packages.amazon-ion'
cd result/lib/python3*/site-packages
python3

(changed folder so python will load modulers relative to built site-packages folder where amazon package/module resides)

>>> import amazon.ion.simpleion
>>> amazon.ion.simpleion.c_ext
True
python -c 'import amazon.ion.simpleion as ion; print(ion.__IS_C_EXTENSION_SUPPORTED)'
True

² part of the output of nixpkgs-review rev --print-result 4b6dea07 (pr fails with urllib.error.HTTPError: HTTP Error 401: Unauthorized)

nixpkgs-review result

Generated using nixpkgs-review.

Command: nixpkgs-review
Commit: 4b6dea07193748d86a1a2ff889e9f6bc0c0e8e37


x86_64-linux

✅ 8 packages built:
  • python312Packages.amazon-ion
  • python312Packages.amazon-ion.dist
  • python312Packages.kestra
  • python312Packages.kestra.dist
  • python313Packages.amazon-ion
  • python313Packages.amazon-ion.dist
  • python313Packages.kestra
  • python313Packages.kestra.dist

Add a 👍 reaction to pull requests you find important.

github-actions[bot]

This comment was marked as resolved.

@ony ony changed the base branch from master to nixos-22.05 November 13, 2025 22:46
@github-actions github-actions bot added the 6.topic: policy discussion Discuss policies to work in and around Nixpkgs label Nov 13, 2025
@ony ony changed the base branch from nixos-22.05 to nixos-25.05 November 13, 2025 22:47
@nixpkgs-ci nixpkgs-ci bot closed this Nov 13, 2025
@nixpkgs-ci nixpkgs-ci bot reopened this Nov 13, 2025
@ony ony changed the base branch from nixos-25.05 to release-25.05 November 13, 2025 22:48
@nixpkgs-ci nixpkgs-ci bot closed this Nov 13, 2025
@nixpkgs-ci nixpkgs-ci bot reopened this Nov 13, 2025
@nixpkgs-ci nixpkgs-ci bot added 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux. 10.rebuild-darwin: 1-10 This PR causes between 1 and 10 packages to rebuild on Darwin. 6.topic: python Python is a high-level, general-purpose programming language. 4.workflow: backport This targets a stable branch labels Nov 13, 2025
@ony ony force-pushed the push-kvwspzupopzp branch from 83f27aa to af0e150 Compare November 13, 2025 23:20
@ony ony changed the base branch from release-25.05 to master November 13, 2025 23:20
@nixpkgs-ci nixpkgs-ci bot closed this Nov 13, 2025
@nixpkgs-ci nixpkgs-ci bot reopened this Nov 13, 2025
@github-actions github-actions bot dismissed their stale review November 13, 2025 23:21

All good now, thank you!

@nixpkgs-ci nixpkgs-ci bot added the 2.status: merge conflict This PR has merge conflicts with the target branch label Nov 13, 2025
@ony ony force-pushed the push-kvwspzupopzp branch from af0e150 to aa76db8 Compare November 13, 2025 23:38
@nixpkgs-ci nixpkgs-ci bot removed 2.status: merge conflict This PR has merge conflicts with the target branch 4.workflow: backport This targets a stable branch labels Nov 13, 2025
@ony ony force-pushed the push-kvwspzupopzp branch from aa76db8 to 4b6dea0 Compare November 14, 2025 00:32
@ony ony marked this pull request as ready for review November 14, 2025 01:04
@ony ony force-pushed the push-kvwspzupopzp branch from 4b6dea0 to 86db55e Compare November 15, 2025 15:01
@nixpkgs-ci nixpkgs-ci bot requested a review from terlar November 15, 2025 15:06
@dotlambda dotlambda changed the title Include c_ext module in python3Packages.amazon-ion python3Packages.amazon-ion: include c_ext module Dec 5, 2025
@ony ony force-pushed the push-kvwspzupopzp branch from 86db55e to 5a9e2c1 Compare December 7, 2025 19:52
@ony ony force-pushed the push-kvwspzupopzp branch from 5a9e2c1 to 6a4a9ba Compare December 7, 2025 20:03
@ony
Copy link
Contributor Author

ony commented Dec 7, 2025

nix shell .\#nixpkgs-review
nixpkgs-review rev --print-result 6a4a9ba4

nixpkgs-review result

Generated using nixpkgs-review.

Command: nixpkgs-review
Commit: 6a4a9ba43c6e770bcf7ffba60f840df46830ab79


x86_64-linux

✅ 8 packages built:
  • python312Packages.amazon-ion
  • python312Packages.amazon-ion.dist
  • python312Packages.kestra
  • python312Packages.kestra.dist
  • python313Packages.amazon-ion
  • python313Packages.amazon-ion.dist
  • python313Packages.kestra
  • python313Packages.kestra.dist

@ony ony requested a review from dotlambda December 7, 2025 20:16
@ony ony force-pushed the push-kvwspzupopzp branch 3 times, most recently from 6f34c2f to 98f52f7 Compare December 7, 2025 21:12
@nixpkgs-ci nixpkgs-ci bot requested a review from natsukium December 7, 2025 21:18
@ony ony requested a review from dotlambda December 7, 2025 21:22
@dotlambda
Copy link
Member

nixpkgs-review result

Generated using nixpkgs-review.

Command: nixpkgs-review pr 461391
Commit: 98f52f7bad3b7300922bc701d3e411c32f54e95e


x86_64-linux

❌ 8 packages failed to build:
  • python312Packages.amazon-ion
  • python312Packages.amazon-ion.dist
  • python312Packages.kestra
  • python312Packages.kestra.dist
  • python313Packages.amazon-ion
  • python313Packages.amazon-ion.dist
  • python313Packages.kestra
  • python313Packages.kestra.dist

@dotlambda
Copy link
Member

I guess leaveDotGit is not a good idea after all.

error: hash mismatch in fixed-output derivation '/nix/store/i46iialzwajmaqdxr0sa1fr6krq3ck22-source.drv':
         specified: sha256-wM4EybnRcIO4iblhxC3lPZuhG4CCUi04hSN4PIZyCnY=
            got:    sha256-QL8NxEcC2Z9pi+tCdm7hEBsj0C/ykMT5R1PgefUguxA=

@ony
Copy link
Contributor Author

ony commented Dec 7, 2025

I guess leaveDotGit is not a good idea after all.

error: hash mismatch in fixed-output derivation '/nix/store/i46iialzwajmaqdxr0sa1fr6krq3ck22-source.drv':
         specified: sha256-wM4EybnRcIO4iblhxC3lPZuhG4CCUi04hSN4PIZyCnY=
            got:    sha256-QL8NxEcC2Z9pi+tCdm7hEBsj0C/ykMT5R1PgefUguxA=

Ehh... I can try to play with postFetch to carry over needed information (pre-compute) and then remove .git (saw this comment mentioning leaveDotGit).
Judging by this comment for .git not being deterministic, there is a 2w old PR that will help to get rid of explicit .git removal.
But this means that we still need custom injection of all that information.

P.S. Thank you for helping and for quick feedback.

This should bring 60x speed up in parsing text Ion.

Resolves NixOS#461284
@ony ony force-pushed the push-kvwspzupopzp branch from aca443b to a76d0f2 Compare December 8, 2025 10:00
@ony ony requested a review from dotlambda December 8, 2025 10:10
@dotlambda
Copy link
Member

nixpkgs-review result

Generated using nixpkgs-review.

Command: nixpkgs-review pr 461391
Commit: a76d0f23d837022b424c7ab38c2601b23a8d2da5


x86_64-linux

✅ 8 packages built:
  • python312Packages.amazon-ion
  • python312Packages.amazon-ion.dist
  • python312Packages.kestra
  • python312Packages.kestra.dist
  • python313Packages.amazon-ion
  • python313Packages.amazon-ion.dist
  • python313Packages.kestra
  • python313Packages.kestra.dist

@dotlambda dotlambda added this pull request to the merge queue Dec 11, 2025
Merged via the queue into NixOS:master with commit eaa9b9c Dec 11, 2025
27 of 29 checks passed
@ony ony deleted the push-kvwspzupopzp branch February 4, 2026 20:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

6.topic: policy discussion Discuss policies to work in and around Nixpkgs 6.topic: python Python is a high-level, general-purpose programming language. 10.rebuild-darwin: 1-10 This PR causes between 1 and 10 packages to rebuild on Darwin. 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants