Skip to content

python3Packages: update boto and fsspec ecosystem#382202

Merged
NickCao merged 10 commits intoNixOS:stagingfrom
NickCao:boto
Mar 1, 2025
Merged

python3Packages: update boto and fsspec ecosystem#382202
NickCao merged 10 commits intoNixOS:stagingfrom
NickCao:boto

Conversation

@NickCao
Copy link
Member

@NickCao NickCao commented Feb 15, 2025

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 25.05 Release Notes (or backporting 24.11 and 25.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Add a 👍 reaction to pull requests you find important.

@github-actions github-actions bot added the 6.topic: python Python is a high-level, general-purpose programming language. label Feb 15, 2025
@NickCao
Copy link
Member Author

NickCao commented Feb 15, 2025

Contains #378522, #382172, and part of #382057

@github-actions github-actions bot added 10.rebuild-darwin: 2501-5000 This PR causes many rebuilds on Darwin and should target the staging branches. 10.rebuild-darwin: 501+ This PR causes many rebuilds on Darwin and should normally target the staging branches. 10.rebuild-linux: 2501-5000 This PR causes many rebuilds on Linux and should target the staging branches. 10.rebuild-linux: 501+ This PR causes many rebuilds on Linux and should normally target the staging branches. labels Feb 15, 2025
@GaetanLepage GaetanLepage mentioned this pull request Feb 15, 2025
Merged
13 tasks
@NickCao NickCao marked this pull request as draft February 15, 2025 15:25
@NickCao
Copy link
Member Author

NickCao commented Feb 15, 2025

Waiting for #382057 to reach staging.

NickCao and others added 10 commits February 15, 2025 15:09
@NickCao
python312Packages.botocore: 1.35.99 -> 1.36.21
e26ff7f 
Changelog: https://github.com/boto/botocore/blob/1.36.21/CHANGELOG.rst
@NickCao
python312Packages.boto3: 1.35.99 -> 1.36.21
de5081c
@NickCao
python312Packages.s3transfer: 0.10.1 -> 0.11.2
609dcd8 
Diff: boto/s3transfer@refs/tags/0.10.1...0.11.2

Changelog: https://github.com/boto/s3transfer/blob/0.11.2/CHANGELOG.rst
@NickCao
awscli: 1.36.40 -> 1.37.21
06deeb6 
Changelog: https://github.com/aws/aws-cli/blob/1.37.21/CHANGELOG.rst
@NickCao
python312Packages.moto: 5.0.26 -> 5.0.28
39887b0 
Changelog: https://github.com/getmoto/moto/blob/5.0.28/CHANGELOG.md
@NickCao
python312Packages.aiobotocore: 2.18.0 -> 2.19.0
0b161bc 
Diff: aio-libs/aiobotocore@refs/tags/2.18.0...2.19.0

Changelog: https://github.com/aio-libs/aiobotocore/releases/tag/2.19.0
@NickCao
python312Packages.aioboto3: 13.1.1 -> 13.4.0
d01c144 
Diff: terricain/aioboto3@refs/tags/v13.1.1...v13.4.0

Changelog: https://github.com/terrycain/aioboto3/blob/refs/tags/v13.4.0/CHANGELOG.rst
@GaetanLepage @NickCao
python312Packages.fsspec: 2024.12.0 -> 2025.2.0
84b2a4b 
Diff: fsspec/filesystem_spec@refs/tags/2024.12.0...2025.2.0

Changelog: https://github.com/fsspec/filesystem_spec/raw/2025.2.0/docs/source/changelog.rst
@GaetanLepage @NickCao
python312Packages.s3fs: enable fixed tests
90bfc1d
@NickCao
python312Packages.gcsfs: 2024.2.0 -> 2025.2.0
ab0eef8 
Diff: fsspec/gcsfs@refs/tags/2024.2.0...2025.2.0

Changelog: https://github.com/fsspec/gcsfs/raw/2025.2.0/docs/source/changelog.rst
@NickCao NickCao marked this pull request as ready for review February 15, 2025 20:13
@GaetanLepage GaetanLepage mentioned this pull request Feb 15, 2025
Closed
13 tasks
@NickCao NickCao merged commit 4e84632 into NixOS:staging Mar 1, 2025
39 checks passed
@NickCao NickCao deleted the boto branch March 1, 2025 20:11
@vcunat
Copy link
Member

vcunat commented Mar 11, 2025 •
edited
Loading

.boto3 update broke python3Packages.karton-core because of CERT-Polska/karton#270
/cc maintainers: @chivay + @fabaff

EDIT: this is close to nixpkgs master now; see PR #386822

@NickCao
Copy link
Member Author

NickCao commented Mar 11, 2025

A workaround is to set the environment variables: AWS_REQUEST_CHECKSUM_CALCULATION=when_required and AWS_RESPONSE_CHECKSUM_VALIDATION=when_required

@vcunat
Copy link
Member

vcunat commented Mar 12, 2025

I noticed on Hydra that gcsfs source won't fetch:

$ nix build -f. python3.pkgs.gcsfs.src
error: hash mismatch in fixed-output derivation '/nix/store/rmcz6gdd49vpzwk7l8g4nffwpv7kp2lv-source.drv':
         specified: sha256-1nPhttEmGI8jreMe5hwOZRALJa5QTNOw2McisV+cR54=
            got:    sha256-CXv0SbwOaEjD5HkJgPRfGk+VfciDjpvwomM5oCi6VIw=

I'm not sure if they've moved the tag or what's happened.

@NickCao
Copy link
Member Author

NickCao commented Mar 12, 2025

I noticed on Hydra that gcsfs source won't fetch:

$ nix build -f. python3.pkgs.gcsfs.src
error: hash mismatch in fixed-output derivation '/nix/store/rmcz6gdd49vpzwk7l8g4nffwpv7kp2lv-source.drv':
         specified: sha256-1nPhttEmGI8jreMe5hwOZRALJa5QTNOw2McisV+cR54=
            got:    sha256-CXv0SbwOaEjD5HkJgPRfGk+VfciDjpvwomM5oCi6VIw=

I'm not sure if they've moved the tag or what's happened.

See #389275

@wegank wegank mentioned this pull request Apr 19, 2025
Closed
13 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@teh teh Awaiting requested review from teh

@mbalatsko mbalatsko Awaiting requested review from mbalatsko

@onny onny Awaiting requested review from onny

@anthonyroussel anthonyroussel Awaiting requested review from anthonyroussel

Assignees

No one assigned

Labels

6.topic: python Python is a high-level, general-purpose programming language. 10.rebuild-darwin: 501+ This PR causes many rebuilds on Darwin and should normally target the staging branches. 10.rebuild-darwin: 2501-5000 This PR causes many rebuilds on Darwin and should target the staging branches. 10.rebuild-linux: 501+ This PR causes many rebuilds on Linux and should normally target the staging branches. 10.rebuild-linux: 2501-5000 This PR causes many rebuilds on Linux and should target the staging branches.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@NickCao @vcunat @GaetanLepage