python3Packages.{torch,torch-bin}: 2.0.1 -> 2.1.1#259068
Merged
ConnorBaker merged 7 commits intoNixOS:masterfrom Dec 1, 2023
Merged
python3Packages.{torch,torch-bin}: 2.0.1 -> 2.1.1#259068ConnorBaker merged 7 commits intoNixOS:masterfrom
ConnorBaker merged 7 commits intoNixOS:masterfrom
Conversation
github-actions
bot
added
the
6.topic: python
Contributor
Author
|
Binaries are missing for |
GaetanLepage
requested review from
SomeoneSerge,
samuela,
teh,
thoughtpolice and
tscholak
Member
|
Thanks for putting this together so quickly @GaetanLepage ! |
ofborg
bot
added
10.rebuild-darwin: 101-500
GaetanLepage
force-pushed
the
pytorch
branch
3 times, most recently
from
ddb3fcb to
0e89147
Compare
ofborg
bot
added
the
8.has: clean-up
samuela
reviewed
Oct 5, 2023
Contributor
|
Good timing, I'm almost done testing #258328. |
38 tasks
Contributor
Contributor
|
On macos 10.15 this appears to break |
Contributor
|
Almost forgot about this. |
2 tasks
SomeoneSerge
reviewed
Oct 9, 2023
Member
|
Awesome! |
GaetanLepage
force-pushed
the
pytorch
branch
4 times, most recently
from
a44cf7a to
9a6cb67
Compare
ConnorBaker
approved these changes
Nov 29, 2023
Contributor
|
@GaetanLepage I'm still unable to build PyTorch or the test I use to verify it (https://github.com/connorbaker/nix-cuda-test) nix build github:ConnorBaker/nix-cuda-test#nix-cuda-test --override-input nixpkgs "github:GaetanLepage/nixpkgs/8672a90ccb487e1a6ca4e07314610e9aa8f22fca"fails with (full log: https://gist.github.com/ConnorBaker/b5ee6bbcc2c078fd13ba285e13c59f6b) CMake Error at /nix/store/vnhl4zdy7igx9gd3q1d548vwzz15a9ma-cmake-3.27.7/share/cmake-3.27/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
Could NOT find CUDAToolkit (missing: CUDAToolkit_INCLUDE_DIR) (found
version "11.8.89")
Call Stack (most recent call first):
/nix/store/vnhl4zdy7igx9gd3q1d548vwzz15a9ma-cmake-3.27.7/share/cmake-3.27/Modules/FindPackageHandleStandardArgs.cmake:600 (_FPHSA_FAILURE_MESSAGE)
/nix/store/vnhl4zdy7igx9gd3q1d548vwzz15a9ma-cmake-3.27.7/share/cmake-3.27/Modules/FindCUDAToolkit.cmake:938 (find_package_handle_standard_args)
cmake/public/cuda.cmake:59 (find_package)
cmake/Dependencies.cmake:44 (include)
CMakeLists.txt:722 (include)I believe you need to add these changes to your patch: https://github.com/ConnorBaker/nixpkgs/blob/6d8c6ac313ec6327a0477238703a0f160fcec445/pkgs/development/python-modules/torch/0001-cmake-Use-cmake-provided-FindCUDAToolkit.cmake.patch#L58-L59. |
ConnorBaker
requested changes
Nov 29, 2023
pkgs/development/python-modules/torch/fix-cmake-cuda-toolkit.patch
Outdated
Show resolved
Hide resolved
GaetanLepage
force-pushed
the
pytorch
branch
from
8672a90 to
edbb0e4
Compare
Contributor
Author
|
|
ConnorBaker
requested changes
Nov 30, 2023
GaetanLepage
force-pushed
the
pytorch
branch
from
edbb0e4 to
63c6160
Compare
ConnorBaker
approved these changes
Nov 30, 2023
1 task
Contributor
|
I'm able to build and run PyTorch to train a small model: $ nom build github:ConnorBaker/nix-cuda-test#nix-cuda-test --override-input nixpkgs "github:GaetanLepage/nixpkgs/63c61609df86dbb0dc581f80afe80f870e62d3f6" --builders ''
$ ./result/bin/nix-cuda-test
Seed set to 42
Using bfloat16 Automatic Mixed Precision (AMP)
GPU available: True (cuda), used: True
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
HPU available: False, using: 0 HPUs
Files already downloaded and verified
Files already downloaded and verified
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
| Name | Type | Params
-----------------------------------------------
0 | criterion | CrossEntropyLoss | 0
1 | model | ViT | 86.3 M
-----------------------------------------------
86.3 M Trainable params
0 Non-trainable params
86.3 M Total params
345.317 Total estimated model params size (MB)
Epoch 9: 100%|████████████████████████████████████████████████████████████████████████████████| 781/781 [01:28<00:00, 8.83it/s, v_num=29, train_loss=2.350, val_loss=2.330]`Trainer.fit` stopped: `max_epochs=10` reached.
Epoch 9: 100%|████████████████████████████████████████████████████████████████████████████████| 781/781 [01:30<00:00, 8.65it/s, v_num=29, train_loss=2.350, val_loss=2.330]Notably, @SomeoneSerge @samuela @Madouura barring any surprise developments I plan to merge this tomorrow. Is that acceptable? |
Contributor
Author
|
Result of 26 packages marked as broken and skipped:
30 packages failed to build:
514 packages built:
|
Contributor
|
Fairseq is overdue for a new release, the maintainers have been contacted, you can ignore the failure. |
samuela
reviewed
Nov 30, 2023
Contributor
The PR's contents looks sane, but why is |
GaetanLepage
force-pushed
the
pytorch
branch
from
63c6160 to
acbfdb5
Compare
Madouura
approved these changes
Nov 30, 2023
Contributor
Madouura
left a comment
Madouura
left a comment
There was a problem hiding this comment.
Built python311Packages.torchWithRocm locally.
No idea why the nixpkgs-review build failed, but works on my machine.
LGTM.
Contributor
|
Merging -- thank you @GaetanLepage! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
8 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description of changes
Update
torchandtorch-bin.Changelog: https://github.com/pytorch/pytorch/releases/tag/v2.1.1
cc @teh @thoughtpolice @tscholak
Things done
sandbox = trueset innix.conf? (See Nix manual)nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage./result/bin/)