Skip to content

libtensorflow: 1.9 -> 1.14.0#63208

Closed
yorickvP wants to merge 1 commit intoNixOS:masterfrom
yorickvP:hstf
Closed

libtensorflow: 1.9 -> 1.14.0#63208
yorickvP wants to merge 1 commit intoNixOS:masterfrom
yorickvP:hstf

Conversation

@yorickvP
Copy link
Contributor

@yorickvP yorickvP commented Jun 16, 2019

This also changes it to a from-source build.

Motivation for this change

Update, also convert to built-from-source package.

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Assured whether relevant documentation is up to date
  • Fits CONTRIBUTING.md.

This still needs testing everywhere. The linux build without cuda compiled one time. I can't test on darwin.

cc: @basvandijk

@yorickvP yorickvP requested a review from Profpatsch as a code owner June 16, 2019 13:01
@basvandijk
Copy link
Member

basvandijk commented Jun 16, 2019

@GrahamcOfBorg build libtensorflow

@ofborg ofborg bot added the 8.has: package (new) This PR adds a new package label Jun 16, 2019
@ofborg ofborg bot requested a review from basvandijk June 16, 2019 13:19
@ofborg ofborg bot added 10.rebuild-darwin: 1-10 This PR causes between 1 and 10 packages to rebuild on Darwin. 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux. labels Jun 16, 2019
@FRidh FRidh mentioned this pull request Jun 16, 2019
Closed
10 tasks
@FRidh
Copy link
Member

FRidh commented Jun 16, 2019

Should close #47025 when this is merged.

@FRidh FRidh mentioned this pull request Jun 16, 2019
Closed
@yorickvP
Copy link
Contributor Author

yorickvP commented Jun 17, 2019 •
edited
Loading

I'm working on upgrading the haskell package: tensorflow/haskell#244

Profpatsch
Profpatsch suggested changes Jun 17, 2019
View reviewed changes
Copy link
Member

@Profpatsch Profpatsch left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I’m not extremely excited about having to maintain multiple versions of bazel.

Can you please put the renaming file action into a separate commit, so that I can see the differences in generic.nix?

@yorickvP
Copy link
Contributor Author

yorickvP commented Jun 18, 2019

I've updated to 1.14.0-rc1, which does compile with bazel 0.26.1. Not ideal but probably better than including another bazel version.

@yorickvP
Copy link
Contributor Author

yorickvP commented Jun 18, 2019

I'm working on cuda support now.

@yorickvP
Copy link
Contributor Author

yorickvP commented Jun 18, 2019 •
edited
Loading

Cuda support compiles now. Feel free to merge.

@Profpatsch
Copy link
Member

Profpatsch commented Jun 18, 2019

cc @basvandijk

@timokau
Copy link
Member

timokau commented Jun 24, 2019

I wish I had seen this before starting to work on #63616! Lots of good work here. We should merge the libtensorflow derivation with the tensorflow wheel build. Probably best to build both at the same time and then split it, to save build time.

@timokau timokau mentioned this pull request Jun 24, 2019
Closed
10 tasks
@yorickvP
libtensorflow: 1.9 -> 1.14.0
14e5034 
This also changes it to a from-source build.
@yorickvP
Copy link
Contributor Author

yorickvP commented Jun 30, 2019

I updated this to 1.14.0 final. @timokau that seems to have some good changes. Using system libraries might help a lot with our dynamic linking problems! Any ideas about how to do the multiple build outputs if this is built for multiple python versions?

@timokau
Copy link
Member

timokau commented Jul 1, 2019

Its probably not easy, maybe not even desirable to do this with multiple outputs. It would be possible to build multiple different bazel targets and put them in different outputs. That would add some complexity however and would always force everyone who wants to build one to build all of them.

@FRidh FRidh changed the title libtensorflow: 1.9 -> 1.13.1 libtensorflow: 1.9 -> 1.14 Jul 11, 2019
@FRidh FRidh changed the title libtensorflow: 1.9 -> 1.14 libtensorflow: 1.9 -> 1.14.0 Jul 11, 2019
@abbradar abbradar mentioned this pull request Jul 13, 2019
Merged
10 tasks
@yorickvP
Copy link
Contributor Author

yorickvP commented Jul 14, 2019

Superseded by #64716

@yorickvP yorickvP closed this Jul 14, 2019
abbradar added a commit to abbradar/nixpkgs that referenced this pull request Jul 14, 2019
@abbradar
tensorflow: cleanup and merge with libtensorflow
7b5a242 
This merges work done by yorickvP and timokau in NixOS#63208 and NixOS#63616 respectively.
Now the derivation builds both libtensorflow and the Python package and puts them into
different outputs.

Quite a bit of improvements were done on the top, including:

* Use official tag revision as source, not a branch;
* Use all system libraries possible (before only one was actually used);
* Move various environment variables to the derivation itself from hooks;
* Use source Python build instead of wheel build to ensure fixup hooks do their important jobs on libraries;
* And more that I forgot!
abbradar added a commit to abbradar/nixpkgs that referenced this pull request Jul 31, 2019
@abbradar
tensorflow: cleanup and merge with libtensorflow
019c136 
This merges work done by yorickvP and timokau in NixOS#63208 and NixOS#63616 respectively.
Now the derivation builds both libtensorflow and the Python package and puts them into
different outputs.

Quite a bit of improvements were done on the top, including:

* Use official tag revision as source, not a branch;
* Use all system libraries possible (before only one was actually used);
* Move various environment variables to the derivation itself from hooks;
* Use source Python build instead of wheel build to ensure fixup hooks do their important jobs on libraries;
* And more that I forgot!
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@basvandijk basvandijk Awaiting requested review from basvandijk

1 more reviewer

@Profpatsch Profpatsch Profpatsch approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

8.has: package (new) This PR adds a new package 10.rebuild-darwin: 1-10 This PR causes between 1 and 10 packages to rebuild on Darwin. 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@yorickvP @basvandijk @FRidh @Profpatsch @timokau