Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pythonPackages.rlp: 1.0.1 -> 1.0.2 (with some new deps) #45644

Merged
merged 10 commits into from
Sep 12, 2018

Conversation

jluttine
Copy link
Member

@jluttine jluttine commented Aug 26, 2018

Motivation for this change

Updated and fixed RLP Python package. Some dependencies are added as separate commits.

fixes #44224

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Fits CONTRIBUTING.md.

Sorry, something went wrong.

@FRidh
Copy link
Member

FRidh commented Aug 26, 2018

It's nowadays possible to use markdown directly, so no need for setuptools_markdown.
https://packaging.python.org/guides/making-a-pypi-friendly-readme/
A more recent setuptools and wheel are needed though.

Could you inform upstream?

@jluttine
Copy link
Member Author

Ok. I noticed nox-review is giving some failures at the moment. I'll try to fix them before merging.

@jluttine
Copy link
Member Author

@FRidh Now nox-review passed. I needed to add some more commits to make the fixes.

doCheck = false;

meta = with stdenv.lib; {
description = "Backport of shutil.which from Python 3.3";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe python 3 should be disabled here?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sha256 = "16sa3adkf71862cb9pk747pw80a2f1v5m915ijb4fgj309xrlhyx";
};

# Tests fail: "ValueError: underlying buffer has been detached"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this comment copy-and-pasted from python-u2flib-host?

Copy link
Member Author

@jluttine jluttine Aug 27, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried to build and the test phase raised that error. I tried to fix it but it has probably something to do with the test environment in nix. Not sure though.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For me this test works perfectly fine.

@Mic92
Copy link
Member

Mic92 commented Aug 27, 2018

Upstream is also informed: ethereum/eth-utils#124

@Mic92
Copy link
Member

Mic92 commented Aug 27, 2018

@GrahamcOfBorg build python3Packages.backports_shutil_which

@GrahamcOfBorg
Copy link

Success on aarch64-linux (full log)

Attempted: python3Packages.backports_shutil_which

Partial log (click to expand)

running install tests
============================= test session starts ==============================
platform linux -- Python 3.6.6, pytest-3.6.3, py-1.5.4, pluggy-0.6.0
rootdir: /build/backports.shutil_which-3.5.1, inifile:
collected 2 items

test/test_which.py ..                                                    [100%]

=========================== 2 passed in 0.06 seconds ===========================
/nix/store/xzxa0jvmxbbnbpivwb89cm5ihkpqrrg6-python3.6-backports.shutil_which-3.5.1

@GrahamcOfBorg
Copy link

Success on x86_64-darwin (full log)

Attempted: python3Packages.backports_shutil_which

Partial log (click to expand)

running install tests
============================= test session starts ==============================
platform darwin -- Python 3.6.6, pytest-3.6.3, py-1.5.4, pluggy-0.6.0
rootdir: /private/tmp/nix-build-python3.6-backports.shutil_which-3.5.1.drv-0/backports.shutil_which-3.5.1, inifile:
collected 2 items

test/test_which.py ..                                                    [100%]

=========================== 2 passed in 0.12 seconds ===========================
/nix/store/rqnij3zfp85735yrfiixcwrxjmq2wi7w-python3.6-backports.shutil_which-3.5.1

@GrahamcOfBorg
Copy link

Success on x86_64-linux (full log)

Attempted: python3Packages.backports_shutil_which

Partial log (click to expand)

running install tests
============================= test session starts ==============================
platform linux -- Python 3.6.6, pytest-3.6.3, py-1.5.4, pluggy-0.6.0
rootdir: /build/backports.shutil_which-3.5.1, inifile:
collected 2 items

test/test_which.py ..                                                    [100%]

=========================== 2 passed in 0.06 seconds ===========================
/nix/store/qpdyvfbqdsj62fwq55vjnim3441c6blg-python3.6-backports.shutil_which-3.5.1

@jluttine
Copy link
Member Author

Is this ok now?


meta = with stdenv.lib; {
description = "A simple, cross-platform, pure Python module for JavaScript-like message boxes";
homepage = "https://github.com/asweigart/PyMsgBox";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Homepage says this depends on tkinter. Btw, you should leave out the quotes.

substituteInPlace setup.py --replace \'setuptools-markdown\' ""
'';

disabled = !(pythonAtLeast "3.5");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pythonOlder

substituteInPlace setup.py --replace \'setuptools-markdown\' ""
'';

disabled = !(pythonAtLeast "3.5");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pythonOlder

@@ -437,6 +439,8 @@ in {

pymatgen = callPackage ../development/python-modules/pymatgen { };

PyMsgBox = callPackage ../development/python-modules/PyMsgBox { };
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd prefer using pymsgbox, also as dir name.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I can change the dir name. How about adding below that:

pymsgbox = PyMsgBox;

I've heard the package name should be equal to the name in PyPI, so that's why I thought it should be available at least as PyMsgBox.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would also prefer if it matches. We should have a policy on that, so we do this consistently for python packages.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about a policy to have both the PyPI name and a lowercase alias?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've written this before several times. We prefer to use normalized names throughout, because that matches well with the naming we use throughout Nixpkgs. The pname to fetchPypi however has to match to the filename used because PyPI does not translate that part of the url. If for fetchPypi the name cannot be normalized, then it may be more convenient to use the same one for buildPython*.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member

@Mic92 Mic92 Aug 30, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok. let's make this convention part of nixpkgs doc: #45822 (comment)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So I should rename backports_shutil_which to backports-shutil-which, right?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

@GrahamcOfBorg
Copy link

Success on x86_64-darwin (full log)

Attempted: python3.pkgs.eth-hash, python3.pkgs.eth-typing, python3.pkgs.eth-utils

The following builds were skipped because they don't evaluate on x86_64-darwin: pythob2.pkgs.eth-hash, python2.pkgs.eth-typing, python2.pkgs.eth-utils

Partial log (click to expand)

while evaluating anonymous function at /private/var/lib/ofborg/builds/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-ndnd/pkgs/development/python-modules/eth-hash/default.nix:1:1, called from /private/var/lib/ofborg/builds/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-ndnd/lib/customisation.nix:69:12:
while evaluating 'makeOverridablePythonPackage' at /private/var/lib/ofborg/builds/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-ndnd/pkgs/top-level/python-packages.nix:40:37, called from /private/var/lib/ofborg/builds/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-ndnd/pkgs/development/python-modules/eth-hash/default.nix:3:1:
while evaluating 'makeOverridable' at /private/var/lib/ofborg/builds/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-ndnd/lib/customisation.nix:67:24, called from /private/var/lib/ofborg/builds/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-ndnd/pkgs/top-level/python-packages.nix:42:12:
while evaluating anonymous function at /private/var/lib/ofborg/builds/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-ndnd/pkgs/development/interpreters/python/build-python-package.nix:27:1, called from /private/var/lib/ofborg/builds/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-ndnd/lib/customisation.nix:69:12:
while evaluating anonymous function at /private/var/lib/ofborg/builds/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-ndnd/pkgs/development/interpreters/python/mk-python-derivation.nix:15:1, called from /private/var/lib/ofborg/builds/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-ndnd/pkgs/development/interpreters/python/build-python-package.nix:44:4:
eth-hash-0.1.4 not supported for interpreter python2.7

/nix/store/sxh6s4d9pghxp8jr3qzybfsqrh37ba4q-python3.6-eth-hash-0.1.4
/nix/store/5q0a18fy1kkm7hh8hnv3kmp9jfkfy3qr-python3.6-eth-typing-1.1.0
/nix/store/vh7di2czkif07fnycf6iqi3ir2sv14is-python3.6-eth-utils-1.2.0

@GrahamcOfBorg
Copy link

Success on aarch64-linux (full log)

Attempted: python3.pkgs.eth-hash, python3.pkgs.eth-typing, python3.pkgs.eth-utils

The following builds were skipped because they don't evaluate on aarch64-linux: pythob2.pkgs.eth-hash, python2.pkgs.eth-typing, python2.pkgs.eth-utils

Partial log (click to expand)

while evaluating anonymous function at /var/lib/gc-of-borg/nix-test-rs-29/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-29/pkgs/development/python-modules/eth-hash/default.nix:1:1, called from /var/lib/gc-of-borg/nix-test-rs-29/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-29/lib/customisation.nix:69:12:
while evaluating 'makeOverridablePythonPackage' at /var/lib/gc-of-borg/nix-test-rs-29/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-29/pkgs/top-level/python-packages.nix:40:37, called from /var/lib/gc-of-borg/nix-test-rs-29/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-29/pkgs/development/python-modules/eth-hash/default.nix:3:1:
while evaluating 'makeOverridable' at /var/lib/gc-of-borg/nix-test-rs-29/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-29/lib/customisation.nix:67:24, called from /var/lib/gc-of-borg/nix-test-rs-29/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-29/pkgs/top-level/python-packages.nix:42:12:
while evaluating anonymous function at /var/lib/gc-of-borg/nix-test-rs-29/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-29/pkgs/development/interpreters/python/build-python-package.nix:27:1, called from /var/lib/gc-of-borg/nix-test-rs-29/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-29/lib/customisation.nix:69:12:
while evaluating anonymous function at /var/lib/gc-of-borg/nix-test-rs-29/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-29/pkgs/development/interpreters/python/mk-python-derivation.nix:15:1, called from /var/lib/gc-of-borg/nix-test-rs-29/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-29/pkgs/development/interpreters/python/build-python-package.nix:44:4:
eth-hash-0.1.4 not supported for interpreter python2.7

/nix/store/y0pyiz25vg9sah0kl4nw2xgq4n6zs755-python3.6-eth-hash-0.1.4
/nix/store/jawr54kbpx7radxd1q1vvsphqbrngypx-python3.6-eth-typing-1.1.0
/nix/store/rs6lbj4nwp47fppzk77pxay3p6qvx7ym-python3.6-eth-utils-1.2.0

@GrahamcOfBorg
Copy link

Success on x86_64-linux (full log)

Attempted: python3.pkgs.eth-hash, python3.pkgs.eth-typing, python3.pkgs.eth-utils

The following builds were skipped because they don't evaluate on x86_64-linux: pythob2.pkgs.eth-hash, python2.pkgs.eth-typing, python2.pkgs.eth-utils

Partial log (click to expand)

eth-typing==1.1.0
eth-utils==1.2.0
pycrypto==3.6.6
pycryptodome==3.6.6
pysha3==1.0.2
toolz==0.9.0

/nix/store/qav2n7z0bw34ggl2iar8525ipjdq6vcx-python3.6-eth-hash-0.1.4
/nix/store/pxvzfjvk8y6y3h9mdlci0akw6livds63-python3.6-eth-typing-1.1.0
/nix/store/0s3ixsbj1zzvvbqppghr6gqz059350pc-python3.6-eth-utils-1.2.0

@dotlambda
Copy link
Member

dotlambda commented Sep 11, 2018

All of the eth-* packages give

Ran 0 tests in 0.000s

You should either provide an appropriate checkPhase or set doCheck = false and add a comment with the reason.
Otherwise, this looks good to go.

@jluttine
Copy link
Member Author

At least eth-utils doesn't include tests in the PyPI release tarball. I'll report it upstream.

@dotlambda
Copy link
Member

Thanks! It's fine to set doCheck = false for now but don't forget to make a PR here once they include them in the PyPI tarball.

@jluttine
Copy link
Member Author

I put doCheck = false to eth-utils and eth-typing as they don't have unit tests in the PyPI release tarball. I reported these upstream: ethereum/eth-utils#130 and ethereum/eth-typing#8.

I enabled tests for eth-hash, but that required some tweaking.

@samueldr
Copy link
Member

Once ported to 18.09, it would contribute to fixing around 12 outstanding builds on x86_64, see ZHF #45960, and the generated reports. (Subscribing to keep track of the changes.)

@jluttine
Copy link
Member Author

Upstream of eth-utils and eth-typing doesn't want to include tests in PyPI release tarball so I'll modify the package definitions to use GitHub sources so they can be tested.

@jluttine
Copy link
Member Author

Done. I also updated the version numbers of eth-utils and eth-typing. Nox review passed.

@dotlambda dotlambda merged commit c15a94b into NixOS:master Sep 12, 2018
@dotlambda
Copy link
Member

Thanks a lot @jluttine!
Would you mind opening a PR against release-18.09 or do you want me to do that?

@jluttine
Copy link
Member Author

How does that work? Shall I cherry pick these 10 commits to that release branch? I can do it but I've never done it before so need to learn it.

@dotlambda
Copy link
Member

dotlambda commented Sep 12, 2018

Well, then it's a good thing to learn :)
You have to use git cherry-pick -x on the release-18.09 branch. Also see https://nixos.org/nixpkgs/manual/#idm140737315686368.
And then, make sure to select release-18.09 when making the PR.
Btw, I think git cherry-pick accepts commit ranges.

@xeji xeji mentioned this pull request Sep 12, 2018
@samueldr
Copy link
Member

samueldr commented Sep 13, 2018

$ git cherry-pick -x c44504410ba5a3bd76a99e7907533779c2b17694^..8d4afa5f2ef0fbdf042069ac71479bba1
104bbc8

[release-18.09 735b1e1] pythonPackages.eth-hash: init at 0.1.14
Date: Sun Aug 26 14:09:30 2018 +0300
2 files changed, 47 insertions(+)
create mode 100644 pkgs/development/python-modules/eth-hash/default.nix
[release-18.09 923a6b2] pythonPackages.eth-typing: init at 1.1.0
Date: Sun Aug 26 14:11:02 2018 +0300
2 files changed, 37 insertions(+)
create mode 100644 pkgs/development/python-modules/eth-typing/default.nix
[release-18.09 d72f81e] pythonPackages.eth-utils: init at 1.2.0
Date: Sun Aug 26 14:12:06 2018 +0300
2 files changed, 37 insertions(+)
create mode 100644 pkgs/development/python-modules/eth-utils/default.nix
[release-18.09 832dcfd] pythonPackages.rlp: fix build
Date: Sun Aug 26 14:13:24 2018 +0300
1 file changed, 13 insertions(+), 3 deletions(-)
[release-18.09 f8c8a23] pythonPackages.python-u2flib-host: init at 3.0.3
Date: Sun Aug 26 15:15:27 2018 +0300
2 files changed, 25 insertions(+)
create mode 100644 pkgs/development/python-modules/python-u2flib-host/default.nix
[release-18.09 ff6e4d0] pythonPackages.ledgerblue: fix build inputs
Date: Sun Aug 26 15:16:08 2018 +0300
1 file changed, 7 insertions(+), 2 deletions(-)
[release-18.09 825f91b] pythonPackage.pymsgbox: init at 1.0.6
Date: Sun Aug 26 15:43:42 2018 +0300
2 files changed, 26 insertions(+)
create mode 100644 pkgs/development/python-modules/pymsgbox/default.nix
[release-18.09 91f488c] pythonPackage.backports-shutil-which: init at 3.5.1
Date: Sun Aug 26 15:44:16 2018 +0300
2 files changed, 23 insertions(+)
create mode 100644 pkgs/development/python-modules/backports-shutil-which/default.nix
[release-18.09 ed0fc7d] pythonPackages.libagent: fix propagatedBuildInputs
Date: Sun Aug 26 15:44:38 2018 +0300
1 file changed, 5 insertions(+), 9 deletions(-)
[release-18.09 29f4541] pythonPackage.backports-shutil-which: enable tests
Date: Mon Aug 27 10:35:54 2018 +0100
1 file changed, 6 insertions(+), 3 deletions(-)

Pushed for ZHF #45960

@samueldr samueldr removed the 9.needs: port to stable A PR needs a backport to the stable release. label Apr 17, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

python36Packages.rlp fails to build
7 participants