arrow-cpp: 8.0.0 -> 9.0.0 by cpcloud · Pull Request #185124 · NixOS/nixpkgs

cpcloud · 2022-08-04T10:56:20Z

Description of changes

Bump arrow from 8 to 9.

Things done

pkgs/development/libraries/arrow-cpp/thrift.patch

cpcloud · 2022-08-04T11:20:02Z

pkgs/development/libraries/arrow-cpp/default.nix

This was necessary to prevent automatic discovery of THRIFT_INCLUDE_DIR as ${thrift}/include/thrift. I'm not sure how this ever worked given that the invocation of find_path didn't change between 8.0.0 and 9.0.0.

pkgs/development/libraries/arrow-cpp/default.nix

cpcloud · 2022-08-04T12:06:01Z

I'm fixing up the pyarrow tests, after which I'll run nixpkgs-review.

cpcloud · 2022-08-04T12:35:28Z

Looks like x86_64 darwin is timing out

pkgs/development/python-modules/pyarrow/default.nix

cpcloud · 2022-08-04T16:55:10Z

Now darwin is building but all of the tests are failing with what appears to be SIGILLs.

veprbl · 2022-08-04T19:22:15Z

On x86_64-darwin:

@nix { "action": "setPhase", "phase": "pythonImportsCheckPhase" }
pythonImportsCheckPhase
Executing pythonImportsCheckPhase
Check whether the following modules can be imported: pyarrow pyarrow.compute pyarrow.csv pyarrow.dataset pyarrow.feather pyarrow.flight pyarrow.fs pyarrow.hdfs pyarrow.json pyarrow.parquet
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "<string>", line 1, in <lambda>
  File "/nix/store/rs23kn5r3qm96cq26b6bzy2w7rs5r39d-python3-3.10.5/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/nix/store/kmkm20pbw58141x7sj9x9d7lfs9ifvbf-python3.10-pyarrow-9.0.0/lib/python3.10/site-packages/pyarrow/__init__.py", line 65, in <module>
    import pyarrow.lib as _lib
ModuleNotFoundError: No module named 'pyarrow.lib'

@GrahamcOfBorg build python3Packages.pyarrow

smancill · 2022-08-05T05:25:36Z

Ah, sorry, I force pushed the branch with a squash and the fix to pyarrow thinking it would go my fork. Having these new permissions is confusing me.

cpcloud · 2022-08-08T09:23:37Z

@smancill @risicle @veprbl @SuperSandro2000 Anything else needed to merge this?

veprbl · 2022-08-08T14:00:02Z

Thank you for the hard work everyone, and @cpcloud especially!

cpcloud · 2022-08-08T14:43:10Z

Thanks for all the review and testing y'all!

smancill · 2022-08-09T01:33:13Z

So, who merges this?

veprbl · 2022-08-09T13:36:21Z

So, who merges this?

You can, I believe

nixos-discourse · 2022-08-10T18:31:38Z

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/prs-already-reviewed/2617/576

cpcloud · 2022-08-10T19:25:54Z

@smancill Any chance you can merge this?

mweinelt · 2022-08-10T23:53:25Z

Result of nixpkgs-review pr 185124 run on x86_64-linux 1

7 packages marked as broken and skipped:

intensity-normalization
python310Packages.caffe
python310Packages.caffeWithCuda
python310Packages.intensity-normalization
python310Packages.rl-coach
python39Packages.intensity-normalization
python39Packages.rl-coach

37 packages failed to build:

napari (python310Packages.napari)
parquet-tools
python310Packages.apache-beam
python310Packages.aplpy
python310Packages.dalle-mini
python310Packages.dask-gateway
python310Packages.dask-glm
python310Packages.dask-jobqueue
python310Packages.dask-ml
python310Packages.dm-sonnet
python310Packages.ibis-framework
python310Packages.pyregion
python310Packages.qiskit
python310Packages.rlax
python310Packages.spacy-transformers
python310Packages.spectral-cube
python310Packages.streamz
python310Packages.tensorflow-datasets
python310Packages.vqgan-jax
python39Packages.apache-beam
python39Packages.aplpy
python39Packages.dalle-mini
python39Packages.dask-gateway
python39Packages.dask-glm
python39Packages.dask-jobqueue
python39Packages.dask-ml
python39Packages.dm-sonnet
python39Packages.ibis-framework
python39Packages.napari
python39Packages.pyregion
python39Packages.qiskit
python39Packages.rlax
python39Packages.spacy-transformers
python39Packages.spectral-cube
python39Packages.streamz
python39Packages.tensorflow-datasets
python39Packages.vqgan-jax

116 packages built:

arrow-cpp
arrow-glib
home-assistant-component-tests.environment_canada
python310Packages.asdf
python310Packages.astroquery
python310Packages.batchgenerators
python310Packages.boxx
python310Packages.bpycv
python310Packages.bsuite
python310Packages.casa-formats-io
python310Packages.clifford
python310Packages.dask
python310Packages.dask-image
python310Packages.dask-mpi
python310Packages.dask-yarn
python310Packages.datafusion
python310Packages.datasets
python310Packages.datashader
python310Packages.db-dtypes
python310Packages.devito
python310Packages.distributed
python310Packages.dremel3dpy
python310Packages.env-canada
python310Packages.glymur
python310Packages.google-cloud-bigquery
python310Packages.handout
python310Packages.image-match
python310Packages.imagecorruptions
python310Packages.imageio
python310Packages.imgaug
python310Packages.intake
python310Packages.intake-parquet
python310Packages.mask-rcnn
python310Packages.moviepy
python310Packages.napari-console
python310Packages.napari-svg
python310Packages.pims
python310Packages.pyarrow
python310Packages.pyfftw
python310Packages.pytest-astropy
python310Packages.pytest-astropy-header
python310Packages.pyvo
python310Packages.qiskit-machine-learning
python310Packages.reproject
python310Packages.runway-python
python310Packages.scikitimage
python310Packages.slicedimage
python310Packages.sparse
python310Packages.stumpy
python310Packages.stytra
python310Packages.sunpy
python310Packages.tensorly
python310Packages.test-tube
python310Packages.tifffile
python310Packages.tokenizers
python310Packages.transformers
python310Packages.zcs
python39Packages.asdf
python39Packages.astroquery
python39Packages.batchgenerators
python39Packages.boxx
python39Packages.bpycv
python39Packages.bsuite
python39Packages.caffe
python39Packages.caffeWithCuda
python39Packages.casa-formats-io
python39Packages.clifford
python39Packages.dask
python39Packages.dask-image
python39Packages.dask-mpi
python39Packages.dask-yarn
python39Packages.datafusion
python39Packages.datasets
python39Packages.datashader
python39Packages.db-dtypes
python39Packages.devito
python39Packages.distributed
python39Packages.dremel3dpy
python39Packages.env-canada
python39Packages.glymur
python39Packages.google-cloud-bigquery
python39Packages.handout
python39Packages.image-match
python39Packages.imagecorruptions
python39Packages.imageio
python39Packages.imgaug
python39Packages.intake
python39Packages.intake-parquet
python39Packages.mask-rcnn
python39Packages.moviepy
python39Packages.napari-console
python39Packages.napari-svg
python39Packages.pims
python39Packages.pyarrow
python39Packages.pyfftw
python39Packages.pytest-astropy
python39Packages.pytest-astropy-header
python39Packages.pyvo
python39Packages.qiskit-machine-learning
python39Packages.reproject
python39Packages.runway-python
python39Packages.scikitimage
python39Packages.slicedimage
python39Packages.sparse
python39Packages.stumpy
python39Packages.stytra
python39Packages.sunpy
python39Packages.tensorly
python39Packages.test-tube
python39Packages.tifffile
python39Packages.tokenizers
python39Packages.transformers
python39Packages.zcs
streamlit
tartube
tartube-yt-dlp

cpcloud mentioned this pull request Aug 4, 2022

grpc: 1.47.0 -> 1.48.0 #182157

Merged

13 tasks

cpcloud commented Aug 4, 2022

View reviewed changes

pkgs/development/libraries/arrow-cpp/thrift.patch Outdated Show resolved Hide resolved

cpcloud force-pushed the arrow-9 branch from 3982a6f to 3b75aa1 Compare August 4, 2022 10:57

cpcloud mentioned this pull request Aug 4, 2022

chore(deps): update dependency pyarrow to v9 ibis-project/ibis#4315

Merged

1 task

ofborg bot requested review from tobim and veprbl August 4, 2022 11:06

cpcloud commented Aug 4, 2022

View reviewed changes

SuperSandro2000 reviewed Aug 4, 2022

View reviewed changes

pkgs/development/libraries/arrow-cpp/default.nix Outdated Show resolved Hide resolved

cpcloud requested review from FRidh and jonringer as code owners August 4, 2022 12:37

github-actions bot added the 6.topic: python Python is a high-level, general-purpose programming language. label Aug 4, 2022

cpcloud commented Aug 4, 2022

View reviewed changes

pkgs/development/python-modules/pyarrow/default.nix Outdated Show resolved Hide resolved

cpcloud mentioned this pull request Aug 4, 2022

chore(deps/nixpkgs): update ibis-project/ibis#4312

Merged

SuperSandro2000 mentioned this pull request Aug 4, 2022

arrow-cpp build failure #185082

Closed

vcunat linked an issue Aug 4, 2022 that may be closed by this pull request

arrow-cpp build failure #185082

Closed

cpcloud force-pushed the arrow-9 branch from 3a8de9f to 3d12e48 Compare August 4, 2022 13:25

ofborg bot requested a review from SuperSandro2000 August 4, 2022 13:42

cpcloud force-pushed the arrow-9 branch from b9e8b12 to e4457d0 Compare August 4, 2022 17:04

ofborg bot requested a review from ndl August 4, 2022 17:13

veprbl force-pushed the arrow-9 branch from 14e5c37 to b95fb46 Compare August 4, 2022 18:12

smancill force-pushed the arrow-9 branch from b95fb46 to b2798df Compare August 5, 2022 05:23

smancill force-pushed the arrow-9 branch from b2798df to b95fb46 Compare August 5, 2022 05:27

cpcloud and others added 10 commits August 8, 2022 05:21

arrow-cpp: 8.0.0 -> 9.0.0

c44d47d

arrow-cpp: remove unused jemalloc input

cf192f5

arrow-cpp: add commentary for vendored jemalloc and mimalloc

56e9e52

python3Packages.pyarrow: get tests working with arrow-cpp 9.0.0

829e7c9

python3Packages.db-dtypes: bump pyarrow upper bound

941d9e9

python3Packages.apache-beam: remove numpy constraint

4b664ed

arrow-cpp: fix failing tests on darwin sandbox

bf34891

python3Packages.pyarrow: fix failing tests on darwin sandbox

574b363

python3Packages.google-cloud-bigquery: bump pyarrow constraint

8b40f59

python3Packages.google-cloud-bigquery: ignore network-based test

e41f448

cpcloud added 2 commits August 8, 2022 05:30

python3Packages.apache-beam: unrestrict pymongo dependency

d9202ab

python3Packages.apache-beam: set enableParallelBuilding to true

21237ee

cpcloud force-pushed the arrow-9 branch from a28504a to 21237ee Compare August 8, 2022 09:32

bobby285271 removed the 12.approvals: 1 This PR was reviewed and approved by one person. label Aug 8, 2022

veprbl approved these changes Aug 8, 2022

View reviewed changes

bobby285271 added 12.approvals: 2 This PR was reviewed and approved by two persons. 12.approved-by: package-maintainer This PR was reviewed and approved by a maintainer listed in any of the changed packages. labels Aug 9, 2022

python3Packages.dask-gateway: fix build

7d79dd8

ofborg bot requested review from costrouc and veprbl August 10, 2022 23:27

mweinelt merged commit 6077031 into NixOS:master Aug 11, 2022

cpcloud deleted the arrow-9 branch August 11, 2022 09:12

Uh oh!

Conversation

cpcloud commented Aug 4, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description of changes

Things done

Uh oh!

Uh oh!

cpcloud Aug 4, 2022

Choose a reason for hiding this comment

Uh oh!

Uh oh!

cpcloud commented Aug 4, 2022

Uh oh!

cpcloud commented Aug 4, 2022

Uh oh!

Uh oh!

cpcloud commented Aug 4, 2022

Uh oh!

veprbl commented Aug 4, 2022

Uh oh!

smancill commented Aug 5, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cpcloud commented Aug 8, 2022

Uh oh!

veprbl commented Aug 8, 2022

Uh oh!

cpcloud commented Aug 8, 2022

Uh oh!

smancill commented Aug 9, 2022

Uh oh!

veprbl commented Aug 9, 2022

Uh oh!

nixos-discourse commented Aug 10, 2022

Uh oh!

cpcloud commented Aug 10, 2022

Uh oh!

mweinelt commented Aug 10, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

cpcloud commented Aug 4, 2022 •

edited

Loading

smancill commented Aug 5, 2022 •

edited

Loading