Skip to content

Comments

build: update dependencies#282

Merged
mbrobbel merged 7 commits intosubstrait-io:mainfrom
wackywendell:dep-update
Nov 4, 2024
Merged

build: update dependencies#282
mbrobbel merged 7 commits intosubstrait-io:mainfrom
wackywendell:dep-update

Conversation

@wackywendell
Copy link
Contributor

@wackywendell wackywendell commented Oct 28, 2024

This PR updates the prost{-*}, pyo3, and cbindgen dependencies. pyo3 in particular was causing a build error in other PRs. This PR should now pass all CI.

Notes

In addition to updating the dependency versions and the code that calls it where necessary, I also found that protoc was missing in a number of CI jobs where it seemed to be needed. I'm not sure how this worked before; my best guess was that the generated code was cached and carried over, and updating prost changed that.

Build Error

in particular, this is intended to fix this error:

error: failed to run custom build command for `pyo3-ffi v0.21.2`

Caused by:
  process didn't exit successfully: `/Users/runner/work/substrait-validator/substrait-validator/target/debug/build/pyo3-ffi-77caf4a9541fc18e/build-script-build` (exit status: 1)
  --- stdout
  cargo:rerun-if-env-changed=PYO3_CROSS
  cargo:rerun-if-env-changed=PYO3_CROSS_LIB_DIR
  cargo:rerun-if-env-changed=PYO3_CROSS_PYTHON_VERSION
  cargo:rerun-if-env-changed=PYO3_CROSS_PYTHON_IMPLEMENTATION
  cargo:rerun-if-env-changed=PYO3_PRINT_CONFIG
  cargo:rerun-if-env-changed=PYO3_USE_ABI3_FORWARD_COMPATIBILITY

  --- stderr
  error: the configured Python interpreter version (3.13) is newer than PyO3's maximum supported version (3.12)
  = help: please check if an updated version of PyO3 is available. Current version: 0.21.2
  = help: set PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 to suppress this check and build anyway using the stable ABI
warning: build failed, waiting for other jobs to finish...

@wackywendell wackywendell force-pushed the dep-update branch 2 times, most recently from 45dfded to c3fa31d Compare October 28, 2024 15:56
@wackywendell wackywendell marked this pull request as ready for review October 28, 2024 18:27
@wackywendell
Copy link
Contributor Author

Ah! I just found a comment explaining why the protoc behavior changed:

In order to rely on as few external dependencies as possible, all protoc
invocations by the various parts of the build invoke the protoc executable
as found/compiled and exposed by prost-build. That is: this
protoc is also abused to generate Python bindings. Unfortunately, prost-build
is planning to remove the build
logic for protoc at the time of writing (and who can blame them), so this will
need to be done differently in the future.

Well, I'm doing it here! I'll add a commit updating that comment.

manywheels is needed for building wheels for many linux versions, so we need it;
put it back. But include protoc in the container to fix the build
@mbrobbel mbrobbel merged commit 3b47781 into substrait-io:main Nov 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants