-
Notifications
You must be signed in to change notification settings - Fork 299
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Delete and withdraw zetasql #34096
Delete and withdraw zetasql #34096
Conversation
Gen AI suggestions to solve the build error: Based on the error output, there are a few issues to address:
Suggestions to fix:
# When in interactive mode
ls -R | grep BUILD # List all BUILD files
resources:
cpu: 33
memory: 100Gi These are quite high but might be necessary for Bazel builds.
This can be ignored for now as it's not blocking the build process. Next steps:
To recreate the patch if needed: git diff > new-multi-arch-for-linux.patch Focus on fixing the patch file paths first, as this is the immediate cause of the build failure. |
Important previous context here: #26865 |
Gen AI suggestions to solve the build error: • Detected Error:
• Error Category: Build Configuration • Failure Point: LLVM toolchain setup in Bazel build system • Root Cause Analysis: The LLVM toolchain rules don't recognize Wolfi OS as a supported Linux distribution for building. This is common when using newer or non-mainstream distributions with build tools that have hard-coded OS checks. • Suggested Fix: environment:
contents:
packages:
- llvm15
- clang15
environment:
BAZEL_ARGS: "--config=g++ --sandbox_add_mount_pair=/usr/lib/llvm15" • Explanation:
• Additional Notes:
• References:
|
Gen AI suggestions to solve the build error: • Detected Error: "Unsupported linux distribution and version: wolfi, 20230201" • Error Category: Build Configuration • Failure Point: LLVM toolchain repository setup during Bazel build • Root Cause Analysis: The LLVM toolchain build rules don't recognize Wolfi OS as a supported distribution. The toolchain is attempting to detect the host OS and failing because Wolfi isn't in its list of known distributions. • Suggested Fix: environment:
contents:
packages:
# ... existing packages ...
- llvm15
- llvm15-dev
environment:
# ... existing env vars ...
BAZEL_ARGS: "--config=g++ --sandbox_add_mount_pair=/usr/lib/llvm15 --action_env=LLVM_COMMIT=llvm-15 --action_env=DISTRIBUTION=debian --action_env=DISTRIBUTION_VERSION=11" • Explanation:
• Additional Notes:
• References: |
Gen AI suggestions to solve the build error: • Detected Error: "Unsupported linux distribution and version: wolfi, 20230201" • Error Category: Configuration/Build System • Failure Point: LLVM toolchain repository initialization during Bazel build • Root Cause Analysis: The LLVM toolchain build rules don't recognize Wolfi OS as a supported distribution. The toolchain is trying to detect the host OS for building LLVM components but fails because Wolfi isn't in its list of known distributions. • Suggested Fix: pipeline:
- uses: git-checkout
with:
repository: https://github.com/google/zetasql
tag: ${{package.version}}
expected-commit: a516c6b26d183efc4f56293256bba92e243b7a61
- uses: patch
with:
patches: multi-arch-for-linux.patch
- runs: |
cd zetasql
bazel build --verbose_failures ${BAZEL_ARGS} $EXTRA_BAZEL_ARGS \
--action_env=BAZEL_DO_NOT_DETECT_CPP_TOOLCHAIN=1 \
--host_platform=@local_config_platform//:host \
--platforms=@local_config_platform//:host \
-c opt ... • Explanation: • Additional Notes:
• References:
|
Gen AI suggestions to solve the build error: • Detected Error:
• Error Category: Dependency/Version • Failure Point: Bazel server startup during build process • Root Cause Analysis: • Suggested Fix: environment:
contents:
packages:
- openjdk-21
- openjdk-21-default-jvm
# ... other packages ...
environment:
JAVA_HOME: /usr/lib/jvm/java-21-openjdk • Explanation: • Additional Notes:
• References:
|
Gen AI suggestions to solve the build error: • Detected Error: "Unsupported linux distribution and version: wolfi, 20230201" • Error Category: Configuration/Dependency • Failure Point: Bazel's LLVM toolchain configuration during the build process • Root Cause Analysis: • Suggested Fix: environment:
contents:
packages:
- llvm15
- llvm15-dev
- clang15
# Add existing packages...
pipeline:
- runs: |
cd zetasql
bazel build --verbose_failures ${BAZEL_ARGS} $EXTRA_BAZEL_ARGS \
--action_env=CC=/usr/bin/clang \
--action_env=CXX=/usr/bin/clang++ \
--action_env=LLVM_CONFIG=/usr/bin/llvm-config \
-c opt ... • Explanation: • Additional Notes:
• References:
|
Gen AI suggestions to solve the build error: • Detected Error: "Unsupported linux distribution and version: wolfi, 20230201" • Error Category: Build Configuration • Failure Point: LLVM toolchain repository fetch during Bazel build • Root Cause Analysis: The LLVM toolchain build rule doesn't recognize Wolfi as a supported Linux distribution. This is common with build systems that have hard-coded distribution checks. • Suggested Fix:
environment:
BAZEL_ARGS: "--config=g++"
EXTRA_BAZEL_ARGS: "--tool_java_runtime_version=local_jdk"
JAVA_HOME: /usr/lib/jvm/java-17-openjdk
# Add these lines:
CC: clang
CXX: clang++
BAZEL_LLVM: "/usr"
- runs: |
cd zetasql
# Configure to use system LLVM
echo "build --action_env=BAZEL_LLVM=/usr" >> .bazelrc
echo "build --action_env=CC=clang" >> .bazelrc
echo "build --action_env=CXX=clang++" >> .bazelrc
bazel build --verbose_failures ${BAZEL_ARGS} $EXTRA_BAZEL_ARGS -c opt ... • Explanation:
• Additional Notes:
• References:
|
f7720ff
to
1917b4a
Compare
Gen AI suggestions to solve the build error: • Detected Error:
• Error Category: Build Configuration/Dependency • Failure Point: • Root Cause Analysis: • Suggested Fix: environment:
contents:
packages:
- bash
- bazel-6
- binutils
- build-base
- busybox
- ca-certificates-bundle
- gcc-12
- git
- openjdk-17
- openjdk-17-default-jvm
- openssf-compiler-options
- patch
- python3
- tzdata
- wolfi-baselayout
- llvm15
- llvm15-dev
- clang15 • Explanation: • Additional Notes:
• References:
|
👋 @hbh7 - As we are implementing the new interrupts/escalation process, would you mind adding and filling out the escalation template on this one? |
e6cebff
to
98b202f
Compare
I've pushed all the fixes I have so far, which I believe resolve the many issues building
|
98b202f
to
f87fed2
Compare
I have it building on x86. But when I went to test it, I found the resulting apk to be empty. And guess what? It always has been: Line 47 in dbdbb84
Why is it here? Should we just remove it? |
Confirmed not needed w/ @Dentrax, who originally added it. |
Let's also add this to |
473b1f8
to
8966801
Compare
Should we? I understood that to be reserved for packages that are somehow negatively impacting the archive. |
Signed-off-by: wolfi-bot <[email protected]>
Signed-off-by: hectorj2f <[email protected]>
See: google/zetasql#157 Signed-off-by: dann frazier <[email protected]>
bazel uses @toolchains_llvm to identify the OS and find a compatible llvm_toolchain to download. wolfi is not yet supported. Upstream uses Ubuntu 18.04 to do their docker-based builds. We can't use the exact same toolchain on wolfi, because it needs older system libs to run. Let's use the latest 6.0.x toolchain for the latest available Ubuntu. This is compatible with wolfi, although it does cause the build to spew warnings about our libtinfo versioning that I haven't looked into. Of course, we should be using our own llvm build, not downloading one from the Internet. This should be possible using toolchain_roots. But this will require some setup magic to organize our LLVM/clang files into the expected structure, perhaps by adding some -compat packages. FIXME: Need checksums for verifying the download FIXME: Doesn't provide ARM support Signed-off-by: dann frazier <[email protected]>
This is what upstream uses in their docker-based builds, and seems to be required. With -6, the build fails with: ERROR: /home/build/.cache/bazel/_bazel_build/79a1bfc8c8b5b6a6d226b38d072f165b/external/llvm_toolchain/BUILD.bazel:161:18: in system_module_map rule @llvm_toolchain//:module-x86_64-linux: Traceback (most recent call last): File "/home/build/.cache/bazel/_bazel_build/79a1bfc8c8b5b6a6d226b38d072f165b/external/toolchains_llvm/toolchain/internal/system_module_map.bzl", line 57, column 29, in _system_module_map template_dict.add_joined( Error in add_joined: add_joined() got unexpected keyword argument 'allow_closure' Note that this requires a bump in JDK versions because our bazel-7 uses class file format 65. That requires openjdk >= 21, so let's just bump to the latest stable. Signed-off-by: dann frazier <[email protected]>
zetasql does not build with gcc. Setting this config in upstream's docker-base build also fails. Drop the build-dep on gcc-12, which probably wasn't doing anything anyway because it was not the system default. Signed-off-by: dann frazier <[email protected]>
boost and civetweb include test files with non-latin filenames. Something in the build system is unable to deal with them: WARN ERROR: /home/build/.cache/bazel/_bazel_build/79a1bfc8c8b5b6a6d226b38d072f165b/external/boost/BUILD.bazel:27:12: Foreign Cc - BoostBuild: Building boost failed: error reading file '@boost//:libs/wave/test/testwave/testfiles/utf8-test-ßµ™∃/file.hpp': /home/build/.cache/bazel/_bazel_build/79a1bfc8c8b5b6a6d226b38d072f165b/external/boost/libs/wave/test/testwave/testfiles/utf8-test-ßµ™∃/file.hpp (No such file or directory) WARN ERROR: /home/build/.cache/bazel/_bazel_build/79a1bfc8c8b5b6a6d226b38d072f165b/external/boost/BUILD.bazel:27:12: Foreign Cc - BoostBuild: Building boost failed: 1 input file(s) are in error Note that it is not the project test suites detecting a regression, rather it is part of the bazel build setup. Until this is root caused, work around it by stripping those files out of the build area. Signed-off-by: dann frazier <[email protected]>
The civetweb dependency fails to build on wolfi: [...]/external/civetweb/BUILD.bazel:30:6: output 'external/civetweb/civetweb/lib/libcivetweb.a' was not created [...]/external/civetweb/BUILD.bazel:30:6: output 'external/civetweb/civetweb/lib/libcivetweb-cpp.a' was not created [...]/external/civetweb/BUILD.bazel:30:6: Foreign Cc - CMake: Building civetweb failed: not all outputs were created or valid These files are being built, but are being installed in the wrong directory. Use CMAKE_INSTALL_LIBDIR to tell it where to install things. Signed-off-by: dann frazier <[email protected]>
The apk we produce is empty, and nothing depends on it. Signed-off-by: dann frazier <[email protected]>
Signed-off-by: dann frazier <[email protected]>
8966801
to
6d2f482
Compare
I've gone ahead and added it, I don't see any value in keeping it around. |
anything that is dead dead and kept around has the danger of people installing; or building a production image out of it. So yeah, things that are dead on arrival; should always be withdrawn. |
zetasql
was added experimentally, but never actually produced anything other than an empty package. It's a non-trivial package to maintain inside wolfi, and there's currently no reason to do so.