Skip to content

GH-14993: [CI][Conda] Fix missing RECIPE_ROOT variable now expected by conda build #15014

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
kou merged 6 commits into from
Dec 20, 2022
Merged

GH-14993: [CI][Conda] Fix missing RECIPE_ROOT variable now expected by conda build #15014

Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
f3b77cc
fix missing RECIPE_ROOT variable now expected by conda build
h-vetinari Dec 18, 2022
6efeeec
Don't build libarrow from source
xhochy Dec 18, 2022
e04c9dc
Debug
kou Dec 19, 2022
b26892d
Use BUILD_PREFIX
kou Dec 19, 2022
3b9c72d
"pkg-config --cflags" may be empty
kou Dec 19, 2022
f018c90
Remove needless codes
kou Dec 19, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 5 additions & 4 deletions dev/tasks/conda-recipes/build_steps.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,31 +10,32 @@
# benefit from the improvement.

set -xeuo pipefail
export FEEDSTOCK_ROOT="${FEEDSTOCK_ROOT:-/home/conda/feedstock_root}"

output_dir=${1}

export PYTHONUNBUFFERED=1
export FEEDSTOCK_ROOT="${FEEDSTOCK_ROOT:-/home/conda/feedstock_root}"
export RECIPE_ROOT="${RECIPE_ROOT:-/home/conda/recipe_root}"
export CI_SUPPORT="${FEEDSTOCK_ROOT}/.ci_support"
export CONFIG_FILE="${CI_SUPPORT}/${CONFIG}.yaml"

cat >~/.condarc <<CONDARC

conda-build:
root-dir: ${output_dir}
root-dir: ${output_dir}

CONDARC

mamba install --update-specs --yes --quiet "conda-forge-ci-setup=3" conda-build pip boa -c conda-forge
mamba update --update-specs --yes --quiet "conda-forge-ci-setup=3" conda-build pip boa -c conda-forge

# set up the condarc
setup_conda_rc "${FEEDSTOCK_ROOT}" "${FEEDSTOCK_ROOT}" "${CONFIG_FILE}"
setup_conda_rc "${FEEDSTOCK_ROOT}" "${RECIPE_ROOT}" "${CONFIG_FILE}"

source run_conda_forge_build_setup

# make the build number clobber
make_build_number "${FEEDSTOCK_ROOT}" "${FEEDSTOCK_ROOT}" "${CONFIG_FILE}"
make_build_number "${FEEDSTOCK_ROOT}" "${RECIPE_ROOT}" "${CONFIG_FILE}"

if [[ "${HOST_PLATFORM}" != "${BUILD_PLATFORM}" ]] && [[ "${HOST_PLATFORM}" != linux-* ]] && [[ "${BUILD_WITH_CONDA_DEBUG:-0}" != 1 ]]; then
EXTRA_CB_OPTIONS="${EXTRA_CB_OPTIONS:-} --no-test"
Expand Down
3 changes: 2 additions & 1 deletion dev/tasks/conda-recipes/r-arrow/build.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,14 @@ export DISABLE_AUTOBREW=1

# arrow uses C++17
export ARROW_R_CXXFLAGS="${ARROW_R_CXXFLAGS} -std=c++17"
export LIBARROW_BUILD=false

if [[ "${target_platform}" == osx-* ]]; then
# See https://conda-forge.org/docs/maintainer/knowledge_base.html#newer-c-features-with-old-sdk
export ARROW_R_CXXFLAGS="${ARROW_R_CXXFLAGS} -D_LIBCPP_DISABLE_AVAILABILITY"
fi

export PKG_CONFIG_PATH="${PREFIX}/lib/pkgconfig${PKG_CONFIG_PATH:+:${PKG_CONFIG_PATH}}"
export PKG_CONFIG_PATH="${BUILD_PREFIX}/lib/pkgconfig${PKG_CONFIG_PATH:+:${PKG_CONFIG_PATH}}"

@xhochy xhochy Dec 19, 2022

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't look right. BUILD_PREFIX is the build system, not host. It has been failing in the cross-compiling case and this change makes me suspicious whether we then link to the wrong libraries.

@kou kou Dec 19, 2022

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, sorry. This is not related to fix the R problem.
We may be able remove this line entirely. I'll try it.


# ${R_ARGS} necessary to support cross-compilation
${R} CMD INSTALL --build r/. ${R_ARGS}
2 changes: 2 additions & 0 deletions dev/tasks/conda-recipes/run_docker_build.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ build_dir=${1}
THISDIR="$( cd "$( dirname "$0" )" >/dev/null && pwd )"
ARROW_ROOT=$(cd "$THISDIR/../../.."; pwd;)
FEEDSTOCK_ROOT=$THISDIR
RECIPE_ROOT=$THISDIR

docker info

Expand Down Expand Up @@ -64,6 +65,7 @@ docker run ${DOCKER_RUN_ARGS} \
-v "${ARROW_ROOT}":/arrow:rw,z \
-v "${build_dir}":/build:rw \
-e FEEDSTOCK_ROOT="/arrow/dev/tasks/conda-recipes" \
-e RECIPE_ROOT="/arrow/dev/tasks/conda-recipes" \
-e CONFIG \
-e R_CONFIG \
-e HOST_USER_ID \
Expand Down
4 changes: 3 additions & 1 deletion r/configure
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,8 @@
# INCLUDE_DIR and LIB_DIR manually via e.g:
# R CMD INSTALL --configure-vars='INCLUDE_DIR=/.../include LIB_DIR=/.../lib'

set -x

# Library settings
PKG_CONFIG_NAME="arrow"
PKG_DEB_NAME="(unsuppored)"
Expand Down Expand Up @@ -104,7 +106,7 @@ else
PKGCONFIG_DIRS=`pkg-config --libs-only-L --silence-errors ${PKG_CONFIG_NAME}`
fi

if [ "$PKGCONFIG_CFLAGS" ] && [ "$PKGCONFIG_LIBS" ]; then
if [ "$PKGCONFIG_LIBS" != "" ]; then
FOUND_LIB_DIR=`echo $PKGCONFIG_DIRS | sed -e 's/^-L//'`
echo "*** Arrow C++ libraries found via pkg-config at $FOUND_LIB_DIR"
PKG_CFLAGS="$PKGCONFIG_CFLAGS $PKG_CFLAGS"
Expand Down