Skip to content

GH-43702: [C++][FS][Azure] Use the latest Azurite and update the bundled Azure SDK for C++ to azure-identity_1.9.0 #43723

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
kou merged 2 commits into from
Aug 17, 2024
Merged

GH-43702: [C++][FS][Azure] Use the latest Azurite and update the bundled Azure SDK for C++ to azure-identity_1.9.0 #43723

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
ee412c7
GH-43702: [C++][FS][Azure] Update the bundled Azure SDK for C++ to az…
kou Aug 16, 2024
3aa3bd6
Use --skipApiVersionCheck
kou Aug 16, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 18 additions & 6 deletions ci/scripts/install_azurite.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,20 +19,32 @@

set -e

# Pin azurite to 3.29.0 due to https://github.com/apache/arrow/issues/41505
node_version="$(node --version)"
echo "node version = ${node_version}"

case "${node_version}" in
v12*)
# Pin azurite to 3.29.0 due to https://github.com/apache/arrow/issues/41505
azurite_version=v3.29.0
;;
*)
azurite_version=latest
;;
esac

case "$(uname)" in
Darwin)
npm install -g azurite@v3.29.0
npm install -g azurite@${azurite_version}
which azurite
;;
MINGW*)
choco install nodejs.install
npm install -g azurite@v3.29.0
npm install -g azurite@${azurite_version}
;;
Linux)
npm install -g azurite@v3.29.0
npm install -g azurite@${azurite_version}
which azurite
;;
esac
echo "node version = $(node --version)"
echo "azurite version = $(azurite --version)"

echo "azurite version = $(azurite --version)"
5 changes: 4 additions & 1 deletion cpp/src/arrow/filesystem/azurefs_test.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -198,7 +198,10 @@ class AzuriteEnv : public AzureEnvImpl<AzuriteEnv> {
self->temp_dir_->path().Join("debug.log"));
auto server_process = bp::child(
boost::this_process::environment(), exe_path, "--silent", "--location",
self->temp_dir_->path().ToString(), "--debug", self->debug_log_path_.ToString());
self->temp_dir_->path().ToString(), "--debug", self->debug_log_path_.ToString(),
// For old Azurite. We can't install the latest Azurite with
// old Node.js on old Ubuntu.
"--skipApiVersionCheck");
if (!server_process.valid() || !server_process.running()) {
server_process.terminate();
server_process.wait();
Expand Down
4 changes: 2 additions & 2 deletions cpp/thirdparty/versions.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,8 +54,8 @@ ARROW_AWS_LC_BUILD_SHA256_CHECKSUM=ae96a3567161552744fc0cae8b4d68ed88b1ec0f3d3c9
ARROW_AWSSDK_BUILD_VERSION=1.10.55
ARROW_AWSSDK_BUILD_SHA256_CHECKSUM=2d552fb1a84bef4a9b65e34aa7031851ed2aef5319e02cc6e4cb735c48aa30de
# Despite the confusing version name this is still the whole Azure SDK for C++ including core, keyvault, storage-common, etc.
ARROW_AZURE_SDK_BUILD_VERSION=azure-core_1.10.3
ARROW_AZURE_SDK_BUILD_SHA256_CHECKSUM=dd624c2f86adf474d2d0a23066be6e27af9cbd7e3f8d9d8fd7bf981e884b7b48
ARROW_AZURE_SDK_BUILD_VERSION=azure-identity_1.9.0
ARROW_AZURE_SDK_BUILD_SHA256_CHECKSUM=97065bfc971ac8df450853ce805f820f52b59457bd7556510186a1569502e4a1
ARROW_BOOST_BUILD_VERSION=1.81.0
ARROW_BOOST_BUILD_SHA256_CHECKSUM=9e0ffae35528c35f90468997bc8d99500bf179cbae355415a89a600c38e13574
ARROW_BROTLI_BUILD_VERSION=v1.0.9
Expand Down
3 changes: 3 additions & 0 deletions python/pyarrow/tests/conftest.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -263,6 +263,9 @@ def azure_server(tmpdir_factory):
tmpdir = tmpdir_factory.getbasetemp()
# We only need blob service emulator, not queue or table.
args = ['azurite-blob', "--location", tmpdir, "--blobPort", str(port)]
# For old Azurite. We can't install the latest Azurite with old
# Node.js on old Ubuntu.
args += ["--skipApiVersionCheck"]
proc = None
try:
proc = subprocess.Popen(args, env=env)
Expand Down