Skip to content

Add python 3.8/3.9 support for Windows GPU and Linux ARM64 #6615

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
snnn merged 9 commits into from
Feb 12, 2021
Merged

Add python 3.8/3.9 support for Windows GPU and Linux ARM64 #6615

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
08a28f8
add python 3.8/3.9
Feb 9, 2021
60ce5d7
update
Feb 10, 2021
a12d38d
update
Feb 11, 2021
1b0b1e9
csharp
Feb 11, 2021
1cdef07
update
Feb 11, 2021
502e957
update
Feb 11, 2021
4c6b09b
update
Feb 11, 2021
5080256
Ke's fix
liqunfu Feb 11, 2021
697c5cb
Merge remote-tracking branch 'origin/master' into snnn/python1
Feb 11, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 0 additions & 9 deletions cgmanifests/cgmanifest.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,15 +45,6 @@
}
}
},
{
"component": {
"type": "git",
"git": {
"commitHash": "e02b83cc5e3c4d30f93dba945162e3aa58d962d6",
"repositoryUrl": "https://github.com/jemalloc/jemalloc.git"
}
}
},
{
"component": {
"type": "git",
Expand Down
2 changes: 2 additions & 0 deletions csharp/test/Microsoft.ML.OnnxRuntime.EndToEndTests/runtest-docker-gpu.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,8 @@ docker run --gpus all --rm \
-e "PackageName=$PackageName" \
-e "RunTestCsharp=$RunTestCsharp" \
-e "RunTestNative=$RunTestNative" \
-e "BUILD_BINARIESDIRECTORY=/home/onnxruntimedev" \
-e "BUILD_SOURCESDIRECTORY=/onnxruntime_src" \
"$DockerImage" \
/bin/bash /onnxruntime_src/csharp/test/Microsoft.ML.OnnxRuntime.EndToEndTests/runtest.sh \
/home/onnxruntimedev/$NUGET_REPO_DIRNAME /onnxruntime_src /home/onnxruntimedev $CurrentOnnxRuntimeVersion
2 changes: 2 additions & 0 deletions csharp/test/Microsoft.ML.OnnxRuntime.EndToEndTests/runtest-docker.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,8 @@ docker run --rm \
-e "DisableMlOps=$DISABLEMLOPS" \
-e "RunTestCsharp=$RunTestCsharp" \
-e "RunTestNative=$RunTestNative" \
-e "BUILD_BINARIESDIRECTORY=/home/onnxruntimedev" \
-e "BUILD_SOURCESDIRECTORY=/onnxruntime_src" \
"$DockerImage" \
/bin/bash /onnxruntime_src/csharp/test/Microsoft.ML.OnnxRuntime.EndToEndTests/runtest.sh \
/home/onnxruntimedev/$NUGET_REPO_DIRNAME /onnxruntime_src /home/onnxruntimedev $CurrentOnnxRuntimeVersion
21 changes: 12 additions & 9 deletions csharp/test/Microsoft.ML.OnnxRuntime.EndToEndTests/runtest.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@
# Licensed under the MIT License.

LocalNuGetRepo=$1
SourceRoot=$2
Copy link
Contributor

@edgchen1 edgchen1 Feb 12, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not keep this parameter and pass the correct value? seems better to not depend on azure devops variable names like BUILD_SOURCESDIRECTORY outside of build definitions.

Copy link
Contributor

@edgchen1 edgchen1 Feb 12, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could add an additional parameter for the binaries directory

Copy link
Contributor Author

@snnn snnn Feb 12, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, then I need to add another parameter, which requires modify more files(all the callers of this script). That's why I choose this.

Copy link
Contributor Author

@snnn snnn Feb 12, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed I would prefer to pass less parameters when possible. The more we have, the more likely we could make some errors during the passing, for example, missing one parameter or the order is wrong.

Copy link
Contributor

@edgchen1 edgchen1 Feb 12, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that's what error handling is for :)
or named command line options
can handle it later

Copy link
Contributor Author

@snnn snnn Feb 12, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I forgot to mention, sometimes the file is called from docker, sometimes it runs directly in the host environment. If we use the azure devops variable names directly, then in the first case we need to pass it, in the second case we don't and we can directly use the vars.

BuildDir=$3
export CurrentOnnxRuntimeVersion=$4
IsMacOS=${5:-false}
Expand All @@ -12,22 +11,26 @@ PackageName=${PackageName:-Microsoft.ML.OnnxRuntime}
RunTestCsharp=${RunTestCsharp:-true}
RunTestNative=${RunTestNative:-true}

set -x
set -x -e

OldDir=`pwd`
cd $SourceRoot
cd $BUILD_SOURCESDIRECTORY

echo "Current NuGet package version is $CurrentOnnxRuntimeVersion"

if [ $RunTestCsharp = "true" ]; then
if [[ $IsMacOS == "True" || $IsMacOS == "true" ]]; then
mkdir -p $BUILD_BINARIESDIRECTORY/models
ln -s $BUILD_SOURCESDIRECTORY/cmake/external/onnx/onnx/backend/test/data/node $BUILD_BINARIESDIRECTORY/models/opset14
fi
# Run C# tests
dotnet restore $SourceRoot/csharp/test/Microsoft.ML.OnnxRuntime.EndToEndTests/Microsoft.ML.OnnxRuntime.EndToEndTests.csproj -s $LocalNuGetRepo -s https://api.nuget.org/v3/index.json
dotnet restore $BUILD_SOURCESDIRECTORY/csharp/test/Microsoft.ML.OnnxRuntime.EndToEndTests/Microsoft.ML.OnnxRuntime.EndToEndTests.csproj -s $LocalNuGetRepo -s https://api.nuget.org/v3/index.json
if [ $? -ne 0 ]; then
echo "Failed to restore nuget packages for the test project"
exit 1
fi

dotnet test $SourceRoot/csharp/test/Microsoft.ML.OnnxRuntime.EndToEndTests/Microsoft.ML.OnnxRuntime.EndToEndTests.csproj --no-restore --verbosity detailed
dotnet test $BUILD_SOURCESDIRECTORY/csharp/test/Microsoft.ML.OnnxRuntime.EndToEndTests/Microsoft.ML.OnnxRuntime.EndToEndTests.csproj --no-restore --verbosity detailed
if [ $? -ne 0 ]; then
echo "Failed to build or execute the end-to-end test"
exit 1
Expand All @@ -45,23 +48,23 @@ if [ $RunTestNative = "true" ]; then

inc="-I build/native/include"

if [ $IsMacOS = "true" ]; then
if [[ $IsMacOS == "True" || $IsMacOS == "true" ]]; then
Copy link
Contributor

@edgchen1 edgchen1 Feb 12, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks like there is a way to convert to lower case: https://www.gnu.org/software/bash/manual/html_node/Shell-Parameter-Expansion.html#Shell-Parameter-Expansion (search for ${parameter,,pattern})
how about ${IsMacOs,,} == "true"?

export DYLD_FALLBACK_LIBRARY_PATH=$LocalNuGetRepo/_tmp:${DYLD_FALLBACK_LIBRARY_PATH}
libs="-L runtimes/osx-x64/native -l onnxruntime"
g++ -std=c++11 $SourceRoot/csharp/test/Microsoft.ML.OnnxRuntime.EndToEndTests.Capi/C_Api_Sample.cpp $inc $libs -Wunused-result -Wformat=0 -o sampletest
g++ -std=c++11 $BUILD_SOURCESDIRECTORY/csharp/test/Microsoft.ML.OnnxRuntime.EndToEndTests.Capi/C_Api_Sample.cpp $inc $libs -Wunused-result -Wformat=0 -o sampletest
libName=$(otool -L ./sampletest | grep onnxruntime | xargs | cut -d' ' -f1 | cut -d'/' -f2)
ln -sf runtimes/osx-x64/native/libonnxruntime.dylib $libName
else
export LD_LIBRARY_PATH=$LocalNuGetRepo/_tmp:${LD_LIBRARY_PATH}
libs="-L runtimes/linux-x86/native -L runtimes/linux-x64/native -l onnxruntime"
g++ -std=c++11 $SourceRoot/csharp/test/Microsoft.ML.OnnxRuntime.EndToEndTests.Capi/C_Api_Sample.cpp $inc $libs -Wunused-result -o sampletest
g++ -std=c++11 $BUILD_SOURCESDIRECTORY/csharp/test/Microsoft.ML.OnnxRuntime.EndToEndTests.Capi/C_Api_Sample.cpp $inc $libs -Wunused-result -o sampletest
# Create link to versioned shared object required at runtime
libname=`ldd sampletest | grep onnxruntime | xargs | cut -d" " -f1`
ln -sf runtimes/linux-x64/native/libonnxruntime.so $libname
fi

# Copy Sample Model
cp $SourceRoot/csharp/testdata/squeezenet.onnx .
cp $BUILD_SOURCESDIRECTORY/csharp/testdata/squeezenet.onnx .

# Run the sample model
./sampletest
Expand Down
64 changes: 64 additions & 0 deletions csharp/test/Microsoft.ML.OnnxRuntime.Tests/InferenceTest.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -719,6 +719,70 @@ private static Dictionary<string, string> GetSkippedModels(DirectoryInfo modelsD
{ "tf_resnet_v1_152", "result mismatch when Conv BN Fusion is applied" },
{ "coreml_Imputer-LogisticRegression_sklearn_load_breast_cancer", "Can't determine model file name" },
{ "mask_rcnn_keras", "Model should be edited to remove the extra outputs" },
{ "test_strnormalizer_export_monday_casesensintive_lower", "ElementType not currently supported"},
{ "test_max_float64", "node test error"},
{ "test_min_uint8", "node test error"},
{ "test_mod_mixed_sign_float64", "node test error"},
{ "test_einsum_transpose", "node test error"},
{ "test_momentum", "node test error"},
{ "test_max_uint16", "node test error"},
{ "test_resize_downsample_scales_linear_align_corners", "node test error"},
{ "test_strnormalizer_nostopwords_nochangecase", "node test error"},
{ "test_cast_STRING_to_FLOAT", "node test error"},
{ "test_cumsum_2d_negative_axis", "node test error"},
{ "test_cast_FLOAT16_to_DOUBLE", "node test error"},
{ "test_adagrad_multiple", "node test error"},
{ "test_einsum_inner_prod", "node test error"},
{ "test_clip_default_int8_min", "node test error"},
{ "test_max_int8", "node test error"},
{ "test_sequence_insert_at_back", "node test error"},
{ "test_mod_mixed_sign_int8", "node test error"},
{ "test_maxunpool_export_with_output_shape", "node test error"},
{ "test_strnormalizer_export_monday_empty_output", "node test error"},
{ "test_strnormalizer_export_monday_insensintive_upper_twodim", "ElementType not currently supported"},
{ "test_clip_default_int8_max", "node test error"},
{ "test_einsum_sum", "node test error"},
{ "test_min_int16", "node test error"},
{ "test_cast_FLOAT_to_DOUBLE", "node test error"},
{ "test_adagrad", "node test error"},
{ "test_min_float64", "node test error"},
{ "test_max_int16", "node test error"},
{ "test_einsum_batch_diagonal", "node test error"},
{ "test_sequence_insert_at_front", "node test error"},
{ "test_cumsum_1d_exclusive", "node test error"},
{ "test_training_dropout_default", "node test error"},
{ "test_cast_BFLOAT16_to_FLOAT", "node test error"},
{ "test_training_dropout", "node test error"},
{ "test_adam", "node test error"},
{ "test_training_dropout_mask", "node test error"},
{ "test_clip_default_int8_inbounds", "node test error"},
{ "test_eyelike_with_dtype", "node test error"},
{ "test_cumsum_1d", "node test error"},
{ "test_conv_with_autopad_same", "node test error"},
{ "test_cumsum_1d_reverse_exclusive", "node test error"},
{ "test_cast_FLOAT_to_BFLOAT16", "node test error"},
{ "test_bitshift_right_uint16", "node test error"},
{ "test_bitshift_left_uint16", "node test error"},
{ "test_pow_types_float32_uint64", "node test error"},
{ "test_cumsum_2d_axis_0", "node test error"},
{ "test_max_uint8", "node test error"},
{ "test_strnormalizer_export_monday_casesensintive_nochangecase", "ElementType not currently supported"},
{ "test_momentum_multiple", "node test error"},
{ "test_cumsum_1d_reverse", "node test error"},
{ "test_pow_types_float32_uint32", "node test error"},
{ "test_if_seq", "node test error"},
{ "test_resize_downsample_scales_cubic_align_corners", "node test error"},
{ "test_einsum_batch_matmul", "node test error"},
{ "test_nesterov_momentum", "node test error"},
{ "test_cumsum_2d_axis_1", "node test error"},
{ "test_strnormalizer_export_monday_casesensintive_upper", "node test error"},
{ "test_min_uint16", "node test error"},
{ "test_adam_multiple", "node test error"},
{ "test_loop13_seq", "node test error"},
{ "test_convtranspose_autopad_same", "node test error"},
{ "test_training_dropout_default_mask", "node test error"},
{ "test_min_int8", "node test error"},
{ "test_cast_FLOAT_to_STRING", "node test error"},
};

// The following models fails on nocontribops win CI
Expand Down
4 changes: 2 additions & 2 deletions onnxruntime/core/codegen/passes/op_ir_creator/math/clip.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,10 +31,10 @@ Status GENERIC_OP_IR_CREATOR_CLASS(Clip)::Evaluate(
min_value = tvm::make_const(tvm::Float(32), std::numeric_limits<float>::lowest());
max_value = tvm::make_const(tvm::Float(32), std::numeric_limits<float>::max());
auto num_inputs = inputs.size();
if (num_inputs >= 2) {
if (num_inputs >= 2 && inputs[1].defined()) {
min_value = inputs[1]();
}
if (num_inputs == 3) {
if (num_inputs == 3 && inputs[2].defined()) {
max_value = inputs[2]();
}
}
Expand Down
9 changes: 7 additions & 2 deletions onnxruntime/test/providers/cpu/model_tests.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,11 @@ TEST_P(ModelTest, Run) {
#endif
// TODO: filter model based on opset
std::set<BrokenTest> broken_tests = {
{"slice_neg_steps", "Type parameter (Tind) bound to different types (tensor(int64) and tensor(int32) in node ()."},
{"cast_BFLOAT16_to_FLOAT", "Unexpected input data type"},
{"loop13_seq", "Creation of empty sequences is currently not supported in the test runner"},
{"sequence_insert_at_front", "shape mismatch, expect {4} got {3}"},
{"cast_FLOAT_to_BFLOAT16", "expect uint16 got bfloat16"},
{"mnist", "Input data isn't in valid range"},
{"BERT_Squad", "test data bug"},
{"constantofshape_float_ones", "test data bug", {"onnx141", "onnx150"}},
Expand Down Expand Up @@ -802,8 +807,8 @@ ::std::vector<::std::basic_string<ORTCHAR_T>> GetParameterStrings() {
#endif
#endif

// TENSORRT has too many test failures in the single node tests
#if !defined(_WIN32) && !defined(USE_TENSORRT)
// TENSORRT/OpenVino has too many test failures in the single node tests
#if !defined(_WIN32) && !defined(USE_TENSORRT) && !defined(USE_OPENVINO)
paths.push_back("/data/onnx");
#endif
while (!paths.empty()) {
Expand Down
2 changes: 1 addition & 1 deletion tools/ci_build/github/android/run_nnapi_code_coverage.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ python3 ${ORT_ROOT}/tools/ci_build/build.py \
--android \
--build_dir build_nnapi \
--android_sdk_path $ANDROID_HOME \
--android_ndk_path $ANDROID_HOME/ndk-bundle \
--android_ndk_path $ANDROID_NDK_HOME \
--android_abi=x86_64 \
--android_api=29 \
--skip_submodule_sync \
Expand Down
2 changes: 1 addition & 1 deletion tools/ci_build/github/azure-pipelines/android-x86_64-crosscompile-ci-pipeline.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ jobs:
--android \
--build_dir build \
--android_sdk_path $ANDROID_HOME \
--android_ndk_path $ANDROID_HOME/ndk-bundle \
--android_ndk_path $ANDROID_NDK_HOME \
--android_abi=x86_64 \
--android_api=29 \
--skip_submodule_sync \
Expand Down
Loading