Skip to content
This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

Move imagenet inference to nightly #16577

Merged
merged 12 commits into from
Oct 22, 2019
Merged

Move imagenet inference to nightly #16577

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
b02efaa
move to nightly
ChaiBapchya Oct 18, 2019
0495d0d
bracket fix
ChaiBapchya Oct 18, 2019
217aef1
test imgnet comment rest
ChaiBapchya Oct 18, 2019
231810b
br
ChaiBapchya Oct 18, 2019
6648a6b
change dir
ChaiBapchya Oct 18, 2019
f6c2e29
change lib
ChaiBapchya Oct 18, 2019
9b6e0a2
uncomment other tests
ChaiBapchya Oct 21, 2019
f46debe
change lib for build
ChaiBapchya Oct 21, 2019
4f1628b
remove extraneous libs
ChaiBapchya Oct 21, 2019
07ed0ea
add cd to access shell script
ChaiBapchya Oct 21, 2019
a9dc617
ready to merge
ChaiBapchya Oct 21, 2019
daa88e6
fix dir structure
ChaiBapchya Oct 22, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions ci/docker/runtime_functions.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1391,6 +1391,14 @@ nightly_test_installation() {
source ./tests/jenkins/run_test_installation_docs.sh docs/install/index.md 1 1686; ${1}
}

# Runs Imagenet inference
nightly_test_imagenet_inference() {
set -ex
echo $PWD
cp /work/mxnet/build/cpp-package/example/imagenet_inference .
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this tested?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes. on ci dev. it exists.

Copy link
Contributor Author

@ChaiBapchya ChaiBapchya Oct 22, 2019 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Spoke prematurely.
http://jenkins.mxnet-ci-dev.amazon-ml.com/blue/organizations/jenkins/NightlyTestsForBinaries/detail/move_imagenet_inference_nightly/20/pipeline/72

While it passed the cp ... line,

Failed at this line in
./imagenet_inference --symbol_file ./model/Inception-BN-symbol.json --params_file ./model/Inception-BN-0126.params --dataset ./data/val_256_q90.rec --rgb_mean '123.68 116.779 103.939' --batch_size 1 --num_skipped_batches 50 --num_inference_batches 500

Error being

./imagenet_inference: error while loading shared libraries: libmxnet.so: cannot open shared object file: No such file or directory

Unable to understand despite me adding libmxnet.so here

https://github.com/ChaiBapchya/incubator-mxnet/blob/daa88e6ff8f4e98742ea46bbe77085147bf2aeca/tests/nightly/JenkinsfileForBinaries#L23

/work/mxnet/cpp-package/example/inference/unit_test_imagenet_inference.sh
}

#Runs a simple MNIST training example
nightly_test_image_classification() {
set -ex
Expand Down
2 changes: 0 additions & 2 deletions cpp-package/tests/ci_test.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,8 +66,6 @@ cp ../../build/cpp-package/example/test_regress_label .
sh unittests/unit_test_mlp_csv.sh

cd inference
cp ../../../build/cpp-package/example/imagenet_inference .
./unit_test_imagenet_inference.sh

cp ../../../build/cpp-package/example/sentiment_analysis_rnn .
./unit_test_sentiment_analysis_rnn.sh
Expand Down
11 changes: 10 additions & 1 deletion tests/nightly/JenkinsfileForBinaries
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@

mx_lib = 'lib/libmxnet.so, lib/libmxnet.a, lib/libtvm_runtime.so, lib/libtvmop.so, 3rdparty/dmlc-core/libdmlc.a, 3rdparty/tvm/nnvm/lib/libnnvm.a'
mx_cmake_lib = 'build/libmxnet.so, build/libmxnet.a, build/3rdparty/tvm/libtvm_runtime.so, build/libtvmop.so, build/3rdparty/dmlc-core/libdmlc.a, build/tests/mxnet_unit_tests, build/3rdparty/openmp/runtime/src/libomp.so'
mx_lib_cpp_example = 'lib/libmxnet.so, lib/libmxnet.a, lib/libtvm_runtime.so, lib/libtvmop.so, 3rdparty/dmlc-core/libdmlc.a, 3rdparty/tvm/nnvm/lib/libnnvm.a, build/cpp-package/example/imagenet_inference'

node('utility') {
// Loading the utilities requires a node context unfortunately
Expand All @@ -36,7 +37,7 @@ core_logic: {
ws('workspace/build-gpu') {
utils.init_git()
utils.docker_run('ubuntu_build_cuda', 'build_ubuntu_gpu_cuda101_cudnn7', false)
utils.pack_lib('gpu', mx_lib)
utils.pack_lib('gpu', mx_lib_cpp_example)
}
}
}/*,
Expand Down Expand Up @@ -69,6 +70,14 @@ core_logic: {
}
}
},
'ImageNet Inference: GPU': {
node(NODE_LINUX_GPU) {
ws('workspace/nt-ImageInferenceTest') {
utils.unpack_and_init('gpu', mx_lib_cpp_example)
utils.docker_run('ubuntu_nightly_gpu', 'nightly_test_imagenet_inference', true)
}
}
},
'KVStore_SingleNode: GPU': {
node('mxnetlinux-gpu-p3-8xlarge') {
ws('workspace/nt-KVStoreTest') {
Expand Down