Python trained model gives different features set in C++

[MyraBaba]

Hi,

We have model trained in python and want to use it from c++ as for getting 512D float features.

c++ gives different output when we try with image-classification-predict.cpp

We just want to get data (as in code) 512d float vector as in python output ?

model:
https://pan.baidu.com/s/1mj6X7MK
or

https://www.dropbox.com/s/ou8v3c307vyzawc/model-r50-arcface-ms1m-refine-v1.zip?dl=0

Isnt there a clear method for C++ feature extraction ?

Best

[lanking520]

@MyraBaba thanks for your question and thanks for trying out the C++ API. Could you please bring me more context on the difference with the result? How would you build your C++ package and your system info?
@leleamol here since he is the developer with C++.

@mxnet-label-bot please add label [C++, Question]

[MyraBaba]

Hi
@leleamol

System is Mac os X and also ubuntu 18
build mxnet with cpp support

We want to use Mxnet in our all production.

We just stuck in first attemp :)

https://github.com/blob/master/deploy/test.py this is the our first test.

We can extract feature in pyhton.

We tried to use predict-cpp api to use model and extract future with same model above first message .

c++ image-classification-predict-cpp we put same model same picture in the above repo Tom-hanks.

it gives different outpu (512D but different float numbers).

Clearly we are missing something because we are new to mxnet .

Basically ; We want to this in c++ as in python and want to get same features:

img = cv2.imread('Tom_Hanks_54745.png')
img = cv2.resize(img , (112,112))

img = model.get_input(img)
f2 = model.get_feature(img)
dist = np.sum(np.square(f1-f2))
sim = np.dot(f1, f2.T)

Best...

[MyraBaba]

@leleamol
@lanking520

any idea ? what could be the basic steps in C++ for :

img = model.get_input(img)
f2 = model.get_feature(img)
dist = np.sum(np.square(f1-f2))
sim = np.dot(f1, f2.T)

We loved the mxnet framework bu fi we cant use it in our C++ environment unfortunately we have to drop mxnet and back to the old system.

for mxnet experts specially on the C , C++ side it should be trivial example which means a lot for the community.

What we can do to make it works ?

[leleamol]

@MyraBaba
The MXNet supports C++ APIs in cpp-package directory. I am working on an example to demonstrate a workflow that will

1. Load the model,
2. Load the parameters, and
3. Run a forward pass.

I will try out your model with that example. Will update this issue with my observations.
Thanks

[MyraBaba]

We almost exhausted :) It certainly should work but we are very new to mxnet framwork. On the other hand training speed and mem usage fantastic.

If we succesfully get same results in c++ as in our python test.py we will change our AI to the mxnet completely .

Thanks for your help and attention.

Best

[leleamol]

With my test program I was able to get the output of float vector of 512. Without using sklearn.preprocessing.normalize on that output (similar to what was done in the test python script), the maximum accuracy was reported to be 2.7216 at index 84. The rest of the entries in 512D vector were 0.03412. ( I have used the same image as that of test.py)

I did not normalize the image before running forward pass.
If this helps, I can share the code of my example.

[MyraBaba]

I will work on your c++ code and try to make progress. I will run under the cpp-package /example/ feature_extract. Right ? I can finalise this week this we will move money and there will be support requested we can discuss separately Best

…

On 22 Oct 2018, at 20:42, Amol Lele ***@***.***> wrote: With my test program I was able to get the output of float vector of 512. Without using sklearn.preprocessing.normalize on that output (similar to what was done in the test python script), the maximum accuracy was reported to be 2.7216 at index 84. The rest of the entries in 512D vector were 0.03412. ( I have used the same image as that of test.py) I did not normalize the image before running forward pass. If this helps, I can share the code of my example. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#12869 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AQscn9zC_lg0ZG6AfMOZUy_oGMhnxeygks5ungN2gaJpZM4XvOz7>.

[leleamol]

@MyraBaba
The cpp-package/example/feature_extract is a good start but it is not very optimized and does not follow the recommended approach to run the inference workflow.
For example, the program is invoking "SimpleBind" operation every time before running the Forward() call. Since, SimpleBind() is an expensive operation, it should be invoked only once.

Please let us know if you need help.

[MyraBaba]

Can you share your code that you mentioned above please , I will start from your code.

[leleamol]

@MyraBaba
I have submitted a PR #12915. It will be under review. I have used the same example to load the model you have mentioned in the issue and run the inference workflow with the image Tom_Hanks_54745.png

[MyraBaba]

@leleamol
Thanks I started to working on it.
inception_inference is asking synset file. All we need 512D feature we will use it for our own purposes. we dont need labels. How we can just produce 512 features ?

I used synset.tx (inception) with below command and get error below.:

How did you run exactly for tom_hanks (112x112) ?

`./inception_inference --symbol /Users/alpullu/Projects/inSAF-face/models/model-r50-am-lfw/-symbol.json --params /Users/alpullu/Projects/inSAF-face/models/model-r50-am-lfw/-0000.params  --input_shape 3 112 112 --image /Users/alpullu/Projects/inSAF-face/deploy/Tom_Hanks_54745.png --synset ./sysnset.txt --mean mean_224.nd 
[13:50:28] inception_inference.cpp:121: Loading the model from /Users/alpullu/Projects/inSAF-face/models/model-r50-am-lfw/-symbol.json

[13:50:28] src/nnvm/legacy_json_util.cc:209: Loading symbol saved by previous version v0.12.1. Attempting to upgrade...
[13:50:28] src/nnvm/legacy_json_util.cc:217: Symbol successfully upgraded!
[13:50:28] inception_inference.cpp:130: Loading the model parameters from /Users/alpullu/Projects/inSAF-face/models/model-r50-am-lfw/-0000.params

[13:50:28] inception_inference.cpp:153: Loading the synset file.
libc++abi.dylib: terminating with uncaught exception of type dmlc::Error: [13:50:28] ../../include/mxnet-cpp/symbol.hpp:211: Check failed: MXSymbolInferShape(GetHandle(), keys.size(), keys.data(), arg_ind_ptr.data(), arg_shape_data.data(), &in_shape_size, &in_shape_ndim, &in_shape_data, &out_shape_size, &out_shape_ndim, &out_shape_data, &aux_shape_size, &aux_shape_ndim, &aux_shape_data, &complete) == 0 (-1 vs. 0) 

Stack trace returned 9 entries:
[bt] (0) 0   inception_inference                 0x0000000107ca386b dmlc::StackTrace() + 299
[bt] (1) 1   inception_inference                 0x0000000107ca35ff dmlc::LogMessageFatal::~LogMessageFatal() + 47
[bt] (2) 2   inception_inference                 0x0000000107ca5ad3 mxnet::cpp::Symbol::InferShape(std::__1::map<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::vector<unsigned int, std::__1::allocator<unsigned int> >, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::allocator<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const, std::__1::vector<unsigned int, std::__1::allocator<unsigned int> > > > > const&, std::__1::vector<std::__1::vector<unsigned int, std::__1::allocator<unsigned int> >, std::__1::allocator<std::__1::vector<unsigned int, std::__1::allocator<unsigned int> > > >*, std::__1::vector<std::__1::vector<unsigned int, std::__1::allocator<unsigned int> >, std::__1::allocator<std::__1::vector<unsigned int, std::__1::allocator<unsigned int> > > >*, std::__1::vector<std::__1::vector<unsigned int, std::__1::allocator<unsigned int> >, std::__1::allocator<std::__1::vector<unsigned int, std::__1::allocator<unsigned int> > > >*) const + 947
[bt] (3) 3   inception_inference                 0x0000000107ca4a64 mxnet::cpp::Symbol::InferExecutorArrays(mxnet::cpp::Context const&, std::__1::vector<mxnet::cpp::NDArray, std::__1::allocator<mxnet::cpp::NDArray> >*, std::__1::vector<mxnet::cpp::NDArray, std::__1::allocator<mxnet::cpp::NDArray> >*, std::__1::vector<mxnet::cpp::OpReqType, std::__1::allocator<mxnet::cpp::OpReqType> >*, std::__1::vector<mxnet::cpp::NDArray, std::__1::allocator<mxnet::cpp::NDArray> >*, std::__1::map<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, mxnet::cpp::NDArray, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::allocator<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const, mxnet::cpp::NDArray> > > const&, std::__1::map<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, mxnet::cpp::NDArray, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::allocator<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const, mxnet::cpp::NDArray> > > const&, std::__1::map<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, mxnet::cpp::OpReqType, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::allocator<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const, mxnet::cpp::OpReqType> > > const&, std::__1::map<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, mxnet::cpp::NDArray, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::allocator<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const, mxnet::cpp::NDArray> > > const&) const + 548
[bt] (4) 4   inception_inference                 0x0000000107c9e6cf mxnet::cpp::Symbol::SimpleBind(mxnet::cpp::Context const&, std::__1::map<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, mxnet::cpp::NDArray, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::allocator<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const, mxnet::cpp::NDArray> > > const&, std::__1::map<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, mxnet::cpp::NDArray, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::allocator<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const, mxnet::cpp::NDArray> > > const&, std::__1::map<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, mxnet::cpp::OpReqType, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::allocator<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const, mxnet::cpp::OpReqType> > > const&, std::__1::map<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, mxnet::cpp::NDArray, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::allocator<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const, mxnet::cpp::NDArray> > > const&) + 127
[bt] (5) 5   inception_inference                 0x0000000107c9d981 Predictor::Predictor(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, mxnet::cpp::Shape const&) + 641
[bt] (6) 6   inception_inference                 0x0000000107ca1d49 main + 1849
[bt] (7) 7   libdyld.dylib                       0x00007fffb9881235 start + 1
[bt] (8) 8   ???                                 0x000000000000000f 0x0 + 15

`

[leleamol]

@MyraBaba The synset file and mean image file are not mandatory. But there is a bug in code that mandates synset file to be specified. I am sending out the fix.
The crash you are seeing is due to incorrect input image shape.
The program assumes the input image shape to be 3 * 224 * 224, if not specified on the command line.
For your model your command line would something like this:

inception_inference --symbol <model_json_file> --params --image --synset <./synset file> --input_shape "3 112 112"

[leleamol]

Here is the PR #12930 after fixing the example.

[MyraBaba]

@leleamol How I can contact you ? have you email ? I could be get paid support from you / also a contribution for your effort .

Best

[lanking520]

@MyraBaba you can join our Slack channel where everybody can ask questions and it's for free!
To join it: http://mxnet.incubator.apache.org/community/contribute.html#slack

[leleamol]

@MyraBaba as @lanking520 suggested you can join the slack channel and can ask questions there.
I was wondering whether the example in PR helpful in solving your issue?

[MyraBaba]

@leleamol Thanks for your PR it is very good start for me.

But time is limited for me and I need the produce same results for same picture as in python version which is requires more deep knowledge mxnet & align and normalisation I think.

[MyraBaba]

I send mail to slack channel and it got bact to me error. FYI

[nswamy]

@MyraBaba you can send a email to [email protected] requesting an invite, we'll invite your email address.

[MyraBaba]

Hi @leleamol

I am trying to integrate and testing your new inference.

do you know the equilavent of the beloq python code in mxnet or Ndarray ? :

distance = numpy.sum(numpy.square(descA512 - descB512))

I will test with my own preprocessed faces first to see difference..

[leleamol]

@MyraBaba
According to the ndarray.h, the NDArray does support "-" and "*" operators with which you can implement "numpy.square(descA512- descB512)" operations. However, numpy.sum equivalent operation is not yet supported for NDArray.

[MyraBaba]

than I can use ** instead of numpy.square right ?

PS: I have trouble to linking #12930 in cmake . what could be the proper way to do in cmake ?

Best

[leleamol]

You should be able to use "*" operator in NDArray to get the square.

Regarding #12930 the example contains Makefile. What is the error that you are getting?

[leleamol]

@mxnet-label-bot add [Pending Requester Info]

[MyraBaba]

@leleamol Hi,

I am back to project and able to compile with Cmake ypur inception_inference.cpp with above mentioned models.

Can you direct me to where is the output of float vector of 512 variable ? can I get this 512 float vector features with this .cpp ? If so I couldnt find it with debugging. Could you possibly let me
know?

Best

With my test program I was able to get the output of float vector of 512. Without using sklearn.preprocessing.normalize on that output (similar to what was done in the test python script), the maximum accuracy was reported to be 2.7216 at index 84. The rest of the entries in 512D vector were 0.03412. ( I have used the same image as that of test.py)

I did not normalize the image before running forward pass.
If this helps, I can share the code of my example.

[leleamol]

Hi @MyraBaba

In the [inception_inference.cpp] https://github.com/apache/incubator-mxnet/blob/master/cpp-package/example/inference/inception_inference.cpp example, you can add following line

LG << array;

after the line [309]https://github.com/apache/incubator-mxnet/blob/master/cpp-package/example/inference/inception_inference.cpp#L309.
It will print out all the values of the output vector.

In the example, we use ArgmaxChannel on this output to get the index with highest probability.
I hope this helps.

[vdantu]

@MyraBaba : Were you able to output the float vector? Just checking in to see how you were doing with using the CPP example for your use-case.

If you were able to successfully use the CPP example, could we close this issue?

[MyraBaba]

when I add the LG << array saw the output.

now I need to test both python and the c++ inference outputs is same.

best

[lanking520]

@MyraBaba Close this issue for now, Please feel free to reopen if problem persist.