Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support 1080P and cudnn v5? #237

Closed
zimenglan-sysu-512 opened this issue Jul 1, 2016 · 72 comments
Closed

support 1080P and cudnn v5? #237

zimenglan-sysu-512 opened this issue Jul 1, 2016 · 72 comments

Comments

@zimenglan-sysu-512
Copy link

so far, faster-rcnn surpports 1080P and cudnn v5?

thanks.

@zimenglan-sysu-512
Copy link
Author

@rbgirshick can you do me a favor? thanks.

@EvanWeiner
Copy link

+1 echo Zimenglan.

Hi @rbgirshick ,

Thank you so much for your hard work on py-faster-cnn. I would like to humbly request if you can please modify faster-rcnn to work with cudnn v5. The new GPUs like 1080 can't use anything less than v5. This would greatly be appreciated.

Thank you,
Evan

@manipopopo
Copy link

manipopopo commented Jul 5, 2016

Maybe you can try to merge caffe master branch into caffe-fast-rcnn.

cd caffe-fast-rcnn  
git remote add caffe https://github.com/BVLC/caffe.git  
git fetch caffe  
git merge -X theirs caffe/master  

Remove self_.attr("phase") = static_cast<int>(this->phase_); from include/caffe/layers/python_layer.hpp after merging.


py-R-FCN use Microsoft-version Caffe which is compatible with cudnn 5.

@zimenglan-sysu-512
Copy link
Author

zimenglan-sysu-512 commented Jul 6, 2016

@manipopopo thanks. it's a real help, it works! now i can use 1080P and v5 of cudnn.
@EvanWeiner you can try. self_.attr("phase") = static_cast<int>(this->phase_) from include/caffe/layers/python_layer.hpp is the key.

@GBJim
Copy link

GBJim commented Jul 13, 2016

Hi @manipopopo
I followed your instruction, but encountered an error when I run /tools/demo.py
The error message is as follwoing:

WARNING: Logging before InitGoogleLogging() is written to STDERR
W0713 03:15:46.031435 22906 _caffe.cpp:122] DEPRECATION WARNING - deprecated use of Python interface
W0713 03:15:46.031481 22906 _caffe.cpp:123] Use this instead (with the named "weights" parameter):
W0713 03:15:46.031488 22906 _caffe.cpp:125] Net('/root/py-faster-rcnn/models/pascal_voc/VGG16/faster_rcnn_alt_opt/faster_rcnn_test.pt', 1, weights='/root/py-faster-rcnn/data/faster_rcnn_models/VGG16_faster_rcnn_final.caffemodel')
[libprotobuf ERROR google/protobuf/text_format.cc:245] Error parsing text-format caffe.NetParameter: 350:21: Message type "caffe.LayerParameter" has no field named "roi_pooling_param".
F0713 03:15:46.033658 22906 upgrade_proto.cpp:79] Check failed: ReadProtoFromTextFile(param_file, param) Failed to parse NetParameter file: /root/py-faster-rcnn/models/pascal_voc/VGG16/faster_rcnn_alt_opt/faster_rcnn_test.pt
*** Check failure stack trace: ***
Aborted (core dumped)

How do I solve this problem?

@manipopopo
Copy link

Make sure that optional ROIPoolingParameter roi_pooling_param = 8266711; is in caffe-fast-rcnn/src/caffe/proto/caffe.proto. Remember to rebuild caffe-fast-rcnn after merging.

@GBJim
Copy link

GBJim commented Jul 13, 2016

Hi @manipopopo
I have successfully compiled the merged Caffe.
But when I test the training speed om VGG16, I found the training speed is around 1.214s / iteration, which is not faster comparing cuDNN v3 and no-cuDNN implementation.
Is it normal?

@manipopopo
Copy link

I'm sorry I can't share you with the precise time one iteration takes. I don't have 1080 at hand. But I vaguely remembered it took about 13 seconds for 20 iterations when I trained ResNet-101 (GTX 1080, cuda 8, cudnn 5) using end-to-end strategy a month ago.

@zimenglan-sysu-512
Copy link
Author

zimenglan-sysu-512 commented Jul 14, 2016

@manipopopo i use 1080P to test the demo images, the prcise time as below:

Demo for data/demo/000456.jpg
Detection took 0.107s for 300 object proposals

Demo for data/demo/000542.jpg
Detection took 0.077s for 161 object proposals

Demo for data/demo/001150.jpg
Detection took 0.089s for 194 object proposals

Demo for data/demo/001763.jpg
Detection took 0.089s for 196 object proposals

Demo for data/demo/004545.jpg
Detection took 0.104s for 300 object proposals

@GBJim
Copy link

GBJim commented Jul 14, 2016

@zimenglan-sysu-512 That's really fast!
My detection speed is 0.260s in K80 machine (VGG16)

@manipopopo
Copy link

@zimenglan-sysu-512
I vaguely remembered it took about 190 ms for 300 proposals per image for ResNet-101 (GTX 1080, cuda 8, cudnn 5) during the test phase.

@hengck23
Copy link

for those who are using windows caffe, here is a fork that support cudnnv5
https://github.com/TadaoYamaoka/caffe/tree/windows

For more information, refer to:
http://tadaoyamaoka.hatenablog.com/entry/2016/06/19/225038

@duanLH
Copy link

duanLH commented Aug 14, 2016

@manipopopo when I run git merge caffe/master It show that:

root@duan:/home/duan/py-faster-rcnn/caffe-fast-rcnn# git merge caffe/mastererror: Your local changes to the following files would be overwritten by merge:
data/cifar10/get_cifar10.sh
data/ilsvrc12/get_ilsvrc_aux.sh
data/mnist/get_mnist.sh
examples/cifar10/create_cifar10.sh
examples/cifar10/train_full.sh
examples/cifar10/train_full_sigmoid.sh
examples/cifar10/train_full_sigmoid_bn.sh
examples/cifar10/train_quick.sh
examples/imagenet/create_imagenet.sh
examples/imagenet/resume_training.sh
examples/imagenet/train_caffenet.sh
examples/mnist/create_mnist.sh
examples/mnist/train_lenet.sh
examples/mnist/train_lenet_adam.sh
examples/mnist/train_lenet_consolidated.sh
examples/mnist/train_lenet_rmsprop.sh
examples/mnist/train_mnist_autoencoder.sh
examples/mnist/train_mnist_autoencoder_adadelta.sh
examples/mnist/train_mnist_autoencoder_adagrad.sh
examples/mnist/train_mnist_autoencoder_nesterov.sh
examples/siamese/create_mnist_siamese.sh
examples/siamese/train_mnist_siamese.sh
python/draw_net.py
scripts/cpp_lint.py
scripts/download_model_binary.py
scripts/travis/travis_build_and_test.sh
scripts/travis/travis_install.sh
scripts/travis/travis_setup_makefile_config.sh
tools/extra/parse_log.py
tools/extra/plot_training_log.py.example
Please, commit your changes or stash them before you can merge.
Aborting

what should i do?

@manipopopo
Copy link

As it said, you should commit your changes or stash them before you can merge. You can try

  1. Use git stash to save your local changes before merging. Afterwards, you can use git stash pop to apply the saved changes on the working directory.
  2. Use git commit -a to commit all the changes before merging.

@longwoo
Copy link

longwoo commented Aug 29, 2016

@manipopopo I am using ubuntu16.04+GTX1080+cuda8.0RC+cudnn v5 and tried your method. After make -j8, it gives me

PROTOC src/caffe/proto/caffe.proto
CXX src/caffe/common.cpp
CXX src/caffe/blob.cpp
CXX src/caffe/layer_factory.cpp
......
CXX src/caffe/syncedmem.cpp
CXX src/caffe/internal_thread.cpp
CXX src/caffe/net.cpp
NVCC src/caffe/solvers/adadelta_solver.cu
NVCC src/caffe/solvers/adam_solver.cu
NVCC src/caffe/solvers/rmsprop_solver.cu
......
NVCC src/caffe/layers/prelu_layer.cu
NVCC src/caffe/layers/threshold_layer.cu
NVCC src/caffe/layers/base_data_layer.cu
CXX/LD -o .build_release/tools/net_speed_benchmark.bin
CXX/LD -o .build_release/tools/test_net.bin
CXX/LD -o .build_release/tools/convert_imageset.bin
CXX/LD -o .build_release/tools/extract_features.bin
CXX/LD -o .build_release/tools/device_query.bin
CXX/LD -o .build_release/tools/train_net.bin
CXX/LD -o .build_release/tools/compute_image_mean.bin
CXX/LD -o .build_release/tools/caffe.bin
CXX/LD -o .build_release/tools/upgrade_solver_proto_text.bin
CXX/LD -o .build_release/tools/upgrade_net_proto_text.bin
CXX/LD -o .build_release/tools/upgrade_net_proto_binary.bin
CXX/LD -o .build_release/tools/finetune_net.bin
.build_release/lib/libcaffe.so: undefined reference to `cv::imencode(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, cv::_InputArray const&, std::vector<unsigned char, std::allocator<unsigned char> >&, std::vector<int, std::allocator<int> > const&)'
.build_release/lib/libcaffe.so: undefined reference to `cv::imread(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int)'
collect2: error: ld returned 1 exit status
.build_release/lib/libcaffe.so: undefined reference to Makefile:619: recipe for target '.build_release/tools/caffe.bin' failed
make: *** [.build_release/tools/caffe.bin] Error 1
make: *** Waiting for unfinished jobs....
`cv::imencode(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, cv::_InputArray const&, std::vector<unsigned char, std::allocator<unsigned char> >&, std::vector<int, std::allocator<int> > const&)'
.build_release/lib/libcaffe.so: undefined reference to `cv::imread(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int)'
collect2: error: ld returned 1 exit status
Makefile:619: recipe for target '.build_release/tools/extract_features.bin' failed
make: *** [.build_release/tools/extract_features.bin] Error 1
.build_release/lib/libcaffe.so: undefined reference to `cv::imencode(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, cv::_InputArray const&, std::vector<unsigned char, std::allocator<unsigned char> >&, std::vector<int, std::allocator<int> > const&)'
.build_release/lib/libcaffe.so: undefined reference to `cv::imread(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int)'
collect2: error: ld returned 1 exit status
Makefile:619: recipe for target '.build_release/tools/convert_imageset.bin' failed
make: *** [.build_release/tools/convert_imageset.bin] Error 1
.build_release/lib/libcaffe.so: undefined reference to `cv::imencode(std:.:__cxx11::basic_string<char, std::char_traits<char>, std::build_release/lib/libcaffe.so: undefined referenceallocator <char> > toconst `cv::imencode(std::__cxx11::basic_string<char, std&::char_traits<char>,, std::allocator<char> >  const&, cv::_InputArray constcv::_InputArray const&, std::vector<unsigned char, std::allocator&<unsigned char> >, std::vector<unsigned char, std::allocator&<unsigned char> ,> std::vector<int, std::allocator<int> &> const&)'
.build_release/lib/,libcaffe.so : undefined reference stdto `cv::imread(std:::__cxx11::basic_string<:char, std::vectorchar_traits<int, std::allocator<int> > const&)'
.build_release/lib/libcaffe.so: undefined reference to `cv::imread(std::__cxx11::basic_string<char>, <std::charallocator, std::char_traits<char>, std::allocator<char> > const&, int)'
<char> > const&, int)'
collect2: error: ld returned 1 exit status
collect2: error: ld returned 1 exit status
Makefile:619: recipe for target '.build_release/tools/compute_image_mean.bin' failed
make: *** [.build_release/tools/compute_image_mean.bin] Error 1
Makefile:619: recipe for target '.build_release/tools/upgrade_solver_proto_text.bin' failed
make: *** [.build_release/tools/upgrade_solver_proto_text.bin] Error 1
.build_release/lib/libcaffe.so: undefined reference to `cv::imencode(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, cv::_InputArray const&, std::vector<unsigned char, std::allocator<unsigned char> >&, std::vector<int, std::allocator<int> > const&)'
.build_release/lib/libcaffe.so: undefined reference to `cv::imread(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int)'
collect2: error: ld returned 1 exit status
Makefile:619: recipe for target '.build_release/tools/upgrade_net_proto_text.bin' failed
make: *** [.build_release/tools/upgrade_net_proto_text.bin] Error 1
.build_release/lib/libcaffe.so: undefined reference to `cv::imencode(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, cv::_InputArray const&, std::vector<unsigned char, std::allocator<unsigned char> >&, std::vector<int, std::allocator<int> > const&)'
.build_release/lib/libcaffe.so: undefined reference to `cv::imread(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int)'
collect2: error: ld returned 1 exit status
Makefile:619: recipe for target '.build_release/tools/upgrade_net_proto_binary.bin' failed
make: *** [.build_release/tools/upgrade_net_proto_binary.bin] Error 1

@manipopopo
Copy link

manipopopo commented Aug 29, 2016

@longwoo
Have you uncommented `OPENCV_VERSION := 3ˋ from Makefile.config?

@longwoo
Copy link

longwoo commented Aug 29, 2016

@manipopopo Thanks. So we have to use opencv3 in this case? I am trying to install opencv3 and rebuild again. By the way, here is my Makefile.config.

USE_CUDNN := 1
CUDA_DIR := /usr/local/cuda
CUDA_ARCH := -gencode arch=compute_20,code=sm_20 \
            -gencode arch=compute_20,code=sm_21 \
            -gencode arch=compute_30,code=sm_30 \
            -gencode arch=compute_35,code=sm_35 \
            -gencode arch=compute_50,code=sm_50 \
            -gencode arch=compute_50,code=compute_50

BLAS := atlas

ANACONDA_HOME := $(HOME)/anaconda2
PYTHON_INCLUDE := $(ANACONDA_HOME)/include \
             $(ANACONDA_HOME)/include/python2.7 \
             $(ANACONDA_HOME)/lib/python2.7/site-packages/numpy/core/include \

PYTHON_LIB := $(ANACONDA_HOME)/lib

WITH_PYTHON_LAYER := 1

INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include
LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib

BUILD_DIR := build
DISTRIBUTE_DIR := distribute

TEST_GPUID := 0

Q ?= @

@manipopopo
Copy link

manipopopo commented Aug 29, 2016

I'm not sure but opencv 2 should be OK.

I didn't know what version your opencv is, so I guessed that you're using 3. The default Makefile is for opencv 2. If you want to build caffe with OpenCV 3, you'll need to uncomment ˋOPENCV_VERSION := 3ˋ. Otherwise you'll see the error messages you post.

@austingg
Copy link

@longwoo by the way, your CUDA_Arch should be set to compute_61 for gtx 1080 card

@longwoo
Copy link

longwoo commented Aug 29, 2016

@manipopopo Yes,opencv 2 is OK .Thanks.
@austingg compute_61 solves the problem. Thank you.

USE_CUDNN := 1

CUDA_DIR := /usr/local/cuda
CUDA_ARCH := -gencode arch=compute_20,code=sm_20 \
    -gencode arch=compute_20,code=sm_21 \
    -gencode arch=compute_30,code=sm_30 \
    -gencode arch=compute_35,code=sm_35 \
    -gencode arch=compute_50,code=sm_50 \
    -gencode arch=compute_61,code=sm_61 \
    -gencode arch=compute_61,code=compute_61

BLAS := atlas

PYTHON_INCLUDE := /usr/include/python2.7 \
        /usr/lib/python2.7/dist-packages/numpy/core/include
PYTHON_LIB := /usr/lib
WITH_PYTHON_LAYER := 1

INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include /usr/include/hdf5/serial/
LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib /usr/lib/x86_64-linux-gnu/hdf5/serial/

BUILD_DIR := build
DISTRIBUTE_DIR := distribute

TEST_GPUID := 0

Q ?= @

@austingg
Copy link

austingg commented Sep 19, 2016

@longwoo By the way, if you only develop on 1080 gpu, you can comment other compute capacity, which will make a faster compile speed.

@vishaltiwari
Copy link

vishaltiwari commented Mar 21, 2017

Hi,

I am trying to build py-faster-rcnn on ubuntu 16.04, using tesla k20c, cudnn: v5. (cuda 8).

This is what I have done:
In Makefile.config:
uncommented the USE_CUDNN:=1 (as without this I am to build it, but only able to run the ./tools/demo.py --cpu , but the default (--gpu) gives the following error).


I0321 16:44:53.692109 26971 net.cpp:744] Ignoring source layer data
I0321 16:44:53.758460 26971 net.cpp:744] Ignoring source layer drop6
I0321 16:44:53.767998 26971 net.cpp:744] Ignoring source layer drop7
I0321 16:44:53.768015 26971 net.cpp:744] Ignoring source layer fc7_drop7_0_split
I0321 16:44:53.768318 26971 net.cpp:744] Ignoring source layer loss_cls
I0321 16:44:53.768327 26971 net.cpp:744] Ignoring source layer loss_bbox
I0321 16:44:53.769767 26971 net.cpp:744] Ignoring source layer silence_rpn_cls_score
I0321 16:44:53.769780 26971 net.cpp:744] Ignoring source layer silence_rpn_bbox_pred


Loaded network $FRCN_ROOT/data/faster_rcnn_models/VGG16_faster_rcnn_final.caffemodel
F0321 16:44:54.035586 26971 syncedmem.cpp:71] Check failed: error == cudaSuccess (2 vs. 0)  out of memory
*** Check failure stack trace: ***
Aborted (core dumped)

But I need performance as I am working on a video dataset, so I have to build it using "USE_CUDNN:=1"

Also, I have merged caffe-fater-rcnn with caffe/master build and removed the line: self_.attr("phase") = static_cast<int>(this->phase_) as mentioned in the previous posts. And tried to rebuild.

At the very end I get this following error:

CXX/LD -o .build_release/tools/test_net.bin
CXX/LD -o .build_release/tools/upgrade_solver_proto_text.bin
CXX/LD -o .build_release/tools/train_net.bin
CXX/LD -o .build_release/tools/upgrade_net_proto_text.bin
CXX/LD -o .build_release/tools/device_query.bin
CXX/LD -o .build_release/tools/net_speed_benchmark.bin
CXX/LD -o .build_release/tools/finetune_net.bin
CXX/LD -o .build_release/tools/convert_imageset.bin
CXX/LD -o .build_release/tools/compute_image_mean.bin
CXX/LD -o .build_release/tools/extract_features.bin
CXX/LD -o .build_release/tools/upgrade_net_proto_binary.bin
.build_release/lib/libcaffe.so: undefined reference to `cudnnSetActivationDescriptor'
.build_release/lib/libcaffe.so: undefined reference to `cudnnCreateActivationDescriptor'
collect2: error: ld returned 1 exit status
Makefile:625: recipe for target '.build_release/tools/upgrade_net_proto_text.bin' failed
make: *** [.build_release/tools/upgrade_net_proto_text.bin] Error 1
make: *** Waiting for unfinished jobs....
.build_release/lib/libcaffe.so: undefined reference to `cudnnSetActivationDescriptor'
.build_release/lib/libcaffe.so: undefined reference to `cudnnCreateActivationDescriptor'
collect2: error: ld returned 1 exit status
Makefile:625: recipe for target '.build_release/tools/upgrade_solver_proto_text.bin' failed
make: *** [.build_release/tools/upgrade_solver_proto_text.bin] Error 1
.build_release/lib/libcaffe.so: undefined reference to `cudnnSetActivationDescriptor'
.build_release/lib/libcaffe.so: undefined reference to `cudnnCreateActivationDescriptor'
collect2: error: ld returned 1 exit status
Makefile:625: recipe for target '.build_release/tools/extract_features.bin' failed
make: *** [.build_release/tools/extract_features.bin] Error 1
.build_release/lib/libcaffe.so: undefined reference to `cudnnSetActivationDescriptor'
.build_release/lib/libcaffe.so: undefined reference to `cudnnCreateActivationDescriptor'
collect2: error: ld returned 1 exit status
Makefile:625: recipe for target '.build_release/tools/compute_image_mean.bin' failed
make: *** [.build_release/tools/compute_image_mean.bin] Error 1
.build_release/lib/libcaffe.so: undefined reference to `cudnnSetActivationDescriptor'
.build_release/lib/libcaffe.so: undefined reference to `cudnnCreateActivationDescriptor'
collect2: error: ld returned 1 exit status
Makefile:625: recipe for target '.build_release/tools/convert_imageset.bin' failed
make: *** [.build_release/tools/convert_imageset.bin] Error 1
.build_release/lib/libcaffe.so: undefined reference to `cudnnSetActivationDescriptor'
.build_release/lib/libcaffe.so: undefined reference to `cudnnCreateActivationDescriptor'
collect2: error: ld returned 1 exit status
Makefile:625: recipe for target '.build_release/tools/upgrade_net_proto_binary.bin' failed
make: *** [.build_release/tools/upgrade_net_proto_binary.bin] Error 1

I am not sure what library is cudnnSetActivationDescriptor' and cudnnCreateActivationDescriptor' that it's not able to find. I tried to look for this error but didn't find anything useful.

Does anyone know how this can be fixed?

Thanks,

@hana9090
Copy link

hana9090 commented Apr 1, 2017

@manipopopo
PLease after I did your step regarding caffe,
when I ((sudo make -j4 && make pycaffe))
I faced this error:

touch python/caffe/proto/init.py
touch: cannot touch 'python/caffe/proto/init.py': Permission denied
Makefile:647: recipe for target 'python/caffe/proto/init.py' failed
make: *** [python/caffe/proto/init.py] Error 1

@manipopopo
Copy link

@hana9090
I have no idea why it happens. 😿
Could you use

make -j4
make pycaffe

or

sudo make -j4
sudo make pycaffe

instead?

@hana9090
Copy link

hana9090 commented Apr 1, 2017

still have same error :\

@manipopopo
Copy link

@hana9090
I got the same error when I ran

sudo make
make pycaffe

After that I ran sudo make pycaffe, it works without error.

You may want to try something like

sudo make clean
make
make pycaffe

@hana9090
Copy link

hana9090 commented Apr 1, 2017

@manipopopo
I did what you write, but still have the same error of permission denied.
Although I change the permission of /proto/ directory from root to user!

@hana9090
Copy link

hana9090 commented Apr 1, 2017

@manipopopo
Thank you :)
it worked with me if I arrange them as following:

sudo make clean
make pycaffe
make -j8

dacox added a commit to kinsolresearch/caffe that referenced this issue May 10, 2017
mbuckler added a commit to mbuckler/caffe-fast-rcnn-cuDNN-V5 that referenced this issue Jul 29, 2017
@soulslicer
Copy link

Made it work with pascal:

Clone this and compile then

https://github.com/soulslicer/caffe-fast-rcnn

Just change proposal_layer.py

     def setup(self, bottom, top):
         # parse the layer parameter string, which must be valid YAML
-        layer_params = yaml.load(self.param_str_)
+        layer_params = yaml.load(self.param_str)
 
         self._feat_stride = layer_params['feat_stride']
         anchor_scales = layer_params.get('scales', (8, 16, 32))
@@ -61,7 +61,7 @@ class ProposalLayer(caffe.Layer):
         assert bottom[0].data.shape[0] == 1, \
             'Only single item batches are supported'
 
-        cfg_key = str(self.phase) # either 'TRAIN' or 'TEST'
+        cfg_key = 'TEST' if self.phase == 1 else 'TRAIN'

@d-li14
Copy link

d-li14 commented Sep 18, 2017

@manipopopo hi, I followed your instructions but met the error below while make pytest

======================================================================
ERROR: test_backward (test_python_layer_with_param_str.TestLayerWithParam)

Traceback (most recent call last):
File "/home/lid/py-faster-rcnn/caffe-fast-rcnn/python/caffe/test/test_python_layer_with_param_str.py", line 46, in setUp
self.net = caffe.Net(net_file, caffe.TRAIN)
File "/home/lid/py-faster-rcnn/caffe-fast-rcnn/python/caffe/test/test_python_layer_with_param_str.py", line 14, in setup
self.value = float(self.param_str)
AttributeError: 'SimpleParamLayer' object has no attribute 'param_str'

======================================================================
ERROR: test_forward (test_python_layer_with_param_str.TestLayerWithParam)

Traceback (most recent call last):
File "/home/lid/py-faster-rcnn/caffe-fast-rcnn/python/caffe/test/test_python_layer_with_param_str.py", line 46, in setUp
self.net = caffe.Net(net_file, caffe.TRAIN)
File "/home/lid/py-faster-rcnn/caffe-fast-rcnn/python/caffe/test/test_python_layer_with_param_str.py", line 14, in setup
self.value = float(self.param_str)
AttributeError: 'SimpleParamLayer' object has no attribute 'param_str'


Ran 51 tests in 10.025s

FAILED (errors=2)
Makefile:536: recipe for target 'pytest' failed
make: *** [pytest] Error 1

Could you please help me?

@manipopopo
Copy link

manipopopo commented Sep 18, 2017

You can ignore make pytest for the present. It doesn't affect py-faster-rcnn.

@d-li14
Copy link

d-li14 commented Sep 18, 2017

@manipopopo Thanks a lot, I run the demo.py afterwards and it passed the test, maybe make pytest is definately meaningless in some sense, but my motivation is just to find out what happened.

@roxma
Copy link

roxma commented Sep 25, 2017

I think this issue should be kept open.

manipopopo's suggestion works, but still it should be fixed by py-faster-rcnn.

@mammadjv
Copy link

@manipopopo Thanks a lot mate! great !

@twangnh
Copy link

twangnh commented Dec 15, 2017

the method of merge and rebuild caffe works! ./tools/demo.py runs fluently, and the train with ./experiments/scripts/faster_rcnn_end2end.sh runs! but after I changed the train.prototxt, just replace the full connected layers with only one full connected layer and add two conv layer after the pretrained backbone network, and then train again, it prompts Error parsing text-format caffe.NetParameter: 1:3: Message type "caffe.NetParameter" has no field named "me"

@vguptai
Copy link

vguptai commented Feb 23, 2018

For folks who are starting to look into this, give this fork a try. Hopefully, it would help you skip a few steps.

https://github.com/vgupta-ai/py-faster-rcnn

@hana9090
Copy link

@manipopopo
I follow your steps for my setup
UBUNTU 17.10
CUDA 8
CUDNN 5

But When I run the demo I have error
I0410 12:13:10.739405 81877 net.cpp:380] conv1_1 -> conv1_1
F0410 12:13:10.898735 81877 cudnn_conv_layer.cpp:53] Check failed: status == CUDNN_STATUS_SUCCESS (4 vs. 0) CUDNN_STATUS_INTERNAL_ERROR
*** Check failure stack trace: ***
Aborted (core dumped)

@manipopopo
Copy link

@hana9090
I tested it again and got no errors.
Are there any processes using the gpu?
Maybe you could search status == CUDNN_STATUS_SUCCESS (4 vs. 0) CUDNN_STATUS_INTERNAL_ERROR.

@hana9090
Copy link

@manipopopo
I download new faster and build again your steps. I have same error :(
How to search about this error? or this status?
Please help

@hana9090
Copy link

nvidia-smi
Tue Apr 10 16:03:28 2018
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 384.125 Driver Version: 384.125 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 Tesla K80 Off | 00003F47:00:00.0 Off | 0 |
| N/A 42C P0 57W / 149W | 0MiB / 11439MiB | 0% Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
| No running processes found |
+-----------------------------------------------------------------------------+

@manipopopo
Copy link

manipopopo commented Apr 10, 2018

@hana9090
I can't reproduce the problem on my computer. (CUDA 9.1, cuDNN 7)
Maybe you could google status == CUDNN_STATUS_SUCCESS (4 vs. 0) CUDNN_STATUS_INTERNAL_ERROR.
It seems that the error is unrelated to py-faster-rcnn.

@hana9090
Copy link

@manipopopo
Thank you it works :)
I just have to run the demo with sudo

@William8Work
Copy link

William8Work commented May 27, 2018

@manipopopo thanks for the steps. I got my through all the make problems.

However, when I run tools/demo.py then I got this error:

WARNING: Logging before InitGoogleLogging() is written to STDERR
W0526 23:23:06.432446 25676 _caffe.cpp:139] DEPRECATION WARNING - deprecated use of Python interface
W0526 23:23:06.432467 25676 _caffe.cpp:140] Use this instead (with the named "weights" parameter):
W0526 23:23:06.432471 25676 _caffe.cpp:142] Net('/home/wli/codes/py-faster-rcnn/models/pascal_voc/VGG16/faster_rcnn_alt_opt/faster_rcnn_test.pt', 1, weights='/home/wli/codes/py-faster-rcnn/data/faster_rcnn_models/VGG16_faster_rcnn_final.caffemodel')
[libprotobuf ERROR google/protobuf/text_format.cc:274] Error parsing text-format caffe.NetParameter: 350:21: Message type "caffe.LayerParameter" has no field named "roi_pooling_param".
F0526 23:23:06.433696 25676 upgrade_proto.cpp:90] Check failed: ReadProtoFromTextFile(param_file, param) Failed to parse NetParameter file: /home/wli/codes/py-faster-rcnn/models/pascal_voc/VGG16/faster_rcnn_alt_opt/faster_rcnn_test.pt
*** Check failure stack trace: ***
Aborted (core dumped)

Any thoughts on how to fix this? much appreciated!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests