Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

make runtest failed #5077

Closed
kkarnatak opened this issue Dec 8, 2016 · 10 comments
Closed

make runtest failed #5077

kkarnatak opened this issue Dec 8, 2016 · 10 comments

Comments

@kkarnatak
Copy link

I have the following error while running the 'make runtest'. I have tried with cpu flag enabled and disabled both, same error.

----------] 5 tests from ImageDataLayerTest/1, where TypeParam = caffe::CPUDevice<double> [ RUN ] ImageDataLayerTest/1.TestReshape [ OK ] ImageDataLayerTest/1.TestReshape (30 ms) [ RUN ] ImageDataLayerTest/1.TestShuffle [ OK ] ImageDataLayerTest/1.TestShuffle (113 ms) [ RUN ] ImageDataLayerTest/1.TestRead [ OK ] ImageDataLayerTest/1.TestRead (114 ms) [ RUN ] ImageDataLayerTest/1.TestResize *** Aborted at 1481237980 (unix time) try "date -d @1481237980" if you are using GNU date *** PC: @ 0x7ffb7938b0ba (unknown) *** SIGSEGV (@0xfffffffffffffff7) received by PID 5373 (TID 0x7ffb62eaa700) from PID 18446744073709551607; stack trace: *** @ 0x7ffb8254d670 (unknown) @ 0x7ffb7938b0ba (unknown) @ 0x7ffb7938b18b (unknown) @ 0x7ffb7938cce8 (unknown) @ 0x7ffb7938b692 (unknown) @ 0x7ffb79386020 (unknown) @ 0x7ffb79384165 tbb::internal::allocate_root_with_context_proxy::allocate() @ 0x7ffb80db6e22 cv::parallel_for_() @ 0x7ffb8117ab2a (unknown) @ 0x7ffb81177bb7 cv::resize() @ 0x7ffb83061997 caffe::ReadImageToCVMat() @ 0x7ffb82f708bb caffe::ImageDataLayer<>::load_batch() @ 0x7ffb82efc906 caffe::BasePrefetchingDataLayer<>::InternalThreadEntry() @ 0x7ffb82984596 (unknown) @ 0x7ffb8254370a start_thread @ 0x7ffb801b40af clone @ 0x0 (unknown) Segmentation fault (core dumped) src/caffe/test/CMakeFiles/runtest.dir/build.make:57: recipe for target 'src/caffe/test/CMakeFiles/runtest' failed

@cdluminate
Copy link
Contributor

What's your operating system? Ubuntu devel seems to suffer from a similar problem.

https://launchpadlibrarian.net/298212926/buildlog_ubuntu-zesty-amd64.caffe_1.0.0~rc3+20161127-g24d2f67-4_BUILDING.txt.gz

@kkarnatak
Copy link
Author

kkarnatak commented Dec 16, 2016 via email

@stjordanis
Copy link

stjordanis commented Dec 17, 2016

I also have ubuntu 16.10, no cpu.

I have the same issue too, although:

  • Pycaffe runs ok
  • lenet runs ok

After I set DEBUG=1, and I ran:

protoc src/caffe/proto/caffe.proto --cpp_out=.
mkdir include/caffe/proto
mv src/caffe/proto/caffe.pb.h include/caffe/proto

because I had issues with caffe.pb.h,

runtest fails after a while when running:
make runtest

I get (excluding some needless lines):
.build_debug/tools/caffe
.build_debug/test/test_all.testbin 0 --gtest_shuffle --gtest_filter="-GPU"
Note: Google Test filter = -GPU
Note: Randomizing tests' orders with a seed of 77085 .

Many test pass but then:
[----------] 5 tests from ImageDataLayerTest/1, where TypeParam = caffe::CPUDevice
[ RUN ] ImageDataLayerTest/1.TestRead
[ OK ] ImageDataLayerTest/1.TestRead (84 ms)
[ RUN ] ImageDataLayerTest/1.TestResize
*** Aborted at 1481935191 (unix time) try "date -d @1481935191" if you are using GNU date ***
PC: @ 0x7f14a0c870ba (unknown)
*** SIGSEGV (@0xfffffffffffffff7) received by PID 16487 (TID 0x7f14871f2700) from PID 18446744073709551607; stack trace: ***
@ 0x7f14a2152630 (unknown)
@ 0x7f14a0c870ba (unknown)
@ 0x7f14a0c8718b (unknown)
@ 0x7f14a0c88ce8 (unknown)
@ 0x7f14a0c87692 (unknown)
@ 0x7f14a0c82020 (unknown)
@ 0x7f14a0c80165 tbb::internal::allocate_root_with_context_proxy::allocate()
@ 0x7f14a3e50e22 cv::parallel_for_()
@ 0x7f14a3737b2a (unknown)
@ 0x7f14a3734bb7 cv::resize()
@ 0x7f14a2c093dd caffe::ReadImageToCVMat()
@ 0x7f14a2d0694c caffe::ImageDataLayer<>::load_batch()
@ 0x7f14a2c9037c caffe::BasePrefetchingDataLayer<>::InternalThreadEntry()
@ 0x7f14a2c2e19b caffe::InternalThread::entry()
@ 0x7f14a2c307d1 boost::_mfi::mf5<>::operator()()
@ 0x7f14a2c306a9 boost::_bi::list6<>::operator()<>()
@ 0x7f14a2c3059e boost::_bi::bind_t<>::operator()()
@ 0x7f14a2c30550 boost::detail::thread_data<>::run()
@ 0x7f14a33f7596 (unknown)
@ 0x7f14a21486ca start_thread
@ 0x7f14a1e820af clone
@ 0x0 (unknown)
Segmentation fault (core dumped)

My config file:

# USE_CUDNN := 1

# CPU-only switch (uncomment to build without GPU support).
 CPU_ONLY := 1

# uncomment to disable IO dependencies and corresponding data layers
 #USE_OPENCV := 1
 #USE_LEVELDB := 1
 #USE_LMDB := 0




# ALLOW_LMDB_NOLOCK := 1

# Uncomment if you're using OpenCV 3
# OPENCV_VERSION := 3

# To customize your choice of compiler, uncomment and set the following.
# N.B. the default for Linux is g++ and the default for OSX is clang++
# CUSTOM_CXX := g++

# CUDA directory contains bin/ and lib/ directories that we need.
CUDA_DIR := /usr/local/cuda
# On Ubuntu 14.04, if cuda tools are installed via
# "sudo apt-get install nvidia-cuda-toolkit" then use this instead:
# CUDA_DIR := /usr

# CUDA architecture setting: going with all of them.
# For CUDA < 6.0, comment the *_50 lines for compatibility.
CUDA_ARCH := -gencode arch=compute_20,code=sm_20 \
		-gencode arch=compute_20,code=sm_21 \
		-gencode arch=compute_30,code=sm_30 \
		-gencode arch=compute_35,code=sm_35 \
		-gencode arch=compute_50,code=sm_50 \
		-gencode arch=compute_50,code=compute_50

# BLAS choice:
# atlas for ATLAS (default)
# mkl for MKL
# open for OpenBlas
BLAS := atlas
# Custom (MKL/ATLAS/OpenBLAS) include and lib directories.
# Leave commented to accept the defaults for your choice of BLAS
# (which should work)!
# BLAS_INCLUDE := /path/to/your/blas
# BLAS_LIB := /path/to/your/blas

# Homebrew puts openblas in a directory that is not on the standard search path
# BLAS_INCLUDE := $(shell brew --prefix openblas)/include
# BLAS_LIB := $(shell brew --prefix openblas)/lib

# This is required only if you will compile the matlab interface.
# MATLAB directory should contain the mex binary in /bin.
# MATLAB_DIR := /usr/local
# MATLAB_DIR := /Applications/MATLAB_R2012b.app

# NOTE: this is required only if you will compile the python interface.
# We need to be able to find Python.h and numpy/arrayobject.h.
#PYTHON_INCLUDE := /usr/include/python2.7 \
#		/usr/lib/python2.7/dist-packages/numpy/core/include


# Anaconda Python distribution is quite popular. Include path:
# Verify anaconda location, sometimes it's in root.
# ANACONDA_HOME := $(HOME)/anaconda
 PYTHON_INCLUDE :=  /home/steliox/anaconda2/include/python2.7 \
  /home/steliox/anaconda2/include \
		 /home/steliox/anaconda2/lib/python2.7/site-packages/numpy/core/include \



# Uncomment to use Python 3 (default is Python 2)
# PYTHON_LIBRARIES := boost_python3 python3.5m
# PYTHON_INCLUDE := /usr/include/python3.5m \
#                 /usr/lib/python3.5/dist-packages/numpy/core/include

# We need to be able to find libpythonX.X.so or .dylib.
#PYTHON_LIB := /usr/lib
 PYTHON_LIB := /home/steliox/anaconda2/lib


# Homebrew installs numpy in a non standard path (keg only)
# PYTHON_INCLUDE += $(dir $(shell python -c 'import numpy.core; print(numpy.core.__file__)'))/include
# PYTHON_LIB += $(shell brew --prefix numpy)/lib

# Uncomment to support layers written in Python (will link against Python libs)
WITH_PYTHON_LAYER := 1


# Whatever else you find you need goes here.
INCLUDE_DIRS :=  /usr/local/include /usr/include/hdf5/serial/
LIBRARY_DIRS :=  /usr/local/lib /usr/lib  /usr/lib/x86_64-linux-gnu/ /usr/lib/x86_64-linux-gnu/hdf5/serial/


# If Homebrew is installed at a non standard location (for example your home directory) and you use it for general dependencies
# INCLUDE_DIRS += $(shell brew --prefix)/include
# LIBRARY_DIRS += $(shell brew --prefix)/lib

# Uncomment to use `pkg-config` to specify OpenCV library paths.
# (Usually not necessary -- OpenCV libraries are normally installed in one of the above $LIBRARY_DIRS.)
# USE_PKG_CONFIG := 1

# N.B. both build and distribute dirs are cleared on `make clean`
BUILD_DIR := build
DISTRIBUTE_DIR := distribute

# Uncomment for debugging. Does not work on OSX due to https://github.com/BVLC/caffe/issues/171
 DEBUG := 1

# The ID of the GPU that 'make runtest' will use to run unit tests.
TEST_GPUID := 0

# enable pretty build (comment to see full commands)
Q ?= @

@mmderakhshani
Copy link

mmderakhshani commented Jan 7, 2017

Hi,
I would like to know, Is your problem solved? when I run make runset,I have got a same issue on ubuntu 16.04! could you please help me?

@stjordanis
Copy link

stjordanis commented Jan 19, 2017

@MOHAMMAD-PY It still fails in the same way exactly as mentioned above...
I just cloned the latest caffe git repo + I have been updating my ubuntu system normally, running v17 now:
I made the same procedure, but....
when I tried
make runtest
the same things happen.

@svanschalkwyk
Copy link

I get Check failed: error == cudaSuccess (35 vs. 0)
CUDA driver version is insufficient for CUDA runtime version
Ubuntu 16.10 (Mint), gcc5, CUDA 8, Driver 378

@rongfeng-china
Copy link

I installed boost1.58 and added linked library and include path in bashrc.
Then I had an error running ‘make runtest', and finally I found that I made a tiny mistake in bashrc.

The boost lib should be added to LD_LIBRARY, rather than LIBRARY...

@kkarnatak
Copy link
Author

Hi Guys,

I started using https://github.com/BVLC/caffe/tree/opencl

and now I am able to successfully execute make runtest. I am using opencl mode. I havent tried CPU only mode but if you have AMD this will work provided you have successfully installed AMD graphic card. I had to downgrade to 14.04 to install AMD graphic card.

@shelhamer
Copy link
Member

From https://github.com/BVLC/caffe/blob/master/CONTRIBUTING.md:

Please do not post usage, installation, or modeling questions, or other requests for help to Issues.
Use the caffe-users list instead. This helps developers maintain a clear, uncluttered, and efficient view of the state of Caffe.

@fan0916
Copy link

fan0916 commented May 9, 2017

hello all
i have issued this today, and finally i've solved this by changing libtbb2 version
i'm using Ubuntu 16.04 and it uses libtbb2 (4.420160526-0ubuntu1) [universe] as default
from belowlink i found that there is a bug in pre-build package
https://software.intel.com/en-us/forums/intel-threading-building-blocks/topic/636519
so i changed to a new libtbb2 package as follow:
libtbb2 (4.4
20151115-0ubuntu3) [universe] http://packages.ubuntu.com/xenial/libtbb2
this solved my problem
regards!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

8 participants