Run code on ROS

[Jason-Lee0]

Hello, I appreciate your work and I am very interested your source code.
As the topic,I want to know is it possibe to run on ROS.

If it's possible,I wish you can tell how to build the cmakefile that can build with ROS,thanks.

Thanks your patience to read.
Have a nice day~~

[bijustin]

Hello,

Unfortunately, I have not run this on ROS before, nor do I have a lot of experience with ROS. I primarily worked with the other repository associated with this project, so @ilnehc may be of more help. Thanks for your interest in this project!

[Jason-Lee0]

Hello,

thanks for your reply,Here's another problem ,I want to ask. How to use realsense camera to run RGB-D part,do you ever try this? Thanks.

Have a nice day

[bijustin]

Hello,

Currently the repository is designed to work with RGBD data from the TUM RGBD dataset. I have not used the realsense camera before, but if you can format the RGBD data to be the same as the dataset, it should work properly. If you mean to run the code in real time with the camera, you would likely have to write the interface for grabbing frames, but if the API to input the RGBD data is used properly, the system should work as well.

[KaninchenM]

Hello,

Currently the repository is designed to work with RGBD data from the TUM RGBD dataset. I have not used the realsense camera before, but if you can format the RGBD data to be the same as the dataset, it should work properly. If you mean to run the code in real time with the camera, you would likely have to write the interface for grabbing frames, but if the API to input the RGBD data is used properly, the system should work as well.

Sorry to trouble you. Which version of opencv did you use?

[KaninchenM]

Hello,

thanks for your reply,Here's another problem ,I want to ask. How to use realsense camera to run RGB-D part,do you ever try this? Thanks.

Have a nice day

Have you compiled the code and run the project? Which version of opencv and cuda did you use? Did you use GPU?

[Jason-Lee0]

Hello, I use opencv 3.4.4 to run ./build.sh.
I found the function in the code that opencv 3.4.4 can support.

You can check the compile error to find where cause the error.
Most time it's because opencv version and environment.

cuda and GPU,I didn't check it.Sorry
But i think I didn't use GPU to run the code.

Have a nice day.

[KaninchenM]

Hello, I use opencv 3.4.4 to run ./build.sh.
I found the function in the code that opencv 3.4.4 can support.

You can check the compile error to find where cause the error.
Most time it's because opencv version and environment.

cuda and GPU,I didn't check it.Sorry
But i think I didn't use GPU to run the code.

Have a nice day.

Yes, you're right.
I want to use nvidia GPU to speed up. Because this project is depending on dynaslam(maskrcnn). As the author say in paper, dynaslam(yolo) took 0.5s/frame. It's still not real time and not take the advantages of YOLO.
But when I tried to use opencv's dnn in GPU model, I found only OpenCV 4.2+ supports nvidia GPU reference. When I came to compile and install OpenCV4.3.0, I got another error which showed CUDA10.0+ should be used (I show it in opencv's issue to make sure,here )
Considering I've complete the environment of dynaslam(maskrcnn) and ran it well, I dont want to ungrade my CUDA recklessly.
I'll try to use C++ code to call darknet.
Thank you.

[Jason-Lee0]

Sorry , Do you use GPU to run the code successful?

I revise the code that can use ROS node like ORB_SLAM2,but it ran to slow.
I want to know it's any idea can use GPU to run the code?

Thanks a lot.
Best wishes.

[KaninchenM]

Sorry , Do you use GPU to run the code successful?

I revise the code that can use ROS node like ORB_SLAM2,but it ran to slow.
I want to know it's any idea can use GPU to run the code?

Thanks a lot.
Best wishes.

I think if the code support GPU bases on your GPU and cuda. The followings are what I found:

1. Opencv's dnn.hpp is given in opencv3.3+, and dnn's GPU model only supports Inter GPU until OpenCV4.2 appears. More in :here
2. OpenCV4.2+ needs cuda10.0+.
   This code is based on
   So, if your environment is cuda10.0+&opencv4.2+, you may try the code under GPU. As I said before, I didn't want change my environment. So I compile the darknet(source code of c++ version:I use this one) and call the generated libdarknet.so by c++.

[Jason-Lee0]

Hello,thanks for your reply.

I found opencv 3.4.2 can use net.setPreferableTarget(DNN_TARGET_OPENCL to use Intel's GPU ,but if you didn't have Intel's GPU, it will change to use CPU . I have Intel's GPU to run , but it still very slow. Maybe it is not the good way.

Can I ask you how to compile the darknet ? Did you revise the code about yolo.cc part ?
In your method, did you run the code with GPU successfully ?

If you success, please tell me what did you do.
Thanks~
Best wishes!!

[ilnehc]

Sorry, we didn't implement the GPU or cuda in this project. As the ROS part, this package is not a ROS package. I personally suggest that you make a new dir in your workspace following the ROS package tutorial, copy relevant code and modify CMakeLists there.

[Jason-Lee0]

Thanks for your reply.

Because your method is very useful in SLAM,so I want to use the code in real time.

I know the code you provide didn’t support GPU OR CUDA. So I am modifying the code to make the code in real time.

I have made the ROS NODE and wrote the ROS Cmakelist like orbSlam2 already. If you are interested,I can provide the ROS code for you.

Now, I still try it to be real-time.
Maybe revise your code,upgrade your open code version. Opencv4.2 start to use CUDA ,maybe it can be real-time.

Thanks your paper and code again~

Best wishes.

[KaninchenM]

I've run successfully on GPU and I'll update some information in issue-2

[Jason-Lee0]

WOW thanks for your supply!!!

Because my final purpose is used to NVIDIA TX2, so I am trying to use tensorrt yolo to replace opencv yolo part.
But I am still working on it,when I have new progress,I will let you know if you have interested in.

For now,If your method can help me, I will very appreciate it.

Thanks a lot,
Best wishes!!

[KaninchenM]

WOW thanks for your supply!!!

Because my final purpose is used to NVIDIA TX2, so I am trying to use tensorrt yolo to replace opencv yolo part.
But I am still working on it,when I have new progress,I will let you know if you have interested in.

For now,If your method can help me, I will very appreciate it.

Thanks a lot,
Best wishes!!

I've updated in issue-2. Hope it will help.

[hongu0603]

Thanks for your reply.

Because your method is very useful in SLAM,so I want to use the code in real time.

I know the code you provide didn’t support GPU OR CUDA. So I am modifying the code to make the code in real time.

I have made the ROS NODE and wrote the ROS Cmakelist like orbSlam2 already. If you are interested,I can provide the ROS code for you.

Now, I still try it to be real-time.
Maybe revise your code,upgrade your open code version. Opencv4.2 start to use CUDA ,maybe it can be real-time.

Thanks your paper and code again~

Best wishes.

hello~ I am interested run this program on ROS , can you provide your ROS code for me ?

Thanks & Best regard

[Jason-Lee0]

Thanks for your reply.
Because your method is very useful in SLAM,so I want to use the code in real time.
I know the code you provide didn’t support GPU OR CUDA. So I am modifying the code to make the code in real time.
I have made the ROS NODE and wrote the ROS Cmakelist like orbSlam2 already. If you are interested,I can provide the ROS code for you.
Now, I still try it to be real-time.
Maybe revise your code,upgrade your open code version. Opencv4.2 start to use CUDA ,maybe it can be real-time.
Thanks your paper and code again~
Best wishes.

hello~ I am interested run this program on ROS , can you provide your ROS code for me ?

Thanks & Best regard

Ok,I can share with you.
Did you already compile ORB-SLAM2 for ROS successfully?

Because the step of compile is just like ORB-SLAM2,I have to ensure you’ve already done it.

If you can , please give me more time, I will release the code on GitHub .

Have a nice day.

[hongu0603]

Thanks for your reply.
Because your method is very useful in SLAM,so I want to use the code in real time.
I know the code you provide didn’t support GPU OR CUDA. So I am modifying the code to make the code in real time.
I have made the ROS NODE and wrote the ROS Cmakelist like orbSlam2 already. If you are interested,I can provide the ROS code for you.
Now, I still try it to be real-time.
Maybe revise your code,upgrade your open code version. Opencv4.2 start to use CUDA ,maybe it can be real-time.
Thanks your paper and code again~
Best wishes.

hello~ I am interested run this program on ROS , can you provide your ROS code for me ?
Thanks & Best regard

Ok,I can share with you.
Did you already compile ORB-SLAM2 for ROS successfully?

Because the step of compile is just like ORB-SLAM2,I have to ensure you’ve already done it.

If you can , please give me more time, I will release the code on GitHub .

Have a nice day.

Yes, i already compile ORB-SLAM2 on ROS successfully.

Yes, thanks for your reply.

Thanks & Best regard

[Jason-Lee0]

@hongu0603 soory , I found my ros wrapper of YOLO-DynaSLAM , it's only used for RGB-D camera . It's mean that I only modified "rgbd-tum-yolo.cc"

Because my previous work is used for RGB-D camera.
Which device do you use ?

Do you mind the code only used for rgbd camera ?

If you want the others camera , you might give me more time ,then I will write the code.
Or you could refer my code to modify the others code you want.

[hongu0603]

@ntut108318099 i used realsense-d435i ,so i think only used for rgbd camera is ok.

[Jason-Lee0]

@hongu0603 I have already release the code. (https://github.com/ntut108318099/YOLO-DynaSLAM)

Please notify me immediately if there have any problems.

[hongu0603]

@ntut108318099 OK , Thank you so much.

[Jason-Lee0]

@hongu0603 Did you have any problem to run the code ?

[hongu0603]

@ntut108318099 hello~ when i run ./build_ros.sh it print error

CMakeFiles/Makefile2:709: recipe for target 'CMakeFiles/RGBD.dir/all' failed
make[1]: *** [CMakeFiles/RGBD.dir/all] Error 2
Makefile:127: recipe for target 'all' failed
make: *** [all] Error 2

[Jason-Lee0]

@hongu0603 Could you put all log on the issue ? I think there was some errors in the log.

[hongu0603]

@ntut108318099 OK ~ thank you so much
Building ROS nodes
mkdir: cannot create directory ‘build’: File exists
[rosbuild] Building package YOLO_DynaSLAM
[rosbuild] using multiarch 'x86_64-linux-gnu' for finding Boost
-- Using CATKIN_DEVEL_PREFIX: /home/yui/henry_ws/src/YOLO-DynaSLAM-ros/Examples/ROS/YOLO_DynaSLAM/build/devel
-- Using CMAKE_PREFIX_PATH: /home/yui/henry_ws/devel;/opt/ros/kinetic
-- This workspace overlays: /home/yui/henry_ws/devel;/opt/ros/kinetic
-- Found PythonInterp: /usr/bin/python (found suitable version "2.7.12", minimum required is "2")
-- Using PYTHON_EXECUTABLE: /usr/bin/python
-- Using Debian Python package layout
-- Using empy: /usr/bin/empy
-- Using CATKIN_ENABLE_TESTING: ON
-- Skip enable_testing() for dry packages
-- Using CATKIN_TEST_RESULTS_DIR: /home/yui/henry_ws/src/YOLO-DynaSLAM-ros/Examples/ROS/YOLO_DynaSLAM/build/test_results
-- Found gtest sources under '/usr/src/gmock': gtests will be built
-- Found gmock sources under '/usr/src/gmock': gmock will be built
-- Found PythonInterp: /usr/bin/python (found version "2.7.12")
-- Using Python nosetests: /usr/bin/nosetests-2.7
-- catkin 0.7.29
-- BUILD_SHARED_LIBS is on
-- Using these message generators: gencpp;geneus;genlisp;gennodejs;genpy
[rosbuild] Including /opt/ros/kinetic/share/roslisp/rosbuild/roslisp.cmake
[rosbuild] Including /opt/ros/kinetic/share/roscpp/rosbuild/roscpp.cmake
[rosbuild] Including /opt/ros/kinetic/share/rospy/rosbuild/rospy.cmake
Build type: Release
-- Using flag -std=c++11.
PROJECT_SOURCE_DIR:
PYTHON LIBS were found!
PYTHON LIBS DIRECTORY: /usr/lib/x86_64-linux-gnu/libpython2.7.so
-- Boost version: 1.58.0
-- Found the following Boost libraries:
-- thread
-- chrono
-- system
-- date_time
-- atomic
Boost was found!
Boost Headers DIRECTORY: /usr/include
Boost LIBS DIRECTORY: /usr/lib/x86_64-linux-gnu
Found Libraries: /usr/lib/x86_64-linux-gnu/libboost_thread.so/usr/lib/x86_64-linux-gnu/libboost_chrono.so/usr/lib/x86_64-linux-gnu/libboost_system.so/usr/lib/x86_64-linux-gnu/libboost_date_time.so/usr/lib/x86_64-linux-gnu/libboost_atomic.so/usr/lib/x86_64-linux-gnu/libpthread.so
-- Configuring done
-- Generating done
-- Build files have been written to: /home/yui/henry_ws/src/YOLO-DynaSLAM-ros/Examples/ROS/YOLO_DynaSLAM/build
[ 0%] Built target rospack_genmsg_libexe
[ 0%] Built target rosbuild_precompile
make[2]: *** No rule to make target '../../../../lib/libDynaSLAM.so', needed by '../RGBD'. Stop.
CMakeFiles/Makefile2:709: recipe for target 'CMakeFiles/RGBD.dir/all' failed
make[1]: *** [CMakeFiles/RGBD.dir/all] Error 2
Makefile:127: recipe for target 'all' failed
make: *** [all] Error 2

[Jason-Lee0]

@hongu0603 Did you run ./build.sh first ? It looks like it didn't find the libDynaSLAM.so file .
If you run ./build.sh first, It should be appear in the "lib" library.

BTW, you need to make sure the opencv version Quiet is the same as your CMakelists ( non-ROS, ROS, DBOW2) .

[hongu0603]

@ntut108318099 Oh~ I forgot change the CMakelist opencv version , now is success, thank you.

[ashsifat]

Hi, This is a great work on implementing the ROS for DYNASLAM, I was successful in getting it working. However, as I am trying to combine with other packages that use catkin_make instead of the the legacy method of cmake, is there any hint on how I can change the packages so that it can use catkin_make from the catkin_workspace?
@ntut108318099

TIA
-Ash

[KaninchenM]

您的邮件已收到，我会尽快回复您！