Blogs

Commercial drone applications require significant autonomy and intelligence which cannot be achieved using conventional Computer Vision algorithms. Deep Learning/Vision algorithms seem to be very promising in helping drones to be used in more advanced and complex commercial applications.

Getting a deep learning application working perfectly on a desktop is nontrivial, and when that application has to run on a single board computer aka Companion Computer controlling a drone, the task becomes quite challenging.

FlytOS provides a framework to alleviate these challenges by helping in an easy integration of your deep learning application with drones. It provides you the right set of Drone APIs and Drone SDK to build advanced drone applications.

We will post a series of tutorials to get a simple Deep Learning application like this video

… up and working on a Nvidia TX1 board running FlytOS.
In this tutorial, we will integrate Caffe, a popular deep learning framework widely used for Deep Vision.

For the demo, we will use a famous object detection and localizing network known as the Single Shot Multibox detector or SSD in short (https://github.com/weiliu89/caffe/tree/ssd).

A little knowledge about ROS will be helpful in understanding the tutorial better.

Making a catkin_ws:

We will have to make a catkin workspace for the ROS package we are going to write. If you already have a catkin workspace set up on your TX1, skip this step. In our case our workspace name is flytos_tx1. Clone the following repository: https://github.com/flytbase/flytos_tx1 as a reference to this tutorial.

Making Caffe:

Change your directory to ~/flytos_tx1/src/ssd_caffe/ and clone the caffe repository here by typing following commands:

git clone https://github.com/weiliu89/caffe.git
cd caffe
git checkout ssd_caffe

Now we have to make and install Caffe. You can refer the following link http://www.jetsonhacks.com/2016/09/18/caffe-deep-learning-framework-64-bit-nvidia-jetson-tx1/ as it is a nice resource on building Caffe for the TX1. Or follow these instructions:

Install some dependencies by typing the following commands:

sudo apt-get install cmake libprotobuf-dev libleveldb-dev libsnappy-dev libhdf5-serial-dev protobuf-compiler libatlas-base-dev libgflags-dev libgoogle-glog-dev liblmdb-dev python-dev python-numpy

sudo apt-get install –no-install-recommends libboost-all-dev

Then copy the example Makefile.config.example into a file name Makefile.config

cp Makefile.config.example Makefile.config

If you want to enable the use of CuDNN libraries for accelerated performance, uncomment the line USE_CUDNN in Makefile.config. Then enter the following command:

cmake -DCUDA_USE_STATIC_CUDA_RUNTIME=OFF

Next, add the following line at the end of Makefile.config to solve some issues related to making Caffe on Ubuntu 16.04:

INCLUDE_DIRS += /usr/include/hdf5/serial/

You can finally make caffe by typing these 2 commands:

make -j4 all

make install

Compiling nodes in our ROS package:

If you look at the CMake file our package ssd_caffe (~/flytos_tx1/src/ssd_caffe/CMakeLists.txt), you can see the 2 lines help Catkin in finding the required Caffe libraries to be linked to your ROS nodes.

set(CAFFE_INCLUDEDIR caffe/include /usr/local/cuda/include)
set(CAFFE_LINK_LIBRARY caffe/lib /usr/local/cuda/lib64)

Change your directory back to the catkin workspace and enter the following command to compile your code:

cd ~/flytos_tx1
catkin_make

Right now there is just an empty file in the source directory.

In the next tutorial, you will learn how to make your first ROS node with imported Caffe libraries. Stay tuned!

Thanks to Zubin and Pradeep for preparing this demo.