Blogs

This blog will describe how to add thermal imaging to your drone without breaking the bank! The above photo shows my drone with the thermal imaging camera, the FlirOne device, which is an inexpensive, lightweight thermal imaging camera designed for Android or IPhone devices.  

A flight test of this setup is shown here

The drone platform is arbitrary, as you could follow the instructions here to add thermal imaging to any drone, either copter or fixed-wing UAVs. The platform I chose is a quad copter that is 3-d printed and the printer files for this model are located here.

The FlirOne camera actually has two cameras; a thermal imaging camera, and an HD (1440x1080) camera for mixing the thermal images with the background. The thermal imaging camera has a resolution of 160x120, with a frame rate of 10fps. This a a relatively slow and low resolution, however, this is a 'budget' camera. Higher resolution cameras are VERY expensive and much heavier which balloons the price of the drone also because you need a larger drone. This camera can be mounted on any drone, as it only weighs 36 grams. 

In my design, I have a 3rd camera, a Raspberry PI 2 Cam, that has a much higher frame rate for FPV viewing in flight, however this is not necessary unless you want a higher frame rate. The FlirOne has an HD camera built in which works perfectly well if you can stand the low frame rate. 

I ended up mounting both the FlirOne and the PI Camera on a 3-d printed dummy GoPro, using these 2-Axis Gimbals. 

Setup Architecture

The FlirOne is designed to work with Android, however, strapping a phone or android device to your drone is not that practical compared to using a Raspberry PI.  Fortunately, there are some good hackers out there that figured out the FlirOne USB interface and they were kind enough to publish the driver for Linux using Video-For-Linux (V4L2).  In my configuration, I chose to use GStreamer to stream the video over WiFi to the GCS (Ground control station) as shown in the following diagram:

From the above diagram, you can see there are 3 video streams available. If you are using a PI cam, you can disable the second video stream from the FlirOne, or alternatively, use only the FlirOne for both normal video and thermal imaging. 

Step 1) Configure Raspberry PI WiFi

In order to get video from your drone, you will need to have a wireless connection of some kind, presumably WiFi. It is beyond the scope of this blog to describe how to connect a Raspberry PI to a WiFi ground station. Please refer to this link for documentation on how to setup WiFi. 

For my configuration, I am using a Netis 2561 500mW, 5GHZ WiFi Dongle, with an antenna tracker on the GCS. Using this adapter with a tracker gives several miles range, which makes the drone usable for search and rescue situation where thermal imaging is valuable.

Step 2) Install Linux Driver for the FlirOne 

The FlirOne camera has a dual video stream over a single USB connection, so in order to utilize both streams, you need to install the Linux loopback driver to split the single USB source into two Linux devices. To install the loopback driver you will need to build it using the development tools, or download a copy that matches your firmware revision. 

Please refer to this forum for some additional information on how to setup a Linux system for the FlirOne.  To build the loopback driver use the following commands on your Raspberry PI. Note: you must connect your PI to the internet first.

sudo apt-get install linux-headers-rpi
sudo wget https://raw.githubusercontent.com/not... -O /usr/bin/rpi-source && sudo chmod +x /usr/bin/rpi-source && /usr/bin/rpi-source -q --tag-update
rpi-source
sudo apt-get install bc
rpi-source

git clone https://github.com/umlaeute/v4l2loopback
cd v4l2loopback
make
make install

Note: installing the headers is optional if you already have an image with the kernel source. The kernel source is not included with the standard Raspbian image.

Next, you must build the FlirOne driver which is located here. Unpack this driver and rum the 'make' command supplied with the package. 

The files associated with this project can also be found here.  A compiled version of the loopback driver and the FlirOne driver is included in this link, however the loopback driver won't work unless you are using the exact same version of the Raspberry PI firmware as my configuration. You will probably need to compile the loopback driver locally. I may publish the entire image if there is interest, but it's 16Gb in size.

Step 3) Configure Startup Script to Start Streams

To make the streams start automatically on boot, I use the startup script "rc.local" in the directory /etc.  I have included a copy of this file in the dropbox folder.

Here are the key commands:

sudo modprobe v4l2loopback devices=5

cd /home/pi/flir8p1-gpl
sudo ./flir8p1 Rainbow.raw &
sleep 5

#Video
gst-launch-1.0 v4l2src device=/dev/video2 ! video/x-raw,width=640,height=480 ! jpegenc ! rtpjpegpay ! udpsink host=192.168.1.1 port=9001 &
gst-launch-1.0 v4l2src device=/dev/video3 ! video/x-raw,width=160,height=128, framerate=1/10 ! rtpvrawpay ! udpsink host=192.168.1.1 port=9002 &

The first command loads the loopback module, configuring it for 5 possible devices. In my config, I am actually only using 3, but if you happen to have more devices, you will need to allow for them.

The second command starts the FlirOne driver that connects the USB output to the loopback driver. It will run in the background continuously.

The final two commands are to pipe the output of the two FlirOne cameras to the ground station over WiFI. I am using the IP address 192.168.1.1 and port 9001 and 9002 on my GCS PC, however you may want to use your own setup with different IP addresses and ports.

Step 4) Configure your Ground Control Station to Display the Video

The final step in getting all of this to work is the GCS. You must install GStreamer on your GCS. If you are not familiar with GStreamer, you can download an installer here.

To display the standard video stream use this command:

gst-launch-1.0 udpsrc port=9001 ! application/x-rtp,encoding-name=JPEG,payload=26 ! rtpjpegdepay ! jpegdec ! autovideosink

To display the thermal image, use this command:

gst-launch-1.0 udpsrc port=9002 !application/x-rtp,media=(string)video, clock-rate=(int)90000, encoding-name=(string)RAW, sampling=(string)YCbCr-4:2:0, depth=(string)8, width=(string)160, height=(string)128, colorimetry=(string)BT601-5, payload=(int)96 ! rtpvrawdepay ! autovideosink

If all goes well, you should see both video streams on your GCS PC like this:

Happy Flying!