Blogs

Congrats to Marco Walter, who won the the 2017 RoboMagellan competition at RoboGames with a rover that used Pixhawk + Ardurover + Odroid U3

While choosing the right camera and streaming the video from a drone to your app has always been a frequently asked question. There we’ve several Drone API and SDK to build awesome apps for your drones.

What comes along is the selection of a suitable drone Gimbal to stabilize and maneuver that camera. And, what becomes more challenging is when you wish to gain control of gimbal’s Roll, Pitch, and Yawmovements through your code or app.

Whether you are trying to automate the drone gimbal to take that perfect Cable Cam Shot or implement something such as Visual Servoing, Optic flow or any other Machine Vision application, controlling and integrating Gimbal Movements is necessary for your algorithm and hence your app.

FlytBase Drone Gimbal API lets you take control of Roll, Pitch and Yaw movements of your Gimbal through simple API calls. You can send commands from your Companion computer to the Gimbal connected to an autopilot.

Drone APIs are available in JS REST, JS Websocket, ROSC++, ROSPy, C++ and Python to build onboard as well as Mobile/Web application.

You can download FlytSIM – Drone App Simulator to test drone APIs and your Apps.

Visual Servoing Demo and Tutorial

In the Demo below we have demonstrated how a complicated drone application can be built easy & quick by using available FlytBase Drone APIs and Mobile/Web SDK.

We implemented a Visual Servoing application using drone gimbal. Visual servoing, also known as vision-based robot-actuator control, is a technique which uses feedback information extracted from a vision sensor (in this case Camera) to control the motion of a robot/actuator (in this case Gimbal). Below is the image showing the architecture of the Visual Servoing application.

Using this application you can select and lock onto the desired object and Gimbal will keep tracking that object (as long as it’s under the scope of the Gimbal)

The application uses following APIs and SDK

– Vision API to detect and track object

– Gimbal API to control attitude setpoint of gimbal (Roll, Pitch, Yaw)

– Web SDK to build a web-based application interface.

– Video Streaming API to stream the video to the app to see and select the object in the video.

( You can use Navigation APIs to give setpoints and control drone navigation from your app.)

You can find the Source Code for the app on Github in FlytBase Drone Sample Apps Repo.

Follow the Readme to learn: how to do visual servoing on your drone.

For any queries please reach out to us on Forums and Join FlytOS Developers group to engage in the interesting discussions with fellow developers. Also, you can follow us on Twitter.

Sign Up Today to download FlytOS and get build powerful drone applications.

For any queries you can reach out to us on Forums or FlytOS Developers Group

Source

    In order to respond to hobbists's requirement, Skywalker starts to launch KIT version, that is Skywalker WALL-E2000 and Skywalker X2.

    Skywalker WALL-E2000 and Skywalker X2 start to sell KIT version. There are some details under below.  

If you have any question, you can contact with me at skywalker-rc@hotmail.com. 

Characteristics

•  Module design, easy to carry, stable flying, large space, specially parachute cabin, camera slot, flight controller,  professional equipment platform.
•  Adopt EPO materials, high strength body and wings ,long time flying.    
•  Fast disassembly components and adjustable tail distance structure.
•  Bearing large weight load, keep stability.
  
•  Seperated layment design, transparant and clear view.

Excerpt from the IEEE Spectrum article:

“Learning to Fly by Crashing,” a paper from CMU roboticists Dhiraj Gandhi, Lerrel Pinto, and Abhinav Gupta, has such a nice abstract that I’ll just let them explain what this research is all about:

[T]he gap between simulation and real world remains large especially for perception problems. The reason most research avoids using large-scale real data is the fear of crashes! In this paper, we propose to bite the bullet and collect a dataset of crashes itself! We build a drone whose sole purpose is to crash into objects [. . .] We use all this negative flying data in conjunction with positive data sampled from the same trajectories to learn a simple yet powerful policy for UAV navigation.

Cool, let’s get crashing!

One way to think of flying (or driving or walking or any other form of motion) is that success is simply a continual failure to crash. From this perspective, the most effective way of learning how to fly is by getting a lot of experience crashing so that you know exactly what to avoid, and once you can reliably avoid crashing, you by definition know how to fly. Simple, right? We tend not to learn this way, however, because crashing has consequences that are usually quite bad for both robots and people. 

The CMU roboticists wanted to see if there are any benefits to using the crash approach instead of the not crash approach, so they sucked it up and let an AR Drone 2.0 loose in 20 different indoor environments, racking up 11,500 collisions over the course of 40 hours of flying time. As the researchers point out, “since the hulls of the drone are cheap and easy to replace, the cost of catastrophic failure is negligible.” Each collision is random, with the drone starting at a random location in the space and then flying slowly forward until it runs into something. After it does, it goes back to its starting point, and chooses a new direction. Assuming it survives, of course.

During this process, the drone’s forward-facing camera is recording images at 30 Hz. Once a collision happens, the images from the trajectory are split into two parts: the part where the drone was doing fine, and the part just before it crashes. These two sets of images are fed into a deep convolutional neural network (with ImageNet-pretrained weights as initialization for the network), which uses them to learn, essentially, whether a given camera image means that going straight is a good idea or not. After 11,500 collisions, the resulting algorithm is able to fly the drone autonomously, even in narrow, cluttered environments, around moving obstacles, and in the midst of featureless white walls and even glass doors. The algorithm that controls the drone is simple: It splits the image from the AR Drone’s forward camera into a left image and a right image, and if one of those two images looks less collision-y than going straight, the drone turns in that direction. Otherwise, it continues moving forward

Great video from Ardupilot Development Team member Jaime Machuca, on working towards  an upcoming Maker Faire in San Francisco's Bay area. The Ardupilot event is sponsored by Intel.

More  information on the project is available here.

Air Force came in second. Most seem to be using 3DR Pixhawks by the looks of the teaser video. Final video coming soon

http://www.uasvision.com/2017/05/09/usaf-academy-cadets-second-in-uas-competition/

Hi,

Recently we did an endurance test on our AirMast drone. We thought you guys might be interested in seeing the video of it.

In the test the drone was made to hover in the air for 8 hours straight. It was powered from the ground through a lightweight cable. The system features an intelligent winch for the cable, allowing the drone to reach an altitude of up to 100 meters.

Here's the video:



Safe flights,
UgCS Team

 Skywalker X5PRO is professional aircraft that we intend to design this as whole set. It has whole equipment, including ground station, camera, catapult.

If you have any question, you could contact with me. 

Almost any sensor yields more interesting results if mounted on a moving platform. It’s time to mount TOF LIDAR on two precision rotary stages arranged for pan and tilt operation. Rig provides real-time position data along with distance to an obstacle. Using simple math we can calculate position in Cartesian coordinate system. Data is collected point by point to reconstruct 3D object model. After 3D reconstruction and colorizing in MeshLab I got amazing result of my room.

More details in this blog post

--

BR,

Saulius Lukse

Full review at our sister site, DIY Robocars, here, but this is how it starts:

I’m a huge fan of the OpenMV cam, which is a very neat $65 integrated camera and processor with sophisticated built-in computer vision libraries, a Micopython interpreter, and a very slick IDE (a car powered entirely by it came in 2nd in the Thunderhill DIY Robocars race). Now there is a competitor on the block, Jevois, which offers even more power and a lower cost. I’ve now spent a week with it and can report back on how it compares.

I tested it on an autonomous rover, to see how it performed in a stand-along embedded environment (as opposed to being connected via USB to a PC):

Read the rest here...

Hello,

This is a synthesized basic explanation of how precision GPS methods work:
-Differential GNSS (DGNSS)
-Real Time Kinematics (RTK)
-Satellite Based Augmentation System (SBAS)
-Precision Point Positionning (PPP)

Cheers,

Hugues

This blog will describe how to add thermal imaging to your drone without breaking the bank! The above photo shows my drone with the thermal imaging camera, the FlirOne device, which is an inexpensive, lightweight thermal imaging camera designed for Android or IPhone devices.  

A flight test of this setup is shown here

The drone platform is arbitrary, as you could follow the instructions here to add thermal imaging to any drone, either copter or fixed-wing UAVs. The platform I chose is a quad copter that is 3-d printed and the printer files for this model are located here.

The FlirOne camera actually has two cameras; a thermal imaging camera, and an HD (1440x1080) camera for mixing the thermal images with the background. The thermal imaging camera has a resolution of 160x120, with a frame rate of 10fps. This a a relatively slow and low resolution, however, this is a 'budget' camera. Higher resolution cameras are VERY expensive and much heavier which balloons the price of the drone also because you need a larger drone. This camera can be mounted on any drone, as it only weighs 36 grams. 

In my design, I have a 3rd camera, a Raspberry PI 2 Cam, that has a much higher frame rate for FPV viewing in flight, however this is not necessary unless you want a higher frame rate. The FlirOne has an HD camera built in which works perfectly well if you can stand the low frame rate. 

I ended up mounting both the FlirOne and the PI Camera on a 3-d printed dummy GoPro, using these 2-Axis Gimbals. 

Setup Architecture

The FlirOne is designed to work with Android, however, strapping a phone or android device to your drone is not that practical compared to using a Raspberry PI.  Fortunately, there are some good hackers out there that figured out the FlirOne USB interface and they were kind enough to publish the driver for Linux using Video-For-Linux (V4L2).  In my configuration, I chose to use GStreamer to stream the video over WiFi to the GCS (Ground control station) as shown in the following diagram:

From the above diagram, you can see there are 3 video streams available. If you are using a PI cam, you can disable the second video stream from the FlirOne, or alternatively, use only the FlirOne for both normal video and thermal imaging. 

Step 1) Configure Raspberry PI WiFi

In order to get video from your drone, you will need to have a wireless connection of some kind, presumably WiFi. It is beyond the scope of this blog to describe how to connect a Raspberry PI to a WiFi ground station. Please refer to this link for documentation on how to setup WiFi. 

For my configuration, I am using a Netis 2561 500mW, 5GHZ WiFi Dongle, with an antenna tracker on the GCS. Using this adapter with a tracker gives several miles range, which makes the drone usable for search and rescue situation where thermal imaging is valuable.

Step 2) Install Linux Driver for the FlirOne 

The FlirOne camera has a dual video stream over a single USB connection, so in order to utilize both streams, you need to install the Linux loopback driver to split the single USB source into two Linux devices. To install the loopback driver you will need to build it using the development tools, or download a copy that matches your firmware revision. 

Please refer to this forum for some additional information on how to setup a Linux system for the FlirOne.  To build the loopback driver use the following commands on your Raspberry PI. Note: you must connect your PI to the internet first.

sudo apt-get install linux-headers-rpi
sudo wget https://raw.githubusercontent.com/not... -O /usr/bin/rpi-source && sudo chmod +x /usr/bin/rpi-source && /usr/bin/rpi-source -q --tag-update
rpi-source
sudo apt-get install bc
rpi-source

git clone https://github.com/umlaeute/v4l2loopback
cd v4l2loopback
make
make install

Note: installing the headers is optional if you already have an image with the kernel source. The kernel source is not included with the standard Raspbian image.

Next, you must build the FlirOne driver which is located here. Unpack this driver and rum the 'make' command supplied with the package. 

The files associated with this project can also be found here.  A compiled version of the loopback driver and the FlirOne driver is included in this link, however the loopback driver won't work unless you are using the exact same version of the Raspberry PI firmware as my configuration. You will probably need to compile the loopback driver locally. I may publish the entire image if there is interest, but it's 16Gb in size.

Step 3) Configure Startup Script to Start Streams

To make the streams start automatically on boot, I use the startup script "rc.local" in the directory /etc.  I have included a copy of this file in the dropbox folder.

Here are the key commands:

sudo modprobe v4l2loopback devices=5

cd /home/pi/flir8p1-gpl
sudo ./flir8p1 Rainbow.raw &
sleep 5

#Video
gst-launch-1.0 v4l2src device=/dev/video2 ! video/x-raw,width=640,height=480 ! jpegenc ! rtpjpegpay ! udpsink host=192.168.1.1 port=9001 &
gst-launch-1.0 v4l2src device=/dev/video3 ! video/x-raw,width=160,height=128, framerate=1/10 ! rtpvrawpay ! udpsink host=192.168.1.1 port=9002 &

The first command loads the loopback module, configuring it for 5 possible devices. In my config, I am actually only using 3, but if you happen to have more devices, you will need to allow for them.

The second command starts the FlirOne driver that connects the USB output to the loopback driver. It will run in the background continuously.

The final two commands are to pipe the output of the two FlirOne cameras to the ground station over WiFI. I am using the IP address 192.168.1.1 and port 9001 and 9002 on my GCS PC, however you may want to use your own setup with different IP addresses and ports.

Step 4) Configure your Ground Control Station to Display the Video

The final step in getting all of this to work is the GCS. You must install GStreamer on your GCS. If you are not familiar with GStreamer, you can download an installer here.

To display the standard video stream use this command:

gst-launch-1.0 udpsrc port=9001 ! application/x-rtp,encoding-name=JPEG,payload=26 ! rtpjpegdepay ! jpegdec ! autovideosink

To display the thermal image, use this command:

gst-launch-1.0 udpsrc port=9002 !application/x-rtp,media=(string)video, clock-rate=(int)90000, encoding-name=(string)RAW, sampling=(string)YCbCr-4:2:0, depth=(string)8, width=(string)160, height=(string)128, colorimetry=(string)BT601-5, payload=(int)96 ! rtpvrawdepay ! autovideosink

If all goes well, you should see both video streams on your GCS PC like this:

Happy Flying!

Introduction to Deep Learning on drones

Getting a deep learning program working flawlessly on the desktop is nontrivial, so when that application must run on an individual board computer handling a drone, the duty becomes quite challenging.

FlytOS offers a framework to ease these issues by aiding in the easy integration of your profound learning software with drones. FlytOS permits a Profound Learning software (like the next video tutorial) to create and running with a Nvidia TX1 rapidly.

In the previous tutorial, we had built Caffe and set up a Catkin work space for our ROS Caffe node.

In this tutorial we will write an ROS node that takes in images from a USB camera topic that FlytOS publishes, detects and localizes objects in the images, draws bounding boxes on the detected objects and finally, publishes the output on an ROS image topic.

Using the inbuilt video streaming capabilities of FlytOS, the video stream can be viewed in the web browser of any device that is connected to the same network the TX1 is connected to.

For object detection and localization, we use a popular Convolutional Neural Network framework called the Single Shot Multi Box detector (SSD). It is one of the fastest localization network available and is suitable for applications where the object localization is required to be performed in real time. In the context of drones, SSD can be used for applications like surveillance, tracking and following objects of interest, counting cattle etc.

The code for this tutorial is available on https://github.com/flytbase/flytos_tx1. We have taken the CPP example provided in the Single Shot Multibox detector repository and converted it into an ROS node.

The example originally takes input in the form of image files or videos and prints the detected bounding boxes coordinates in the terminal. We modified the input to be ROS image messages being published in FlytOS.

The output is in the form of ROS image messages with the bounding boxes drawn on each image. Images below illustrate example output images from the node.

Running the Code

• For this example, you will have to download a model and pre-trained weights available from this link. If you want to read up on the dataset that this model was trained on and the different classes that it can classify, visit this link. There are more models available at https://github.com/weiliu89/caffe/t… which you can use to experiment. Extract the downloaded zip file and place the deploy.prototxt file and the .caffemodel file in a folder named model in your home folder.

• You should also have FlytOS running on your Nvidia TX1. You can follow this tutorial if it is not the case.

• We use a USB webcam attached to the TX1 for the video source. If you are using the TX1 Development Board, the USB webcam will probably show up with the device name /dev/video1 . You will have to edit the file /flyt/flytos/flytcore/share/vision_apps/launch/cam_api.launch and change the line
  <param name="video_device" value="/dev/video0"/> to
  <param name="video_device" value="/dev/video1"/>

• You will need superuser permission to do so. Reboot FlytOS for the changes to take effect.

• If you have followed the previous tutorial you will have the node ready to run. Make sure you have sourced your catkin workspace in your terminal. You can do that by typing:

source ~/flytos_tx1/devel/setup.bash

• Then launch the SSD node by typing the following command in your terminal (the node takes .prototxt and .caffemodel file paths as arguments):

rosrun ssd_caffe ssd_all_bbox ~/model/deploy.prototxt
~/model/VGG_VOC0712_SSD_300x300_iter_60000.caffemodel

• The model will take a few seconds to load, and then the node will start publishing on the /flytpod/flytcam/detected_objects topic.

• You can view the image stream in your TX1’s browser by opening the link http://localhost/flytconsole, then clicking the video tab and selecting the /flytpod/flytcam/detected_objects topic in the drop-down list that is shown.

This stream can also be seen on any other computer on the same network connection. Just open the following address in your browser:

http://<ip-address-of-TX1>/flytconsole

The screenshot below shows live streaming being viewed in flyt console:

Code explained

We will now explain the modifications we applied to the ssd_caffe.cpp example to convert it into an ROS node.

• Including ROS related header files

//ROS related includes
#include <ros/ros.h>
#include <image_transport/image_transport.h>
#include <sensor_msgs/image_encodings.h>
#include <std_msgs/Header.h>
#include <cv_bridge/cv_bridge.h>

• The following array holds the name of all output classes that the network can classify from.

std::string class_labels[] = {"__background__","Aeroplane","Bicycle","Bird","Boat","Bottle", "Bus", "Car", "Cat", "Chair","Cow", "Diningtable", "Dog", "Horse","Motorbike", "Person", "Foliage","Sheep", "Sofa", "Train", "Tvmonitor"};

• Creating a subscriber and a callback function for the images on /flytpod/flytcam/image_rawtopic:

image_transport::ImageTransport it(nh);
image_transport::Subscriber sub = it.subscribe(“/flytpod/flytcam/image_raw”, 1,
imageCallback);

void imageCallback(const sensor_msgs::ImageConstPtr& msg)
{
try
{
new_img_header = msg->header;
new_img_ptr = cv_bridge::toCvCopy(msg, sensor_msgs::image_encodings::BGR8);
}
catch (cv_bridge::Exception& e)
{
ROS_ERROR(“Could not convert from ‘%s’ to ‘bgr8’.”, msg->encoding.c_str());
}
}

• Cropping the image to a square size and then passing it to the detector

cv::Mat img = img_uncropped(cv::Rect((int(img_uncropped.cols –

img_uncropped.rows)/2),0,img_uncropped.rows,img_uncropped.rows));
std::vector<vector > detections = detector.Detect(img);

• Drawing the bounding box and writing the class name along with the confidence

cv::addWeighted(color, 0.5, roi, 0.5 , 0.0, roi);
cv::putText(img,class_labels[int(d[1])],cv::Point(static_cast(d[3] * img.cols),static_cast(d[4] * img.rows) +25), cv::FONT_HERSHEY_TRIPLEX,0.8,white,1,8);
cv::putText(img,confidence_text,cv::Point(static_cast(d[3] * img.cols),static_cast(d[4] * img.rows) +50), cv::FONT_HERSHEY_TRIPLEX,0.8,white,1,8);

• Publishing the final image

sensor_msgs::ImagePtr pub_msg = cv_bridge::CvImage(std_msgs::Header(),"bgr8", img).toImageMsg();
image_pub.publish(pub_msg);

You are good to go.

For any queries, you can reach out to us on Forums or FlytOS Developers Group

Source

 There are distinguished gathering of Models in China Expo 2017 from April.21-23, including cars. aircrafts, trains and so on, attracted hobbists or dealer from overseas and domestic market. For this event, I believe that the fans have long been waiting to see the exhibition.

I will take on some photos of exhibition to you. This is Beijing exhibition hall that held all kinds of exhibition here.

The red ribbon that used for placing in front of airspeed head. It should take off before flight. This is design by Skywalker.

There are many model aircrafts that you could see under below. Skywalker EVE-2000, Skywalker Titan, Skywalker X2, Skywalker X8, Skywalker X6, Skywalker X5PRO.

Skywalker Titan is our new aircrafts in 2017. I think maybe you would like it if you have preferencial favour to big fuselage. There are still some special inner constructure design which makes it more convenient and professional. It is aimed at professional mappying survey. 
In views of some parts are still in the process of testing, we will advocate this aircrafts afterwards. 

Skywalker X5PRO is also our new aircraft that launched at the end of 2016. This aircraft has excellent function, long-rang flight. Therefore, we intend to design this aircraft as whole set. It has special design catapult, software, ground station. 

Every year Google proposes to fund students for the summer with its GSoC program, where open source organizations and students are announced  through a highly selective process. This year was the first time Ardupilot applied as its own entity, and Google just granted it  four student  positions. No doubt due in part to the high number of student proposals submitted to Ardupilot, and the high quality of many proposals.

This is great news for the drone and robotics community, professional companies, DIYers and professionals alike, and the open source world.  With lots of new development in perspective, adding to the already fast paced and steady stream of Ardupilot development and innovation. 

Congratulations to the student accepted, who will be individually mentored through the summer by some members of the development team. More information about the proposals and selected students can be found here.

Impressive design! APM on Pixracer controller. More here. 

Urus Studio IDE is done! First picture!

Urus Studio is an OpenSource multiplatform IDE designed to build C/C++ Standard ANSI C software and advanced autopilot and industrial real time systems with the Ardupilot power and Urus System for Urus embedded nanokernel, you will can build for Linux, Windows and MacOS Desktop too.

The first beta will be available on few days more!

Thank you APM Team and Code::Blocks Team!

Beta release will be available from the URUS repository on Github coming soon:

https://github.com/UrusTeam

Stay connected!

Walabot is a 3D imaging sensor that lets you uncover things hidden in the world around you. This is very cheap and seems that there is a wide range of applications for working related drones. Just wanted your thoughts.

This is Part 2 of an article I published last week: Low-Latency Live Wireless Flight Video on all your devices (And YouTube Live Streaming it)

Now i'm on to using it with a Convolutional Neural Network.

Well......

Where are we at today. Some years ago, I remember clumsily mashing together my first remote controlled drone using an arduino and a lot of blind faith. Right now I want to say that I feel like it didn't really do much; but it did, and it does, because not only did it fly and stay in the air, it was able to follow gps co-ordinates and balanced itself against the wind, keeping itself in a single space in the sky.  I thought and still do think that is quite a spectacular and amazing thing for humans to have achieved.

Of course, as time progresses, I got used to it doing all of that, and I decided to walk along the path a little more and try to educate myself in autonomous aviation. I learned to build gimbals, and photgrammetrically map routes that I had plotted my flying-arduino to fly. I experimented with sensors, writing code to teach me about many things from sonar sensing, to radiation activity levels. I even learned how to reverse engineer 32-bit micro controllers. I discovered what does work well, and what does not work so well at-this-period-in-time. I feel happy knowing that there are little elves in the background of life who are working on all these things and refining it all to ensure that next Christmas' toys will work perfectly.

I keep going along the autonomous aviation path, but I find myself having to take more frequent breaks from it all to keep my mind (relatively) sane and healthy - I have worked on gardens, and deviated towards electric ebike building. But I have always felt happy knowing that I will return to autonomous aviation to pick up again.

For a while I have found myself leaning towards wireless video communication. Trying to understand the complexities preventing and enabling us to have fast wireless video feeds with low-latency and good connectivity is a big task, there are some people here who each have differing ways of making this happen, all have varying approaches that have some shortcomings and also better implementations than others. I think an open standard will arrive eventually, but not without some internet bloodshed and flamewars going on.

In the meantime, I discovered recently a way to get a live flight video feed on all my screen devices which has enabled me to move forward quite comfortably towards new experiments such as Live Streaming my flight footage on YouTube, and this weekend I have been teaching myself how to run Convolutional Neural Networks (or CNNs) with my homebuilt live flight stream.

It seems to be becoming this year's buzzword - Neural Networks, Machine Learning, Artificial Intelligence, Deep Dreaming, Deep Learning, Training.

There are people desperate out there to champion themselves as an expert or leader of this emerging field, but it is quite contradictory isn't it, in such a fast evolving area of study how can anyone be an expert. We're all finding our way through the dark, it's just some who are adding "I am finding my way through the dark way better than you" to their resume. I have lots of respect for some inspiring people out there on the internet who do wonderful things and it allows me to have better judgement against others.

So, my forays into Convolutional Neural Networks (CNNs), I have thought about somethings to get me up and running as best I can from the get-go. It helps if you can address some things:

1) Hardware - Is it a good idea to use something cheap and less powerful, or something mighty but more expensive? Does it have to be 32-bit or 64-bit? Is it possible to use cloud processing over an internet connection, or does it have to be processed on the hardware locally/offline? Do your chosen peripheral devices work with the hardware?

2) Software - Is it at a stage where it is functional, and also is it functional on your chosen hardware? Will it compile without a blizzard of errors?

3) Time - Are you making good use of it, or are you wasting it trying to force your way to a result? IS it worth abandoning one hardware/software combination as a lost cause in order to try another setup which might bring better success.

Two systems I decided to use are my Raspberry Pi 2, and my Nvidia Jetson TK1. Both have limitations and a fair amount of compromise is required to get a CNN running on either device. I chose my Jetson as a first and obvious choice to begin.

CNN on Jetson TK1

What am I doing?

-On my Nvidia Jetson TK1 Ubuntu Board, I am running two shell scripts. The first one captures a photo image from the drone during its flight (By running the script, not automated or triggered yet).

-The second shell script takes that photo image and runs it through a convolutional neural network (CNN) trained to detect objects, the type of object, and the number of objects. For example, Horses, and the number of horses in a field.

For my experiments I knew that my TK1 board is considered an ageing device. But, I believe you must "do what you can, with what you have, where you are." and for me, tackling the problems of using an ageing hardware device to use in a fast-evolving field is not impossible, but an exercise in self-discovery.

Using the Jetson TK1 board, I was somewhat aware of it's limitation as a 32-bit board. Most CNNs are now developed for 64-bit platforms. An immediate hurdle as one tends not to compile well with the other. Perseverence in some code editing and file/folder renaming helped me to solve that problem, and I got my CNN running on my Jetson.

Initially, I ran my first tests using only the TK1 CPU due to not configuring the code properly. This meant that processing an image took up to 35 seconds.

Setting the CNN up to run using CUDA gpu on the TK1 (Cuda 6.5), it is clearly much faster. Running the same image, it was processed in 0.8 seconds.

I think I will use GPU processing exclusively.... Or so I thought.

(Killed! Gah!)

One of the problems I face using the TK1 with an offline CNN designed for 64-bit is my limited memory availability on the TK1 board. I am constantly facing 'killed' when processing, and I believe this is due to memory limits.

My further experiments will be to try a cloud based CNN or 'Lite' version. However, I can confirm that I am able to capture live flight images from my drone in the sky, capture them on my Jetson TK1

with less than 0.1 millisecond latency, and run my CNN to process the image in under 20 seconds to give me a set of objects identified by the CNN within that image, and it is usually correct.

My goals are:

- To capture live stream video and live process that footage, but without memory limit errors.

- To run a 'take photo' script using a particular visual object source, then process it using the CNN.

- To have the Jetson TK1 perform a command upon detection of chosen object (for example, make LED lights flash, or tweet message, or make the drone do something)

Uses for this other than 'Military or Law Enforcement' - Talking with friends about this stuff, they laugh and call me a military facilitator. But they already probably use CNNs to help them detect threats, make decisions without waking up the General, and to detect civillians within target areas so as to avoid mistakes. But what other uses could this have for normal life?

- Mountain Rescue: Detecting people in Avalanche areas, or lost hill walkers.

- In London it could be used to count the number of houses in the area who use coal fires (heat/smoke identification & counting).

- The number of Diesel vehicles which are in poor condition (billowing excessive smoke)

- Detecting White Rhino Numbers in Chitwan National Park

- The number of tents in a field (Festival safety perhaps)

- Bird Nesting types & numbers

Thanks for reading, and I hope it gives you a little bit of inspiration.

http://dalybulge.blogspot.co.uk/2017/04/part-2-low-latency-live-wireless-flight.html

Hello fellow aviators,

I am part of an undergraduate team of Aerospace Engineers at Oklahoma State University. We have spent all semester doing research on and designing low RCS target drones to help train our Military. This weekend is our final capstone competition. A portion of the scoring metric is Facebook "likes" on our team's marketing video. I wanted to show case our capstone program here at OSU and encourage everyone to check out the competition and to like our video on Facebook.

Feel free to reach out to us here or on FB for any questions about our build. We are finalizing our marketing literature that goes over the objectives, design process, and specs for our 5 aircraft!

Go OSU Orange Team!

Orange Team's Video: https://www.facebook.com/SpeedfestUSA/posts/1498723576812706

Speedfest Facebook page: https://www.facebook.com/SpeedfestUSA/

Speedfest Website: http://speedfest.okstate.edu/

- RAE