Hello, I would like to show you my recent project with official name:
An On-Drone Dynamic Object Track and Follow Solution for Quadcopters
Quadcopters are flying drones with a highly constrained payload capacity, limited computation power as well as communication capabilities. Such quadcopters can be equipped with a low weight camera and an additional computer. There are many applications where the autonomous tracking and following of a moving object is desired. For example filming of fast moving athletes during sport events, studying wild life in regions difficult to access or surveillance and pursuing of criminals. These reasons made us create a new vision-based dynamic object track and follow solution for quadcopters. The solution can be deployed on a consumer grade drone with an additional on-board computer. The outcome of our solution is a quadcopter able to track and follow an object chosen by the user in a video frame during initialization. In our project, we first evaluate a few promising computer vision algorithms for object tracking based on their suitability for a quadcopter. Afterwards we choose three the most suitable trackers whose performance is tested on a use case video and evaluated based on various criteria. The most suitable tracker is implemented in the Robot Operating System framework which is used for communication between the drone and the Ground Control Station . The setup is tested on two drone platforms as well as in a simulation. We found the right combination of an on-board computer and tracker which are suitable for our project. The result of a drone able to track and follow an object is shown in a video.
More Informations you can find here!
This is just test video testing the parts of the algorithm on ardrone. The final quadcopter which will be able to carry the computer is still in development.
In case of any questions, do not hesitate to contact me!
Comments
@Patric, Yes, just very basic HSV was used. But HSV is still for this application not great choice.
TX1 is great choice and I will take a look on it. The problem is that the development board is really large and it is not possible to mount it on UAV. I will look on another possibility or some smaller board. Maybe I will try to design some basic one for USB and GPIOs. If you know about some another possibility just let me know :)
Juraj, if you go with the TX1, make sure you do an education discount ($300 instead of $600). If you google "tx1 education discount" it should come up. Awesome project!
Thanks Juraj for the info.
Must admit that I was quite impressed when, a few years ago, I first saw the speed and accuracy of the predator from Zdenek Kalal. It is very interesting to see that you were able to get it flying. This is a relaly cool demonstration of the TK1, just like Randy wrote, cant stop to see it run on the TX1 !
Getting back to hardware, you mus be using really basic HSV conversion to get 30fps on the RPi2 ?
Just adding some Gaussian and it throttles down to 15 Fps and then putting cvHoughCircle (for Balloon_Finder) and it goes to 6Fps I am still hesitating to upgrade to ODROID XU4 because I do not see any real good GP implementation, I know that @Kabir is using I7 on his development , but he's looking for Nvidia as wel, but looking at NVIDIA FORUM, it seems that TX1 still needs some fixing..So TK1 might be the solution for the next year.. What do you think ?
@Patric: yes, I did fly with it. All systems worked perfectly with APM, just for hovering I needed around 75% of max powe. This setup is not the best one and I did not have time to order new motors and rebuild it.
I have started with Ronan0912s ROS TLD but I had to change really a lot of things from his implementation. As U can see in the video I created new GUI where ROS TLD from Ronan0912 is just a single widget.
There is not much to see from cuda OpenTLD because it uses libopencv4tegra which is already accelerated with CUDA. The issue is that this libopencv4tegra is not compatible with ROS. There is necessary tons of configurations and some changes in ROS-base packages to make them use the libopencv4tegra and not normal opencv. If somebody would be interested in this tutorial, I can create one.
@Juraj: Did you had a chance to finally fly the costume drone
What about the ROS TDL TRACKER is this based on existing code (like Ronan0912) or it is your own brew?
It would be really interesting if you could show more of the CUDA openTDL Code
its really super......
This is really great, well done!
Yes it is, it uses nvidia accelerated opencv libraries.
On the same Nvidia TK1 or different Board and did you use OpenTLD?
I've been able to achieve 30fps, limited by camera framerate on a ARM CPU only. No parallelisation, just fairly optimised algorithm.
Yes, TK1 is able to make 23fps with the tracker, what is really great, the reason for that are the Nvidia Cuda Cores and parallel computing. This nvidia boards are great choice for computer vision.
I am already looking forward form TX1. I will definitely give it a try.
The problem of current quadcopter is that it uses APM 2.8, which unfortunately doesn’t have implemented mavlink functionality which overrides the desired roll/pitch/yaw angles but just the rc values from radio. I have written interpreter which gives to possibility to interpret the roll pitch and yaw angles in rc values which are than sent to apm, so it acts as normal transmitter. Is available here .This is very poor solution and probably I will have to either change it in apm code or switch to pixhawk.
-
1
-
2
of 2 Next