Blogs

Developer

Object Tracking - Working on Mission Planner Integration

Posted by Adam Rivera on October 11, 2011 at 7:30am


I am currently working on integrating an object tracking algorithm of my own making into the Mission Planner. The algorithm combines Lucas-Kanade and SURF and attempts to learn negative and positive reinforcement values as it runs. Right now it is in the early stages of development, but I hope to have something fully functional within a few months.

 

My hope is to use this to aid in automatic centering of the tracked object by sending commands to the APM to move the camera gimbal based on the distance of the tracked object from the center of the frame. In the videos below you will see two demonstrations of this tracking algorithm. In both cases it seems to perform well with background noise.

 

NOTE: I apologize for the terrible frame rate of the videos. I don't have very good screen capture software. The second video is a little higher quality.


Tags: surf, lucas-kanade, mission planner, object tracking
E-mail me when people leave their comments –
Follow

You need to be a member of diydrones to add comments!

Join diydrones

Comments

  • Developer
    Adam Rivera October 11, 2011 at 4:16pm

    Chuck: This can certainly be adapted for that. If someone wants to buy me one of these http://www.pololu.com/catalog/product/1350 I will be happy to write my code to work with that as well. :-D

  • Chuck October 11, 2011 at 4:06pm

    I'm in with Yusuf.

    I want to film my planes, from the ground.  I have no friends to hold the camera.

     

     

  • Nick October 11, 2011 at 1:36pm

    Thanks I cant wait to try it out!

  • Developer
    Adam Rivera October 11, 2011 at 12:15pm

    Ben: I know what you are saying about the resolution. I'll try to find some really low res video just for you. :-) I would use the one that you linked to, but it is edited in a way that a sequence only lasts like 2 seconds.

  • Developer
    Adam Rivera October 11, 2011 at 12:10pm

    Also, I have no experience in Matlab/Octave, so it became very difficult for me to understand the code and to debug the issues I encountered. I figured it would be good experience just to write my own and see what happens. So far I have been pleasantly surprised with the performance. I work to improve it daily.

  • Developer
    Adam Rivera October 11, 2011 at 12:08pm

    Yusuf: I have seen OpenTLD. It is amazing! Unfortunately it has heavy dependencies on Matlab. I did compile and run in using Octave (a Matlab alternative) but it is a bit buggy at the moment. My approach to object learning is similar to what is used in OpenTLD, but it is not quite as sophisticated. Perhaps as I develop this I will be able to come closer to the performance of OpenTLD. If someone successfully removes the dependencies on Matlab, and I haven't gotten any further writing my own, I would certainly integrate it.

  • Yusuf Pirgali October 11, 2011 at 11:55am

    Adam, i think this is great, as i have been looking for someone to either use the OPENTLD software or use their own. As i would like to also track a flying device and use it to move the filming camera at the object, for self filming and antenna following. So if you have not seen the OPENTLD then go here..https://github.com/zk00006/OpenTLD/wiki

    zk00006/OpenTLD
    OpenTLD is an open source library for real-time 2D tracking of a single object in video. This repository is no longer under development. For latest v…
  • Developer
    Adam Rivera October 11, 2011 at 11:46am

    Ben: No problem and thanks for the well wishes! I just downloaded a new video that was just posted here. I am going to put together a more realistic usage video. I'll post again when I have that finished. Good luck with your FPV setup.

  • Developer
    Adam Rivera October 11, 2011 at 11:07am

    Ben:

    1. The software will run on one's ground station computer. If the machine you are using as a ground station (running Mission Planner) does not support multi-threading or GPU processing, it will revert to a single-thread, no GPU process. 
    2. Any interference in the image could potentially degrade the accuracy of the tracking algorithm, but it is designed to handle such interference. A good example of that can be seen above in the second video. Notice that the helicopter being tracked flies behind some heavy (noisy) brush cover. The software is able to detect the tracked object within a certain threshold of noise. Also, the learning aspect of the software will begin to interpret noise in the frame as positive and negative features, essentially ignoring it.

    That being said, it is not perfect as you can see from the video. There will be times when it loses "sight" of the object. The above test footage I used was more of a worst case scenario with a fast moving object in a high noise environment. If you happen to have some FPV footage on hand that you can send to me, I would be happy to use it as a test and post the results up here.

    Thanks,

    Adam

  • Developer
    Adam Rivera October 11, 2011 at 9:21am

    Dimitri: This work is based on the OpenCV library. What I have done is taken a few existing algorithms and thrown a layer on top to improve accuracy and speed. There isn't one single algorithm that functions perfectly on its own. I have added:

    1. Multi-threading for performance.

    2. GPU processing for performance.

    3. Several mathematical computations to improve accuracy.

    4. Learning (negative and positive).

    5. Logic to deal with occlusion based on the negative and positive learning.

    6. Configurable accuracy threshold

    7. Re-acquisition of target when it is out of frame or if we have lost certainty.

     

    As I stated above, it uses the Lucas-Kanade algorithm for optical flow tracking and SURF for object acquisition based on a feature set. The way in which I have improved and integrated those two with learning is what makes this unique.

This reply was deleted.