Visual Tracking Gimbal Control

Hey peeps,

So I've been playing around with drone videography lately, find it really hard to film stuff while controlling the camera gimbal to keep things i'm interested in frame.

Being a roboticist, I hacked together a working prototype that uses the video stream to track anything I want, using computer vision algorithms:
https://www.youtube.com/watch?v=QJW9ReKU-0I&list=UUy6SYEB_Gk40j9NOy8RI5aQ

Currently, I have a video tracking software running on PC and control a gimbal to keep the region of interest I select in frame. I'm also planning  to dev. an android/ipad app so that you can just multi-touch define a subject with 2 fingers.

Is anyone out there also be into something like this? Would love to hear your feedback!

Thanks in advanced

You need to be a member of diydrones to add comments!

Join diydrones

Email me when people reply –

Replies

        • Actually I used an off the shelf stepper ptu I found in the lab, but yes brushless will be the next step. Right now i'm running it on an i7, but already looking to deploy it on an ARM board. I'm running at 100hz with decent tracking performance though, so it's not that intensive.

          Another direction I'm hoping to go with this is to create a smart camera for hackers with tracking API and other goodies, but I'll have to see what people are ultimately interested in :)

          • Interesting use of the stepper unit. Always a fan of frankensteining stuff laying around - cool to see you executed it so well. ARM would be ideal for applications as well as some other things I think would be interesting to integrate. I'll send you a message. As for the the smart camera, that seems like a great idea for the hacker/DIY crowd, lots of open source tracking code out there but all seems mediocre or unable to deal with more complex tracking or simply too CPU intensive to use in anything portable - willing to help or test anyway I can or collaborate if you are interested. 

  • Very cool. I'd definitely love to have it to use with my Iris.
  • Awesome. I sure there would be a large (and growing) demand for this. I would love to be able to use ROI to keep the camera facing a certain direction and then visual tracking to follow a specific subject. This would allow for one person to get shots that currently require a pilot and a camera operator working together simultaneously. 

    • Thanks for the response! Yes that was definitely the intention. I am suck at flying and this would be pretty much the only way for me to get any reasonable footage. The intention is to democratize drone videography for casual users like me :)

  • T3
    That's way cool. Very cool programming, I'm impressed.
    • Thanks!

This reply was deleted.

Activity

DIY Robocars via Twitter
RT @a1k0n: @DanielChiaJH @diyrobocars @circuitlaunch Here's my car's view of that race. About 8.4 second lap times for laps 2 and 3... both…
yesterday
DIY Robocars via Twitter
RT @DanielChiaJH: Great racing against @a1k0n today at @diyrobocars! Pretty cool to both break sun-9s at the track today I think I got very…
Sunday
DIY Robocars via Twitter
Broadcasting the @circuitlaunch race live now at https://m.twitch.tv/diyrobocars Races begin around 2:00pm PT
Saturday
DIY Robocars via Twitter
RT @a1k0n: ran a huge number of hyperparameter tuning experiments yesterday; now I can train a new policy, far with better quality, in 15 m…
Saturday
DIY Robocars via Twitter
RT @a1k0n: Did I get rid of hand-tuned parameters? Yes. Am I still hand-tuning more parameters? Also yes. I have a few knobs to address the…
Sep 26
DIY Robocars via Twitter
RT @a1k0n: I'm not going to spoil it, but (after charging the battery) this works way better than it has any right to. The car is now faste…
Sep 26
DIY Robocars via Twitter
RT @a1k0n: Decided to just see what happens if I run the sim-trained neural net on the car, with some safety rails around max throttle slew…
Sep 26
DIY Robocars via Twitter
Sep 24
DIY Robocars via Twitter
RT @SmallpixelCar: @a1k0n @diyrobocars I learned from this. This is my speed profile. Looks like I am too conservative on the right side of…
Sep 24
DIY Robocars via Twitter
RT @a1k0n: @SmallpixelCar @diyrobocars Dot color is speed; brighter is faster. Yeah, it has less room to explore in the tighter part, and t…
Sep 24
DIY Robocars via Twitter
RT @a1k0n: I'm gonna try to do proper offline reinforcement learning for @diyrobocars and throw away all my manual parameter tuning for the…
Sep 23
DIY Robocars via Twitter
RT @circuitlaunch: DIY Robocars & Brazilian BBQ - Sat 10/1. Our track combines hairpin curves with an intersection for max danger. Take tha…
Sep 22
DIY Robocars via Twitter
RT @SmallpixelCar: Had an great test today on @RAMS_RC_Club track. However the car starts to drift at 40mph. Some experts recommended to ch…
Sep 11
DIY Robocars via Twitter
RT @gclue_akira: 世界最速 チームtamiyaのaiカー https://t.co/1Qq2zOeftG
Sep 10
DIY Robocars via Twitter
RT @DanielChiaJH: Always a good time working on my @diyrobocars car at @circuitlaunch. Still got some work to do if I’m to beat @a1k0n howe…
Sep 10
DIY Robocars via Twitter
RT @SmallpixelCar: My new speed profile for @RAMS_RC_Club track https://t.co/RtLb7TcgIJ
Sep 10
More…