It doesn't just do image processing, but actually computes the trajectory of the camera in post and calculates a shake-free video from that. Really impressive work from Microsoft Research, which they call "Hyperlapse" (because it is a combination of time lapse and hyperpath finding). With a drone, where the path is already known, you could possibly do this in real-time: stabilization without a gimbal.
We present a method for converting first-person videos, for example, captured with a helmet camera during activities such as rock climbing or bicycling, into hyperlapse videos: time-lapse videos with a smoothly moving camera.
At high speed-up rates, simple frame sub-sampling coupled with existing video stabilization methods does not work, because the erratic camera shake present in first-person videos is amplified by the speed-up.
 Scene Reconstruction |
Our algorithm first reconstructs the 3D input camera path as well as dense, per-frame proxy geometries. We then optimize a novel camera path for the output video (shown in red) that is smooth and passes near the input cameras while ensuring that the virtual camera looks in directions that can be rendered well from the input. |
| Next, we compute geometric proxies for each input frame. These allow us to render the frames from the novel viewpoints on the optimized path. |
 Proxy Geometry |
 Stitched & Blended |
Finally, we generate the novel smoothed, time-lapse video by rendering, stitching, and blending appropriately selected source frames for each output frame. We present a number of results for challenging videos that cannot be processed using traditional techniques. |
We are working hard on making our Hyperlapse algorithm available as a Windows app.
Stay tuned!
Comments
This is really cool, Im very new to Structure from Montion (SfM). but it seems as if it has exploded in the last two years. Besides 3D reconstructions for model building, Ive seen a few examples used like this. I cant wait to when this will work on fly..
Selling my gimbal today :-D
@ Jack +1
The fact that they can't easily produce a user app looks like a lot of manual intervention was required, but not to worry, they've partnered with 3DR.
This is really an amazing accomplishment, but the most amazing part is that it is something that they think they can supply to us for normal use.
The computational requirements have to be extreme, but apparently not as bad as you would think.
Fantastic accomplishment, I suspect the underlying techniques are going to find a lot of new and as yet unforeseen applications as well.
I don't think the point is that it substitute for a good brushless gimbal, the point is that it can produce usable coherent pictorial information from an incoherent mess.
Looked at the technical video: it is awfully complex and it explains how data/information assumptions or losses are inevitable. It does not show so much in their first example but it shows a lot of artifacts or non natural people moves in the second and third examples. It is very innovative but certainly does not come close to a good brushless gimbal professional video.
I almost skipped the second 'Technical' video, but it is worth watching. A remarkable approach and achievement. Just when it appears that media saturation with drone-captured videos has been reached, something new is announced and applied. First-person (first-drone?) views, dronies by remote control, dronies by tethering, drone-swarm views...
That's impressive. Perhaps the most innovative reinventions of post stabilization in a long time.
But it is worth mentioning, that there are certain problems that you will still need mechanical stabilization, like gimbals and/or camera motion based sensor tilt stabilization. Most noticeably to preventing fast camera movements that cause motion blur in the footage, that will severely degrade quality in the resulting video even after post processing.
The golden rule in video production "crap in, crap out", can be circumvented to some degree. But still very much applies.
hope they make a real programm not just an app (and worst: with cloud data upload)!
Thats interesting, little bit like the new photosynth to watch. No doubt we will see this over there as well.