Hello. As a long time lurker here, I'd like to share my latest project with the DIY Drones community.
In working with orthophoto generation from UAV imagery using the current crop of still-photo stitching based programs, I have found the process to be complicated and frustrating.
I thought there must be a simpler way, and it occurred to me that pushbroom scanning from video is another way to make nice orthophoto-like images. This works without going through the complex 3D reconstruction process that the expensive stitching programs are using.
I have created a program called Aeriality Video Pushbroom. It is simple, it is available now, and it is very low cost. You can find it at www.aeriality.io
What it does is take an aerial video like this:
And it creates a constant-perspective strip-map image like the one one the left.
Even though it is a simpler process than the 3D reconstruction based stitching, pushbroom scanning has some advantages in comparison.
- Its much faster. Get results in minutes instead of hours.
- It continues to work well where stitching often fails to find key points, such as over farm fields, bare soil, or water.
- When stitching has problems you get holes, gaps, and out of place or highly distorted images. With Aeriality Video Pushbroom errors in a good image are limited to small localized distortions.
- Allows simpler hardware, flight planning and operations. You don't have to worry about image overlap or shutter triggering. And post-flight you only need to work with a single video file instead of hundreds of still photos.
- It runs locally on your machine. No need to wait to upload lots of large files to a cloud service, and no worries about the privacy or security of your data.
- Very efficient for long surveys. You can summarize a video into to an output image faster than real time viewing of the video. Potential applications in surveying power lines, pipelines, fences, towers, agriculture, and search and rescue.
Also, with this software I am testing a theory of business, sales, and pricing, that people will pay a reasonable price for something if it creates value for them. I would prefer to make money only if you are getting value from my work. In fact, I want to create more value for you than I capture.
With that in mind, I made Video Pushbroom with individual $3 licenses that can be used for each single video you want to process (only verifying its cryptographic hash against our server). And if you like the software and find that you are using it a whole lot, or want to use it without an internet connection to our server, there is an unlimited use license that covers you as well. To make it fair, if you ever want to upgrade to the unlimited license, the price of any single uses you have purchased will be deducted from the unlimited price. At the current pricing, that means if Aeriality Video Pushbroom is so useful to you that you have used it 33 times, you can keep using the software for free after that.
This being the internet, I realize that some people like to tear things down. So I thought I would pre-respond (prespond?) to some potential criticisms:
This doesn't actually do any alignment of the pushbroom scan, aren't you aware of all of the work that does this? Photogrammetry is a hard problem and you are not even trying to solve it. This sucks and is not as good as X.
Yes, I am aware of that (clearly, since I'm attacking my own straw-man here). The software will get better and add more features, though with a priority on simplicity. I am aware of the research literature and common techniques used by the computer vision, photogrammetry, and remote sensing communities, and have an advanced background in those fields. I hope that people will find the current software useful as-is, so I am releasing it for sale now (and not selling a dream on Kickstarter).
Thanks for reading. I hope you try my software, www.aeriality.io
thanks
-Taylor
Edit: The original strip map got cut off in the post. A full version is below
Comments
Hi Waladi and Andreas, Sorry I missed this question the first time around.
The shortest answer is, it depends on the resolution of the video and of the stills.
To go into a bit more depth, a typical case would be 1080p video, which has 1920 columns of pixels, vs a 12 megapixel still camera which would have 4000 columns.
Typically a camera can use the same field of view in video and still modes, so the amount of ground that is imaged will be the same, however. What changes is the amount of ground covered in each pixel, which is typically called the ground sampled distance (GSD), given as a the length of the side of a square pixel (e.g. 4 cm).
So, there are tradeoffs that can be made. You can get equivalent GSDs by flying lower to cover less area but more finely resolved. Or by setting the zoom of the camera differently in video vs stills to get equivalent GSDs. In the case of 1080p video vs 12mp stills you might need 2 strips to cover the same area at the same GSD.
But it is also not quite an apples to apples comparison, because stitching may not even work with just a single strip of stills (especially without geo-referencing). Stitching programs will often turn straight lines of stills into a bowed or S-curved strip because they are not aware of the strip nature of the images and do not constrain the solution in that way.
Also, better cameras can also take higher resolution video. For instance this one. We probably will not see such high resolution video in a consumer camera though until the next generation of video standards comes around.
So. I hope that answers your questions thoroughly.
best
-Taylor
Hi Taylor,
Great software!!
I second to Andreas question:
1) If i make a strip with still photos, and then a trip out of a aerial video. Will i have the same resolution in both cases? If no how much do change?
Thanks
Waladi
@Paul Thanks! A lot is possible. Things are just getting started now that UAV technology is becoming so accessible.
@lianpinkoh I was able to code up a quick alignment strategy using just the image you posted. It is far from perfect, but it does help a bit. The good thing is that this is actually an area with some significant research behind it. So I should be able to come up with something that will work well in the future. And it will work a bit better in the software as it can be tied in with the pushbroom scanning rate.
Still though I would recommend a gimbal for best results
I will definitely be adding alignment in as an option in a future version of the Aeriality software.
Wow! REALLY?
Taylor... I was thinking that this exact scenario... lots of water at an angle where the surface texture is all constantly moving waves... would be nearly impossible to stitch together at all. You really think you can make something useable out of this kind of material, even if it is shot correctly with a gimbal?
I'm impressed!
I do have to admit though... this finished product would already be pretty useful as "stock footage" for certain photographic effects and as surface texture to use in 3D modeling.
Awesome!
mnem
Huh. Something new under the sun.
@lianpinkoh Cool! Thanks for sharing that. What a beautiful image, even with the distortions.
The buildings and pier at the top look lined up in the vertical dimension but warped crosstrack. And there is a spot that looks roughly correct in about the bottom 3rd quarter of the image. So it looks to me like things were set correctly on the sweep, but there was just a lot of roll motion. I think you will need a gimbal currently to get good results on a fixed wing platform.
I think this video would be a great to help me code up alignment to reduce the effects of roll motions. If you don't mind, could you share your video with me? Maybe PM me with a link on dropbox, or if you like I can give you a login to an account to upload it to.
thanks
-Taylor
This is the output from a non-gimbaled GoPro camera pointing downwards, carried by a swept-wing during a straight part of a mission in Belize. Perhaps I did something wrong but it seems like a gimbal is necessary for this pushbroom approach to work?
.. to complement, your final image is really better than the M-ICE.
i see some distortion on the path in middle of threes when using ice.
Yes, the first time I found that MS ICE did a video stitching mode my heart sank a bit. But after investigation I found a couple of serious problems with using it for aerial videos.
First. Their processing mode was made for video "panoramas" taken from a cell-phone camera. They are looking for a rotating motion where the camera is roughly centered in one spot. When you run it against a straight line flight you tend to get a dog-leg or S curve image out. At first glance it does seem to work fine with the video I posted, the issues really become apparent with longer videos. Even with my demo video it puts a significant bend in the output-- there was not that much lateral translation in the flight.
Second. It is orders of magnitude slower than pushbroom scanning. This is because they are essentially taking many small still images out of the video, then aligning these with the traditional stitching techniques.
Third. When it does go wrong, you have no idea what happened. There is no control over the process.
Fourth. Its free and has no support. Its really just a demo tool from the folks at Microsoft research. It is good at a lot of what it does, but they have no customers and are not trying to make it work for you. I'm working to build a great, simple, low cost, tool for aerial imaging from drones. I will help you solve your problems whenever they come up.
So, well, I'm not trying to trick anyone or be overly salesman-ish here. But I do feel that I should defend my work and not let it be torpedoed. Take a video, try it in ICE, try it in Aeriality. See which one comes out on top for you. And I think with the pricing model for Aeriality, free evaluation, and then cheap to use after that, there is really not much of a risk to trying it.
Yesterday i tried with the video shared here (youtube) and it worked fine.