Hello. As a long time lurker here, I'd like to share my latest project with the DIY Drones community.
In working with orthophoto generation from UAV imagery using the current crop of still-photo stitching based programs, I have found the process to be complicated and frustrating.
I thought there must be a simpler way, and it occurred to me that pushbroom scanning from video is another way to make nice orthophoto-like images. This works without going through the complex 3D reconstruction process that the expensive stitching programs are using.
I have created a program called Aeriality Video Pushbroom. It is simple, it is available now, and it is very low cost. You can find it at www.aeriality.io
What it does is take an aerial video like this:
And it creates a constant-perspective strip-map image like the one one the left.
Even though it is a simpler process than the 3D reconstruction based stitching, pushbroom scanning has some advantages in comparison.
- Its much faster. Get results in minutes instead of hours.
- It continues to work well where stitching often fails to find key points, such as over farm fields, bare soil, or water.
- When stitching has problems you get holes, gaps, and out of place or highly distorted images. With Aeriality Video Pushbroom errors in a good image are limited to small localized distortions.
- Allows simpler hardware, flight planning and operations. You don't have to worry about image overlap or shutter triggering. And post-flight you only need to work with a single video file instead of hundreds of still photos.
- It runs locally on your machine. No need to wait to upload lots of large files to a cloud service, and no worries about the privacy or security of your data.
- Very efficient for long surveys. You can summarize a video into to an output image faster than real time viewing of the video. Potential applications in surveying power lines, pipelines, fences, towers, agriculture, and search and rescue.
Also, with this software I am testing a theory of business, sales, and pricing, that people will pay a reasonable price for something if it creates value for them. I would prefer to make money only if you are getting value from my work. In fact, I want to create more value for you than I capture.
With that in mind, I made Video Pushbroom with individual $3 licenses that can be used for each single video you want to process (only verifying its cryptographic hash against our server). And if you like the software and find that you are using it a whole lot, or want to use it without an internet connection to our server, there is an unlimited use license that covers you as well. To make it fair, if you ever want to upgrade to the unlimited license, the price of any single uses you have purchased will be deducted from the unlimited price. At the current pricing, that means if Aeriality Video Pushbroom is so useful to you that you have used it 33 times, you can keep using the software for free after that.
This being the internet, I realize that some people like to tear things down. So I thought I would pre-respond (prespond?) to some potential criticisms:
This doesn't actually do any alignment of the pushbroom scan, aren't you aware of all of the work that does this? Photogrammetry is a hard problem and you are not even trying to solve it. This sucks and is not as good as X.
Yes, I am aware of that (clearly, since I'm attacking my own straw-man here). The software will get better and add more features, though with a priority on simplicity. I am aware of the research literature and common techniques used by the computer vision, photogrammetry, and remote sensing communities, and have an advanced background in those fields. I hope that people will find the current software useful as-is, so I am releasing it for sale now (and not selling a dream on Kickstarter).
Thanks for reading. I hope you try my software, www.aeriality.io
thanks
-Taylor
Edit: The original strip map got cut off in the post. A full version is below
Comments
Great idea, are you saying that it currently can't stitch a lawnmower pattern into a photomap? Ie, it only stitches adjacent pairs and not sidelap pairs?
Dear Taylor ,this is very interesting application.As there are very limited application that can provide better results in a less time for uav images, It would be more useful it you add more features georeferncing and mosaicing.
It looks really great and I can imagine plenty of projects where this type of processing would be good enough. well done
I was wondering how do you correct for attitude changes?
Hi Taylor, this is interesting and might be useful for some anti-poaching applications. We'll give it a go with your demo version.
This looks great.
How does your software logic work to identify two successive lines of pixels to stack? Knowing that the drone does not fly at constant speed, shadows, luminosity variations, etc?
Bravo for letting people try it out for only $3!
Hi Chris,
I like your ideas about stills and strip maps augmenting each other. You could maybe run a gps enabled point and shoot with an intervalometer, then use those stills to provide rough reference points for the strip map. A keypoint based feature matching could be used to reference the two together, but it wouldnt need the full homography solution (which is what the stitching programs are doing).
I have also been thinking (after Keith Geary from Low Level Earth Observation mentioned something similar to me), that you could reference the strip maps against some base map like Google Maps imagery. Though the specifics of the imagery will be different between the two, you should often be able to find some gross level features that will match up. This would allow for again a very rough georeferencing of the imagery, which may be good enough in some applications. And it would lend to a simple hardware setup.
Thanks for the input.
What a great idea to used this "push-broom" method for creating strip maps, just like the weather satellites.
It would seem to me that this could make a great companion to a standard still image camera, and vise versa.
The video based strip map could be used to reconcile alignment problems with still image photos and... 'going the other way around', occasional still photos could supplement the video strip map and perhaps provide geo-referenced points, and a small number of still photos could make it possible to sync the flight log with the video.
Hi Dan,
A couple of thoughts. I'm working on a good stitching method for piecing together the lawnmower pattern style of survey. I think that will come eventually, but it does start to run up against the same issues that make the 3D reconstruction based stitching a difficult problem. My low budget approach to that so far has been to simply place the parallel strips next to each other in a sort of manual mosaic. It is surprisingly not bad if you can fly the survey legs accurately (ie without being pushed around by the wind too much).
In other cases it may be that you don't really need a totally accurate orthophoto, but 5 or 6 strip-maps provide a much better understanding of what is on the ground than a few hundred photos that are difficult to orient against each other.
So I am not trying to replace the full blown photogrammetry suite here, but to provide a low cost, simple alternative that may be good enough for a lot of uses.
For the second question, how could this be georeferenced. My plan is to integrate APM log files and let you write out the images as geo-tiffs. So then you just have to sync-up to the log file. If the video is running at the start of the flight you can sync against the arming tone. Or maybe having the autopilot emit a tone at each waypoint would be useful too. Another option is ground control point type referencing, where maybe a single point at each end of the strip can be referenced, and the points in between interpolated. Again, I am not thinking something that is truly centimeter perfect, but that is useable and simple for some applications for which other products are overkill.
I think there are a lot of options here, and hopefully something that we can find a good solution for with feedback from the community.