Point cloud densification and orthophoto generation

This is a continuation of previous blog posts about how 3D reconstructions work. In this image you see a dense reconstruction of the same data set used before. I removed points that have less than 3 matches. You can see gaps where there were trees. Vertical surfaces also have very little data because they are typically in the shade and therefore have poor gradients for matching.

An interesting detail is that I processed the same dataset in visualsfm as well and I actually got a lower-quality result there. The main issue with that output was that the radial distortion parameters weren't estimated correctly. Fortunately, you can recognize such cases, because the point cloud has an overall arcing shape. You can still see that also in this data there's a slight arcing effect, as the roof isn't 100% straight, it sort of arcs over.

When that happens, the densification process has much more difficulty in generating the point cloud and it only generates 30-40% of the points that it would generate in other cases. In other words, the better the estimation of camera parameters and the positions, the better the results of the densification. If you work with visualsfm, you should pay attention to that arcing effect.

As a final step, it's possible to generate an orthophoto from this data. There are actually two main strategies for creating orthophotos. One is through homographies and concatenating them together. A homography describes a relation between two photos that have overlap. You can then apply a transform to every image and insert the results in a work image, the final orthophoto. Such ortho's are iteratively created and when you apply blending to it and exposure compensation and the correct warping algorithm, you get results like this, similar to MS ICE:

It looks very nice, but there are mismatches in the data, for example the hedge at the bottom  and the slight mismatch near the road and the shadow of the shed. You can also see how this method prunes data of low confidence and just leaves it out of the image. So if you have poor overlap or images with different exposures, you should expect poor matches and more artifacts in the end result with this method.

In the example below I ran the surfacing algorithm to generate a 3D mesh of the area and then reprojected the original photos on that mesh. The mesh is then viewed from an orthographic projection from above, hence "orthophoto". This strategy should theoretically yield better results because perspective issues are more reduced (it would simply project pixels of a wall on a vertical surface which later should be invisible). That only works though if the model has a good estimation of height and vertical surfaces. In practice vertical surfaces aren't reproduced very easily, as these are usually in shady areas, which makes matching difficult. You'd typically get blobby artifacts around such regions. Here you can also see that in areas where the height of the 3D model was estimated poorly, you see an overlap between images, because the pixels get projected on either side on the ground. The roof of the shed demonstrates that issue.

This image doesn't look as nice as the first one because it didn't yet apply a strategy for blending and exposure compensation. In areas where the 3D model is poor, you'd expect artifacts. This happens primarily near the borders.

As a final note... in a deliverable of data you typically also have a digital surface model. That can be created very easily from a 3D mode as well by simply testing the height of the data at each pixel of the image. It is essentially a "depth map" of the data.

Views: 2122

Comment by Thorsten on April 24, 2014 at 5:37am

Thanks for sharing Gerard! Awesome work! 

I am wondering why the projections are that different between the methods?

Comment by Gerard Toonstra on April 24, 2014 at 6:02am

Thorsten, can you elaborate?  You mean why the projections on the 2nd are curved versus the straight edges in the first?

Comment by Gary Mortimer on April 24, 2014 at 10:48pm

Great stuff as ever thanks for posting.

Comment by Gil Rosenthal on April 25, 2014 at 5:37am

Hi Gerard - Thumbs Up!

I took this route for a while, but got fed up with having to stand on my head (with camera) to

get the software to do its thing and play nice, with acceptable professional results.

I switched to Acute 3D (after robbing a bank), and my life is all roses and cheesecake -

The quality of the results blew me so far out of the water, I'm pretty sure I am not in Kansas anymore.

Call them up (eMail Pascal) and give it a trial run. You will not regret the experience.


Comment by Gil Rosenthal on April 25, 2014 at 5:38am

PS, forgot to mention -

Acute are now coming out with a UAV specific bundle, at a manageable price.

Comment by Thorsten on April 25, 2014 at 6:48am

Gerard, yes that's what I mean. The homography version seems the way to go, since there are by far less artifacts (apart from the blending and exposure compensation). 

Gil, Acute 3D 6500$. Any comparison to Pix4D or Photoscan?

Comment by Gerard Toonstra on April 25, 2014 at 7:23am

Thorsten, the difference is that homographies are created from 2D image information only. The 3D reconstruction method uses triangulation and bundle adjustment to refine results. The 3D method is more dependent on accurate reproduction of 3D geometry to produce good results, as it otherwise overprojects.

One thing I'm still trying out is to derive the homography from the 3D cameras and see if that improves results a bit. The added benefit is that it reduces memory requirements for the stitching process.

The curves in the 3D ortho are the result of inaccurate geometry near the edges, whereas homographies project on a plane. The homography solution correctly eliminated 2 overexposed images, which seem to have poor correlation with the others and don't map correctly in the final solution. So I think homographies probably produce a better result in the end, but it still has to be evaluated for accuracy. In the end, if you have a couple of GCP's, it should be possible to georeference it better through GDAL. 


You need to be a member of DIY Drones to add comments!

Join DIY Drones

DIY Drones Monthly


Season Two of the Trust Time Trial (T3) Contest 
A list of all T3 contests is here. The current round, the Vertical Horizontal one, is here

© 2016   Created by Chris Anderson.   Powered by

Badges  |  Report an Issue  |  Terms of Service