Blogs

This is a continuation of previous blog posts about how 3D reconstructions work. In this image you see a dense reconstruction of the same data set used before. I removed points that have less than 3 matches. You can see gaps where there were trees. Vertical surfaces also have very little data because they are typically in the shade and therefore have poor gradients for matching.

An interesting detail is that I processed the same dataset in visualsfm as well and I actually got a lower-quality result there. The main issue with that output was that the radial distortion parameters weren't estimated correctly. Fortunately, you can recognize such cases, because the point cloud has an overall arcing shape. You can still see that also in this data there's a slight arcing effect, as the roof isn't 100% straight, it sort of arcs over.

When that happens, the densification process has much more difficulty in generating the point cloud and it only generates 30-40% of the points that it would generate in other cases. In other words, the better the estimation of camera parameters and the positions, the better the results of the densification. If you work with visualsfm, you should pay attention to that arcing effect.

As a final step, it's possible to generate an orthophoto from this data. There are actually two main strategies for creating orthophotos. One is through homographies and concatenating them together. A homography describes a relation between two photos that have overlap. You can then apply a transform to every image and insert the results in a work image, the final orthophoto. Such ortho's are iteratively created and when you apply blending to it and exposure compensation and the correct warping algorithm, you get results like this, similar to MS ICE:

It looks very nice, but there are mismatches in the data, for example the hedge at the bottom  and the slight mismatch near the road and the shadow of the shed. You can also see how this method prunes data of low confidence and just leaves it out of the image. So if you have poor overlap or images with different exposures, you should expect poor matches and more artifacts in the end result with this method.

In the example below I ran the surfacing algorithm to generate a 3D mesh of the area and then reprojected the original photos on that mesh. The mesh is then viewed from an orthographic projection from above, hence "orthophoto". This strategy should theoretically yield better results because perspective issues are more reduced (it would simply project pixels of a wall on a vertical surface which later should be invisible). That only works though if the model has a good estimation of height and vertical surfaces. In practice vertical surfaces aren't reproduced very easily, as these are usually in shady areas, which makes matching difficult. You'd typically get blobby artifacts around such regions. Here you can also see that in areas where the height of the 3D model was estimated poorly, you see an overlap between images, because the pixels get projected on either side on the ground. The roof of the shed demonstrates that issue.

This image doesn't look as nice as the first one because it didn't yet apply a strategy for blending and exposure compensation. In areas where the 3D model is poor, you'd expect artifacts. This happens primarily near the borders.

As a final note... in a deliverable of data you typically also have a digital surface model. That can be created very easily from a 3D mode as well by simply testing the height of the data at each pixel of the image. It is essentially a "depth map" of the data.