The right tool for the job

There have been recent posts on the “wall” about scientific and “toy” cameras for mapping. The focus is on NDVI which is simply an index that provides information about the difference between reflected red and near-infrared radiation from a target. It's an index because it is unitless and it is normalized so values always fall between -1 and +1. It tends to be a good indication of plant vigor and has been correlated to different aspects of plant productivity.

In any digital camera that I'm familiar with a pixel starts its life as a voltage. The next step is where the the scientific and point-and-shoot cameras diverge. In a scientific camera voltage is simply calibrated to output radiance and in a point-and-shoot camera it follows a more complex processing path to output something pleasing to the human eye. Scientific cameras are trying to measure physical variables as accurately as possible and point-and-shoot cameras are trying to make a good looking photograph – science vs art. Point-and-shoot cameras are more complex than scientific imagers but they use lower quality and mass produced parts to keep the costs down whereas the scientific cameras use precision everything which are produced in low volumes. That's a brief summery but the bottom line is that the two different cameras are designed for different uses. To imagine that a camera designed for making pretty pictures can be used for scientific studies seems a bit ludicrous – or does it? It' depends on what you want to do.

There is a good bit of work going on to try and convert point-and-shoot camera from an art tool to a scientific tool. This is an area that fascinates me. I realize there are serious limitations when working with low quality sensors and imaging systems but some (perhaps many) of those radiometric and geometric imperfections can be modeled and adjusted using calibration techniques and software. For example, there are a few articles in the peer-reviewed literature about people calibrating commercial digital cameras (usually DSLRs) to record radiance and the results are pretty encouraging. I have been developing my own work flow to calibrate point-and-shoot cameras although I'm using simple DIY approaches since I no longer have access to precision lab equipment that would allow me to more accurately characterize my cameras. If anyone is interested I post my calibration experiments on the Public Labs web site (http://publiclab.org/). I'm always looking for feedback to advance this work so comments are welcome. My intent is to convert simple cameras to the best scientific tools that is possible.

When deciding which instrument to use you need to consider the goals of the project and available financial resources. For the financial resources you need to consider purchase cost, maintenance and replacement costs if it gets damaged. There is no comparison from a cost perspective. On the bargain side of scientific imagers you should expect to pay a few thousand dollars and if you want a large format mapping camera it's in the ball-park of $1 million. The precision/scientific-grade cameras are very expensive, require careful maintenance and recalibrating (can also be costly), and if you have one in a UAV that crashed you will likely lose a lot. You can get a used digital camera and convert it to an NDVI capable imager for well under $100 or purchase one designed for mapping like the Mapir for about $300.

What about accuracy, precision and stability? Clearly instruments designed with these qualities in mind will be better than something made to make pretty pictures. A more appropriate question is what is good enough for our purposes? I'll focus on NDVI mapping and it's important to realize different applications (e.g., creating 3D point clouds, ortho-mapping, land cover classification) will have other qualities to consider. One important factor to consider is radiometric accuracy. Although I'm trying to improve what we can get from point-and-shoot cameras I realize I will never attain the accuracy or precision possible with scientific imagers. How important are radiometric qualities for NDVI mapping? In most of the applications I see on this and similar forums people are mostly interested in relative changes in NDVI throughout an image and not absolute NDVI values. Some folks want to monitor NDVI over time and in that case it's important to be able to standardize or normalize NDVI but that is possible with calibration work flows. For these applications a well designed and calibrated point-and-shoot cameras can perform good enough to provide the information required such as to spot problem areas in an agricultural field. One point that is often overlooked is that close-range imaging and NDVI typically do not go well together. The problem is that we are imaging scenes with leaves, stems and soil and at the fine resolution provided by most point-and-shoot cameras we are trying to get the NDVI values from very small areas on the ground. For example, we can see different parts of a leaf and each part of the leaf is angled somewhat differently which will effect the NDVI value. Our scenes tend to be very complex and you can have the most accurate and precise instrument available and you might still be disappointed because of the physical issues (bi-direction reflectance, mixed pixels, small area shadows...) that create noise in the images. It is certainly nice to reduce as many sources of noise as possible but with a scientific camera I'm not convinced (at least not yet) that the improved radiometric performance is significant enough to overcome all of the noise coming from the scene to justify their use.

As far as the Mapir camera I received one of these last week and am trying to set time aside to calibrate it and see how well it performs. My initial reaction is that it is a nice compact camera well suited to small UAV mapping. I would prefer a red dual-pass filter but I expect that and other enhancements will become available in future versions. I like the fact that someone is focused on developing practical low-cost mapping cameras.

I welcome any comparisons between cameras and hope we can work together to improve the output we get from simple inexpensive cameras.  

You need to be a member of diydrones to add comments!

Join diydrones

Email me when people reply –

Replies

    • Moderator

      Ned and/or John,

      Do you think it would be possible to have a hosted site that allows an upload of NIR+VIS or NGB mosaics, identify calibration targets, and then provide an assessment of accuracy either overall or in selected regions, using this(below) method? 

      I can't reference specific pages in that article and don't want to quote a large amount here, so please search it for this sentence- "  This term is inversely proportional to the reflectance values of the calibration targets: "

      Also, Thanks for linking this article, Dries. Interesting stuff.

    • There are already quite a few field sites identified for use for different satellite sensors (the article you reference notes a few) but I'm not certain how well these would work for very high resolution imaging. There could be fine scale spectral variations that aren't an issue for 1m and greater resolution but might be at centimeter resolutions. My guess is that portable targets are the way to go but for certain areas it might be possible to document invariant natural features that would work well. 

    • Moderator

      I was mainly referencing the formula for assessing accuracy and reliability of the data. Using mobile calibration sites, what else would we need to perform a similar process? A spectrometer?  

    • I'm interested in the formulas too. 

      Being a field person our group would collect all the ground truthing data while the manned aircraft and/or satellite would fly over.  Now that we're into sUAS the platforms are different but the remote sensing is much the same.  As Ned mentioned the resolution is so much better that instead of looking at a canopy you can now see individual plants and everything around the plant (mixed pixels).

      For manned/satellite missions for invariant targets we would use white/grey gravel parking lots for light targets and ponds for dark targets if the ponds we deep enough and little vegetation.  Irrigation ponds are good.  For sUAS you could use farm roads around the field for light targets but not sure about the dark target as the scenes are so small.  Maybe a kiddy swimmng pool, paint the bottom and sides flat black then fill it with water.  Water is a great absorber of energy.

      LW

    • Moderator

      LW,

      I think in order to try to develop a standard, we need to be using targets with a specific spectral signature and high resolution, correct? 

      A large part of that article was explaining the attempt to make data collected from different platforms at different times, able to be converted to a standard that makes them easy to compare. 

      From what I understand (and hopefully someone will correct me if I'm wrong), the method you're describing would be used for FFC to normalize the current survey and create relative results, but isn't the equivalent of trying to obtain the absolute NDVI value, or R/R. 

    • I think you would need quality targets, a field spectrometer and a well calibrated and stable sensor so you can record radiance. You would also want to be comfortable using ATCOR and MODTRAN and there is a bit of a learning curve for that. You would probably also want a very stable mount for the sensor. There is a fair amount of work setting up an imaging field campaign like the one described in the paper and likely outside the scope of the DIY world. That said I think more simple approaches can get reasonably close. For me the ideal would be to have a system along the lines of the one described in the paper that could be used to further develop DIY systems. For me the main factor holding me back is a lack of reference targets and imagery to do calibration and validation. Nothing a little cash can't solve but that's hard to come by too.

    • Moderator

      And I think that ultimately goes back to John's recommendation of upgrading to purpose-built and calibrated sensors, if that is the goal.

      So, if we're trying to establish the most obvious line where we need to consider upgrading to calibrated sensors, Does making the jump from creating a relative reflectance map to an absolute reflectance map seem like the most obvious position?

      I know I'm beating some points into the ground, but If I'm trying to compile a list of information on NDVI to make this process easier on other people just starting out, I'd like to make sure I'm not adding any poorly-informed opinions from myself.  

    • I think your "making the jump" comment is reasonable as a general rule but it's important to realize it's not a crisp "line" and the choice between a $50 point-and-shoot camera and a several thousand dollar sensor is probably more of a gradient than a jump (increasingly so). There are intermediate options. It's a minor detail but I would use caution using the term "reflectance map" unless you are certain that is what you are creating. 

      It might be interesting to make a matrix of different camera/platform options vs cost to get a rough idea what you can expect for a certain budget. Another matrix could be created for the task vs camera/platform options. I tend to get lost in the caveats and "it depends" thinking so I tend to get discouraged trying to put these guides on paper.

    • Moderator
      I'm hesitant to base the guide around describing the capabilities of each commercial option. I feel that you started this post because of a tug-o-war that was going on between commercial interests (that are admittedly well informed) and the DIY'ers. In my opinion, the goal is to explain in plain language the reason some options are more accurate than others, and what needs to be accomplished for the desired result.
      If anything, the "bare minimum" requirements to create some sort of diagnostic tool is what should have a hard line and be clearly defined. Beyond that, I'd like to see the guide remain commercially neutral and serve as a tool that can be referenced for information to make informed decisions...not a buying guide.
    • Moderator

      Taken from your first link-

      "...a leading project to improve satellite platforms is the Quality Assurance Framework for Earth Observation QA4EO [2]. Based on a set of “inter-operable” guidelines, the QA4EO has provided a framework for the intergovernmental Group on Earth Observations (GEO) to establish a representative, unequivocal and universal quality indicator (QI). The QA4EO advises on how to run similar comparisons for earth observation measurements—whether between sensors or between ground measurement devices."

      Still reading, but I would love to have a standard of measurement for what we are doing that allows us to prove the accuracy or inaccuracies in each data set, especially in a way that growers can use. Many (most?) of them aren't interested in the best, the most accurate, the highest resolution, etc...but it would be nice to be able to use a universal system and show them what their options are for the level of accuracy and reliability they can get with different sensors for NDVI, 

       

This reply was deleted.