Discussions

Video Encoder Sees Through Fog

Posted by Lew Payne on April 21, 2010 at 10:34am
Would someone knowledgeable care to comment on this Rohm video encoder IC, which claims to "integrate the industry's first fog reduction function along with a unique, hardware-based Adaptive Image Enhancer (AIE) that provides real-time image correction for significantly improved visibility in dark, harshly backlit, or unclear/foggy environments."

In particular, the "before" and after pics just seem too good to be true...

090907_2.jpg


I cannot think of an algorithm or process where so much lost detail could be restored.  Does anyone know something more about what they're doing here, and has anyone adapted this to their UAV platform for clearer video on cloudy or foggy days?

You need to be a member of diydrones to add comments!

Join diydrones

Tags: rohm, encoder, video
Email me when people reply –
Follow

Replies

  • Lew Payne May 18, 2010 at 1:35pm
    Apparently, the technique for "seeing through fog" is called stochastic resonance.

    The folks at Princeton have used it to do various tricks with noisy signals.

    [Hats off to Jason Fleischer and Dmitry Dylov]
  • T3
    Brian Wolfe April 22, 2010 at 5:55pm
    I think Rohm has a nice product here that will find a market but there is a certain amount of "marketing" going on with their selected images. They all are washed out - be it from fog or simply the contrast being turned down and the brightness turned up. This is a situation that is mostly corrected with AGC on a decent camera. I think their market is lower end cmos imagers that don't have good AGC circuits and can't easily gain and offset correct in the analog domain. They may also be throwing in a bit of edge enhancment. I think it's a pretty impressive chip to be doing all this in real time on a video stream.
  • Bill Porter April 22, 2010 at 5:07pm
    I have not seen this with color,

    but,

    I have seen the various DoD R&D projects on image correction like this (mostly in underwater environments with low to no visibility) and it's almost scary how well it works.
  • Gregg "Cabe" Bond April 22, 2010 at 4:54pm
    Short of actual water droplet fog, most loss of contrast is due to UV screwing with sensors. Slap a UV filter in front of it and see if it knocks out some of the haze.

    http://en.wikipedia.org/wiki/UV_filter

    Small filters for the keychain cam optics, et al... http://www.camerafilters.com/pages/uv.aspx
  • Overwatch April 22, 2010 at 12:18pm
    I've done some more research and now I'm leaning somewhat closer to "possible" than before. My statements about the loss of color definition stay, but looks like I've overestimated their effects. Pics look fine even with just 20 shades per channel. I've written a simple glsl shader that selects areas with low contrast and increases it. The resolution is not that great (e.g. the lamp post on the left gets bloomed out, because it's smaller than the 64x64 contrast kernel), but it does "reveal" details previously lost to the eye.



    I'd like to see an image coming from that IC that's larger than 80 pixels though. I mean, what is this, the 19th century?
  • Lew Payne April 22, 2010 at 11:27am
    I've placed an order for two (2) BU6521KV video encoder chips. Both Future and Digikey are out of stock, but the Rohm representative has assured me they have plenty in stock and would supply them to distributors as needed. I intend to use the first one for destructive testing - that's where you inadvertently hook something up backward and blow out the chip. The second one is to do the actual testing on, once I correct my inevitable mistake.
  • Lew Payne April 21, 2010 at 12:19pm
    Okay guys, let's not assume we're using FPV sets or some low-res video camera. In other words, let's not impose artificial limitations in order to try and attack the technology; but rather, let's investigate the technology itself.

    Check out this video demonstration. Note the segments with the cars, and how the only thing left is a "swirl" of fog. It looks too good to be true, but Rohm isn't some fly-by-night company. They're big, established, and have an excellent industry reputation. Very, very curious...
  • Overwatch April 21, 2010 at 11:48am
    I think the demo image is intentionally so ludicrously small that it tricks our minds to fill in the blanks and make us think that the IC actually does miracles. Reality is going to look worse.

    Even if we assume whole 8 bits per color channel ("True Color") - which we know we never pull out of our FPV sets, the quality would not be anywhere near what the image shows. A fog that obscures 50% of the incoming light effectively reduces the available shade space to a half, so instead of 8 bits (256 shades) we end up with 7 bits (127 shades). The fog in the image looks like 80% fog (verified in 'shop). That leaves us with just 50 shades after removing the fog and stretching the contrast back to full, i.e. crappy colors and banding.

    But it will bring out the details that would be otherwise difficult to spot. And it will also enhance all the CMOS, RF and other noises.

    Riccardo, the IC could be doing some kind of a nonlinear reduction, e.g. reducing the middle of the image a little bit more than the sides to make the colors uniformish. But the demo image is faked with a simple contrast reduction, you're right about that.

    This I guess is a little bit closer to reality, with real fog and all.
  • Lew Payne April 21, 2010 at 11:37am
    Page 3 of the spec sheet for the BU6521KV has a block diagram, which reveals that the image data is first fed to a "fog reduction function" before being fed to an AIE (auto-image enhancement) function. Therefore, the "fog reduction" function is actually more than just standard image enhancement techniques. You'll also note the BU6520KV (as opposed to 21KV) on the same page, which has AIE but lacks "fog reduction." This furthers my assumption that the "fog reduction" is more than just standard AIE techniques.

    Now, the question is, does anyone have a hunch as to what this function may be doing? As I said, it's more than just AIE. Wow... I just noticed how redundant I'm being by repeating myself.
  • Jack Crossfire April 21, 2010 at 11:30am
    A dedicated chip for auto levels for $12? It can't even do HD. Maybe if it did unsharp mask, image stacking, & stabilization.
This reply was deleted.

Activity