User programmable sub-gram optical flow / vision sensor

For a long time I've been wanting to make an ultra minimalist vision / optical flow sensor for the hobbyist and experimentalist community. I've been pursuing this as a small IR&D pet project at Centeye. We're almost there.

The above photo shows one of these sensors next to a millimeter scale. The part count is small- One of our 64x64 custom image sensors, an Atmel ATmega644 processor, several resistors and capacitors, and some lightweight flat optics we developed. Two complete sensors are shown, including with mounted optics (yes it's that thin!). Total mass is about 440mg. The primary interface is via I2C/TWI, which will allow many sensors to be hooked up to a common bus. A secondary connector includes the interface with the ISP for uploading firmware.

We chose to use an ATmega processor since they are loved by hardware hackers and are easy to use. Ideally for a single sensor, one can upload any number of different "application firmwares" to the sensor to make it whatever one wants, limited by just the processor and the base resolution. One firmware will turn it into an optical flow sensor . Another firmware will let it track bright lights. Yet another firmware could turn it into something else. Or someone could write their own firmware, whether by tweaking existing source code (yes I plan to share it) or writing something completely new.

An ATmega644 may not sound like much for image processing- 64kB flash, 4k SRAM, 2k EEPROM, 20MHz max. Neither does a 64x64 array. But the reality is if you are witty you really don't need at lot of resolution or processing power to get some nice results. (We once did an altitude hold demo with just 16 pixels an 1MIPS back in 2001.)

We've already made our first batch of these (about 20) and handed them out to a few close collaborators. Based on feedback we are preparing our second run. The new sensors will be slightly larger and heavier (thicker PCB) but more rigid, and use strictly 0.1" headers for all IO and power (including programming). Mass should still be under a gram.

We also have an even smaller version in the works, shown below with a chip mounted and wire bonded (sorry about the mess). This board uses ATtiny and the 7mm x 8mm board alone weighs about 95mg. I think we can get a whole sensor made for about 120mg, if only I had the time! (Maybe some brave person here would like to take a stab at programming it???)

Views: 1615

Comment by Morli on July 6, 2010 at 8:36pm
Good work , very interesting. Are you planning to sell them? if so sign me up pls.Do you have project website where we can see more details of the same. tnx
Comment by Rana on July 6, 2010 at 9:06pm
Barrows, really nice achievement by you !
With optical flow sensor, one can achieve position hold with respect to doors and windows also waypoint navigation is also possible by using continuous video recognition using this optical flow sensor.

Comment by Brian Wolfe on July 6, 2010 at 9:06pm
Nice work indeed. I might be willing to take a stab at the programming of your smaller unit. What time frame are you thinking for having it ready to play with?? I have lots of questions concerning frame rate, sensitivity, FOV, integration time ect.. but maybe we could ease into that.

Comment by OpticalFlow on July 6, 2010 at 10:42pm
Interesting. What eposure mode does the imaging sensor have: is it global shutter or rolling shutter? What frame rate can the sensor run at ? (I have a gut feeling the optical flow computation on the ATMega will be the limiting factor w.r.t frame rate).
Comment by JOHN PIZZICARA on July 6, 2010 at 10:52pm
Cool, how about a phased array compound eye demo for detecting oil well leaks quantitatively?
Comment by Geoffrey L. Barrows on July 7, 2010 at 5:20am
Thanks for your comments! To answer your questions: We do plan to sell them eventually. Not sure of price yet- gold finish + wire bonding the chip to the board + mounting optics makes this a little more complicated than a straight circuit board. The Centeye website has some details on our older work, but I'll probably announce this hardware here.
The pixel circuits are actually continuous-time with a logarithmic response so there really is no shutter. Each pixel produces a voltage and that voltage is read out when needed. The log-response allows it to operate over a large range of light levels.
As for frame rate- that depends on the algorithm and how many pixels you wish to process. It can vary from a few Hz to 100s of Hz, maybe even a kHz in some cases though I haven't yet obtained the latter with this particular hardware. (An older 10MIPS PIC-driven sensor, our original Ladybug, got 1.4kHz with an 88 pixel image sensor.)
Comment by Geoffrey L. Barrows on July 7, 2010 at 5:33am
Regarding the ATtiny sensor- Four of our image sensor designs are very low resolution (8x8, 16x16, or 4x32) and are operated by a clock- pulse the clock and the chip advances to the next pixel, which is output as an analog signal. For the simplest chips, this means you can operate it with ground, power, clock, clear_counter, and analog output e.g. just five connections. I have breakout boards for these chips (about 1" x 1") and would be happy to send out some to people here, if you promise to try them out. (It takes us time to wire bond the chip and glop it.) This is for the chip alone e.g. you'd have to supply a processor and a program, but I can advise you there.
Comment by lionel on July 7, 2010 at 6:10am
This is really great. Even if it weights 20 grams, I will be very interested in having an optical flow building block to be integrated into the UAV for collision avoidance or terrain following
Comment by Maik on July 7, 2010 at 6:32am
I wonder, could you come up with a clever way to use those to make landing smoother? Something about reading the apparent speed of the ground passing by (in pixels per second), comparing that to the actual ground speed, thus being able to measure height?
I have not really stumbled upon a cheap way for smooth autonomous landings. Pressure sensors seem to drift, and sonars or anything of the sort is too expensive, isn't it?
Comment by Curt Olson on July 7, 2010 at 7:10am
Geoff: what kind of data do you hand up stream from your sensor? Is it a 2d vector representing the sensed flow rate in "sensor" space?


You need to be a member of DIY Drones to add comments!

Join DIY Drones

© 2014   Created by Chris Anderson.   Powered by

Badges  |  Report an Issue  |  Terms of Service