I'm learning to love things open source! To see what the fuss was about, I obtained an Arudino Duemilanove board from Sparkfun and decided to play around with it. It didn't take very long for me to assemble a (very) simple optical flow sensor using this board and one of my 16x16 Tam vision chips.
The circuit is very simple- the only electronic components were the Arduino board, a 16x16 Tam vision chip I developed at Centeye, and a single bypass capacitor. The vision chip and the bypass capacitor reside on a one inch (25.4mm) square breakout board. This particular Tam chip is pretty simple to operate- aside from the power signals, it requires two digital inputs, clock and reset counter, and generates one analog pixel output. A counter on the chip determines which pixel is to be output (by row/column) at the output analog line. Every time the clock is pulsed, the counter increments and the next pixel is selected. Pixels are read out one row at a time. The pixel circuits themselves operate in continuous time and are always generating a voltage in response to light. The counter merely determines which pixel voltage is sent to a voltage buffer before being sent off-chip.
A simple Arduino program reads out the pixel signals, digitizes them with the Arduino/Atmel's ADC, and computes a simple one-dimensional optical flow measurement. For the optical flow algorithm, I chose a variation of the Hassenstein Reichardt algorithm, an venerable algorithm from the 1950's that was one of the first proposed neural models for visual motion sensing. The Arduino program then dumps the simple running graph of the optical flow onto the serial dump terminal.
The optical flow algorithm is very simple. Let pA and pB be the signals output by pixels A and B respectively. Let lp( ) be a simple low-pass filter function, which can be implemented as a running average. The optical flow estimated from pixels A and B is merely lp(pA*lp(pB)-pB*lp(pA)), with the outer low pass filter having a longer time constant than the inner low pass filters. If we have an array of pixels A, B, C, D, and so on, then we compute this algorithm once for pA and pB, then again for pB and pC, and again for pC and pD, and so on, and average the results. This certainly isn't the best algorithm one could use, but it was very simple to throw together and I was actually curious to see how it would work.
For this implementation, I'm only reading in the middle 8x8 block of pixels and row-averaging them to form an eight-pixel line image. Thus the optical flow output you see is really from eight pixels worth of image data, or seven individual optical flow measurements averaged together as described in the last paragraph.
The first video above shows the response when the 16x16 Tam chip is exposed to light and a moving card casts a moving shadow across the chip. The second video shows the response when a lens is placed over the chip, so that the image of my moving hand is tracked. The pictures below show the two complete sensor setups, with and without lens, and a close-up of the Tam chip on it's breakout board.
The purpose of this experiment was to see how easy it would be to throw together a simple optical flow sensor using an Arduino board and a simple image sensor chip. The results are certainly crude, but the concept works. I think with some more work a decent Arduino-based sensor can be made, and it could be as easy to hack as pretty much any other Arduino project. (Arduino rocks!)
For those that are curious, I have another post on another forum that shows simple ASCII images taken from the image sensor, and discusses the operation of the chip in greater detail.
(Note: The "Tam" chip here is similar to but not the same as the "Tamalpais" chip used in the recent post on the 125mg sensor. Both are 16x16, but the Tam has larger pixels and is simpler to operate while the Tamalpais is smaller and has better on-board amplification. There is a story behind both names...)
Comments
We could, of course, redesign that board to weigh < 0.5grams including image sensor and optics, though it would be a bit more expensive...
Right now the image sensor chips are in bare die form in waffle packs (see picture below) and we haven't really set up to sell them. I have a little more than 200 each of the 16x16 and 4x32 versions. They need to be wire bonded to a PCB to be useful. Only five wire bonds are needed for each chip (GND, PWR, CK, RESET, ANALOG) so with a few of the remaining breakout boards we could wire bond up a few and send them out.
Right now we are having a number of other chips (more sophisticated 64x64 and 128x256 versions) being wire bonded to small PCBs- I hope to get the first of these back in a week or two, depending on the ability of the wire bonding house to honor quoted production times. :) (We only have a manual bonder in-house)
I would very much like to create one or more Arduino based sensors. A simple version could be a mini-shield, essentially a breakout board that plugs directly into a regular Arduino. For smaller versions we could make a split-type with the Atmel and chip on one board and the FTDI on a second board. You'd plug the latter into the former for programming/debugging. We could probably make that weigh a gram or so, depending on optics. The source code and electronics design would be "open" in the same spirit as the rest of the Arduino systems. (The chip we cannot open since we used third-party IP. Optics is more complicated.)
For those that are interested, I have placed hastily written instructions for the Tam series of chips and my horribly written and commented Arudino sketch here:
http://embeddedeye.ning.com/forum/attachment/download?id=6317171%3A...
http://embeddedeye.ning.com/forum/attachment/download?id=6317171%3A...
@Ravi- thanks! I have already hooked up an Arduino to an external CYE8 sensor- it is easy since all you need is TWI/I2C and it took very little time to get it going. For everyone else: CYE8 is an actual open-source sensor design that I have not really promoted yet- we're still waiting on those wirebonded chip assemblies to get back!!! The project site is here:
http://code.google.com/p/centeye-8bit-sensor/
Regarding optics: Optics has always been the most problematic. There are plenty of solutions for larger image sensors or if you don't care about weight, but there is very little for the 1mm focal plane size. At this scale, we either hand-machine a lens mount and use a 1mm Edmund Scientific lens, or use proprietary printed optics. The latter is cool- we can print up a whole sheet of pinholes for very cheap, and get a pretty wide field of view as well, but the penalty is reduces light sensitivity and low resolution. This is what we used on the related 125mg sensor. We have multiple patents pending for that structure but I'd like to share that at the very least with the experimentalist/hobbyist community.
Another option for lenses would be to simply design one and have that injection molded. This is on our to-do list...
Quite a bit of interest!
Great to see a video. Do you plan on integrating your other sensor module (the CYE8) for use with arduino? Or is that being made for a more general application target?
Are you pursuing the printed lens on the Tam series?
Saw the new website design, nice job with WP. Do you have a page with all the pictures? Seeing some cool ones on the sidebar.