Awhile ago we (Centeye) started ArduEye, a project to implement an open source programmable vision sensor built around the Arduino platform. The first ArduEye version used a simple Tam image sensor chip and a plastic lens attached directly to the chip. After much experimentation and some feedback from users, we now have a second generation ArduEye.
The second generation ArduEye is meant to be extremely flexible, ultimately allowing one to implement a wide variety of different sensor configurations. A basic, complete ArduEye is shown below, and contains the following basic components:
An Arduino- Currently we are supporting Arduino UNO-sized boards (e.g. UNO, Duemilanove, Pro) and the Arduino MEGA. When the ARM-driven DUE comes out, we will surely support that as well.
A shield board- this board plugs into the Arduino, and has a number of places to mount one or more image sensor breakout boards. This shield also has places to mount an optional external ADC as well as additional power supply capacitors if desired.
A Stonyman image sensor on a breakout board- The Stonyman is a Centeye-designed 112x112 resolution image sensor chip with an extremely simple interface: 5 digital lines in, which are pulse in predefined sequences, and one analog line out, which contains the pixel. The Stonyman chips are wirebonded directly to a 1-inch square breakout board, which can plug into the shield.
Optics- Possibilities include printed pinholes, printed slits, and cell-phone camera lenses, depending on what you want to do.
Example application- The "application" is an Arduino sketch programmed into the Arduino. This sketch determines what the ArduEye does. One sketch can make it track bright lights, another sketch can measure optical flow, and so on. We are releasing, initially, a base sketch that demonstrates light tracking, optical flow, and odometry. Let us know what other example applications you would like to see.
ArduEye libraries- These libraries are to be installed in your Arduino IDE's "libraries" file, and include functions to operate the Stonyman image sensor chip as well as acquire and process images, including measuring optical flow.
GUI- Finally, we created a basic GUI that serves as a visual dump terminal for the ArduEye. You can now communicate with the ArduEye via either the GUI or the basic Arduino IDE's serial terminal. The GUI was written in Processing.
We designed the system to allow easy hacking to implement a wide variety of vision sensors by exploring combinations of optics, image sensing, and image processing. I personally find it useful, and actually use this system for prototyping things at Centeye- I can prototype a new vision sensor in just a couple hours. The target applications are quite broad and include just about anything that may use embedded vision, whether robotics, sensor nets, industrial controls, interactive electronic sculptures (yes this has come up), and so forth.
The video at the top shows some of the basic things you can do with this ArduEye. You'll see the ArduEye interfacing with a host PC using both the Arduino IDE's serial terminal and the ArduEye GUI. For more details, including links to the hardware design files and source code, go to the ArduEye wiki site. The site is a work in progress, but is adequate to get people started. The sample "first application" and GUI is what was used to generate the above video.
Right now we are having 200 Stonyman breakout boards being assembled- they should be ready within a month. We'll make more if this is well-received. We can assemble a few in-house at Centeye- I'll do this if enough people twist my arm and promise to really play with the hardware. :)
Please let me know your thoughts. In particular, are there any other "sample application" sketches you'd like us to implement?
Comments
Arduino evolving?
eh?
Still crooked pins?
Still has a USB Dildo (Strap-on).
Still virtual USB driver that required a serial port pull down (ask yourselves - would you children buy a product with a comm port pull-down in 2012? - if so could they use it?)
The combination of an evolving Arduino platform with a flexible, fast, and evolving modular optical sensor line certainly looks promising. I'd like to procure and test one for use as an intelligent bubble detector as soon as you can deliver - this would be for an industrial control-like application with high multiples of sensors per unit. Keep us posted.
Looks Great - congratulations - we'll have to pick some up when they're ready...
Thanks for your comments!
Regarding cost: That is still TBD at the moment- we have a good grip on how much it costs to make silicon and wire bond it to breakout boards, but mounting printed optics is a different story- That is pretty easy- look here, though we've found some ways to make it easier- but is still different enough from general PCB manufacture that we haven't been able to get a quote yet.
@Ellison- Sorry if this answer sounds vague, but processor overhead really depends on what you want to do and in particular how fast you want to do it. There is also the issue of timing, e.g. interleaving frame acquisition and processing with whatever else you are doing. I'll take some generic "frame rate" measurements and post them here tomorrow- you can then extrapolate frame rate at 100% overhead (e.g. Arduino 100% committed) to lower levels.
A point aside- I think this algorithm would scream on the upcoming Arduino Due.
@Jack- We are running the serial port at maximum speed- 115.2kbps, and using 2 bytes per pixel- this is a waste, but at least it allows the 10 bit depth from the ADC to be sent down without splitting 10-bit words between bytes. To speed things up we could use a "delta" method, where we send down the first pixel as a literal value, and then for subsequent pixels send down the change from the previous pixel. With a lossless Huffman code and bit-packing, we could probably get a speedup of a factor of four or so.
Regarding 112x112 (=12,544 pixels), you'd be getting 1Hz or so max at 115.2 ksps, assuming 8 bits / pixel. But we could do some funky things to compress to 1 bit/pixel or maybe less... I haven't thought about this system as supporting FPV though...
Must be some snazzy compression to get 112x112 video through the FTDI, but the 16x16 video shown is manageable.
-Jeff
I am interested in ordering one of these... what is the cost...
Wow looks promising and fast enough to implement velocity sensing on a drone. How much does a shield like that cost, and what's processor overhead on the Arduino?