I would like to "formally" announce two open source projects related to optical flow / programmable vision sensors. These are based on some of the optical flow techniques developed at Centeye, but in the spirit of "open source" are meant to be hacked/modified/copied any way a user deems fit. In both cases, the source code has been opened up under a modified FreeBSD license, while the board design has been released under a Creative Commons Attribution-ShareAlike license, the same license that applies to the Arduino boards.
The first project is the CYE8 sensor, an optical flow sensor based on an Atmel ATmega644 (possibly to be replaced by an ATmega1284) 8-bit processor, and using a Faraya64plus sensor head. (A "sensor head" is a vision chip wire bonded to a 9mm x 9mm PCB with a board to board connector on the other side.) We fabricated our first two iterations last year (the first one described here), and are now readying the third iteration this month.
The hardware of the second project was introduced in a recent blog post and is based on the Sparkfun Arduino Pro Mini platform (which uses a similar but smaller Atmel microcontroller) and comprises a simple shield board that interfaces the Arduino with a sensor head. Currently we have used only a FarayaSmall sensor head, but this board will support other chips as well.
At the current time, these projects are hosted on another open Ning network Embedded Eye, since we are trying to capture a broad set of applications beyond drones. If these projects take off, we can set up Huddle spaces accessible across both Ning networks, or move the project elsewhere. (I'll take suggestions- I'm still learning about how to do projects like this.) The CYE8 project is located here and the Arduino-based project is located here. These forum pages include initial board designs and source codes.
Based on interest, we will likely also launch projects built around an Atmel AVR32 processor (faster than the AVR8's) and/or an XMOS quad-core processor (if you have real need for speed).
One common theme of these two projects (which has strengths and weaknesses) is that they utilize vision / image sensor chips designed by Centeye. (This is not a requirement of the license- it is just how they were designed.) The strengths are that since we designed the vision chips, we can probably reveal as many details of the inner workings of these chips as we want. We all have heard complaints of chip manufacturers being too vague about what is inside, so I hope that this is a welcome change. The weakness is that we basically have to burn new wafers every time we want more sensors, so they are not as available as, say, a part from Digikey.
We are actually going to start a new run of silicon soon with the intent of increasing manufacturing quantity. It is tempting to use this as an opportunity to explore semi-open chips designs. I'd be happy to share a (virtual/real) beer with anyone interested in discussing (whether here, at EE, or directly) the various issues associated with the design and manufacture of chips of this type.
I look forward to speaking with everyone soon!
Geof
Comments
Yes, I think optical flow could be used for more precise position control than you can get with GPS so it would help the heli/quad from moving around as it comes down.
@Johann,
The lower resolution of this sensor makes it appropriate for height sensing (on a fixed-wing vehicle) and avoiding large obstacles (with different firmware or with multiple of these sensors). It would not be enough though to recognize objects like a person's head or other shape. A higher resolution version, with more processing power would though.
Geof
@Randy-
-Actually for the CYE8 we are getting about 300-400Hz or so when processing an 8x8 image. (I think the 20Hz you have is a figure from the 125mg sensor, which used a slower processor, a vision chip w/o an ADC, and no code optimization. You are right that 20Hz is not fast enough to be practical.)
-Lenses- Optics has always been a challenge for us- our focal planes are pretty small (1mm x 1mm for the Faraya64) and thus require a short focal length lens if we want a decent field of view. The trouble is anything available commercially is designed for cell phone cameras, which are larger and have a higher resolution thus require a more complicated design. So a 1-1.5mm focal length lens would cost $50+ from Sunex. Otherwise you would use a longer focal length lens (less expensive) but at the cost of having a narrower field of view. This is fine, perhaps, for tracking the ground, but limits your options elsewhere say for obstacle avoidance. This is why we experimented with the tiny pinhole optics and then made our own injection molded lens (which we have not yet attached to the CYE8).
-First of all I enjoyed reading about the mouse sensor hack- that was very cool! As for a comparison- the mouse sensor is faster, a self-contained unit, and is widely available. On the other hand, it requires more light (probably can only be used outside but I haven't tried myself), provides only one optical flow measurement, and has a fixed algorithm. The CYE8 on the other hand is fully open and can have it's algorithms tuned to a given application, at least within the constraints of the processor. We have worked out techniques (so far only implemented in MATLAB) to compute things like rotation and/or divergence in addition to shifting, so it might be possible to add these things as well. One can also do other things like have multiple algorithms running at the same time and fusing the measurements to obtain a more robust measurement. So these would be the advantages of the CYE8 or another open sensor.
-Right now I have a few CYE8s of version 2 left that can be sent out. Generally I tell academic or commercial customers that they are $200 apiece, since they are partially hand-made in small quantity. (That price includes a full credit towards the purchase of future products- We're definitely not making money off this!) But at this stage, for a bonafide hobbyist or someone who is sincere about developing firmware or other applications, we can reduce that to a nominal amount to cover postage. I am sensitive to the fact that $200 is a bit high for many people.
Geoffrey,
a few questions - will the 20hz refresh rate that can be achieved using the Atmel processors be enough? I'm sure most robots won't consume the results faster than that - just worried that if the robot is moving quickly the firmware won't be able to keep up with the movement.
- I know the optics you're developing are very small/light which is great but how about allowing larger lenses to be attached like a m12 lens? Would that be pointless because you can get the same quality from the tiny lenses?
- what do you think the advantages of the final product could be over the mouse sensor hack? Obviously it's open source so probably can be used for a wider range of purposes. Do you think it would be possible eventually to measure things like rotation?
-Randy
P.S. is it possible to buy one of the sensors to try it out? I'm more comfortable with software rather than hardware.
We have a lot of experience with the AVR32, actually more than the 8-bit AVR, since we have used it for a number of our helicopter flight experiments- we managed to get it to operate 8 vision chips in a sensor ring (more info here). It has nice specs compared to the 8-bit- 60MHz vs. 20MHz, much more memory. It does result in a (slightly) more expensive board because you need 3.3V for this part vs. 4V-5V or so for the vision chip (which has it's own regulator).
We've had a number of internal discussions on whether to use the AVR8 or the AVR32. Although we liked the additional capabilities of the AVR32, the AVR8 seems much easier to use, and a lot more people seem to know how to program the AVR8. So we decided to go with that for the first project.
Yet another possibility would be to make a sensor based on a BeagleBoard, however none of us have experience with this platform. Also, we have a "shield" board for the Surveyor SRV-1 camera, which does work and provides access to the 400MHz Blackfin, but we are not sure of the current viability of Surveyor.
hi..Would like to know more about the same.
SID