Visually stabilizing a Crazyflie, including in the dark

I've been working on adding visual stabilization to a Crazyflie nano quadrotor. I had two goals- First is to achieve the same type of hover that we demonstrated several years ago on an eFlite 'mCX. Second is to do so in extremely low light levels including in the dark, borrowing inspiration from biology. We are finally getting some decent flights.

Above is a picture of our sensor module on a Crazyflie. The Crazyflie is really quite small- the four motors form a square about 6cm on a side. The folks at Bitcraze did a fantastic job assembling a virtual machine environment that makes it easy to modify and update the Crazyflie's firmware. Our sensor module comprises four camera boards (using an experimental low-light chip) connected to a main board with an STM32F4 ARM running. These cameras basically grab optical flow type information from the horizontal plane and then estimate motion based on global optical flow patterns. These global optical flow patterns are actually inspired from similar ones identified in fly visual systems.The result is a system that allows a pilot to maneuver the Crazyflie using control sticks, and then will hover in one location when the control sticks are released.

Below is a video showing three flights. The first flight is indoors, with lights on. The second is indoors, with lights off but with some leaking light. The third is in the dark, but with IR LEDs mounted on the Crazyflie to work in the dark.

There is still some drift, especially in the darker environments. I've identified a noise issue on the sensor module PCB, and already have a new PCB in fab that should clean things up.

Views: 3267

3D Robotics
Comment by Chris Anderson on March 28, 2014 at 9:32am

Awesome! Can't wait to start using your technology with other projects ;-) 

Comment by Gary McCray on March 28, 2014 at 10:07am

Really interesting application of optical flow and great results.

Seems like if it was combined with a simple ranging system it could be used for navigation in addition to position maintenance.

You could even use the existing cameras to measure the angular offset of a small laser diode spot.

Comment by Geoffrey L. Barrows on March 28, 2014 at 11:32am

@Chris- Agreed!

@Gary- You are right about measuring angular offsets of laser diode spots. Actually if you are in the dark then you can even get by with just an LED and a plastic lens.

Comment by Greg Dronsky on March 28, 2014 at 1:49pm

Great work! it would be even greater if you implemented it to APM! That would be a great leap forward in race between APM and other AP's:)

Comment by Shah Selbe on March 28, 2014 at 2:39pm

Very cool job here!

Comment by Geoffrey L. Barrows on March 29, 2014 at 10:24am

@Greg- Chris and I have been discussing something like this for some time now. The chips on this demo are experimental and a bit different from other chips we have sold in the past. But it would be possible to make something based on the Stonyman chips, or on another chip (Whiteoak) we designed for the Harvard Robobees project and is extremely easy to use. There are even other options- using two chips to make a simple stereo rangefinder (like the fantastic demo by Delfly) or even an open-source alternative to the Leap Motion controller.

Comment by microuav on March 29, 2014 at 12:05pm
Very Nice like all your previous work.
Thank you for mention our DelFly explorer stereo vision system.
Maybe we could work together on a night version of the stereo system?
You make the light weight hardware and we include the algorithms and let it fly autonomous on the DelFly explorer.
Let me know what you think of the idea!
Kind regards
Comment by Jack Crossfire on March 29, 2014 at 2:03pm

The STM32F429 is a beast, relatively speaking.   The QFP ARM microcontrollers still suck compared to what they're offering in BGA's.  Why can't someone just make a super board cam using the latest DSLR sensor technology?

Comment by Greg Dronsky on March 30, 2014 at 2:15pm

@Geoffrey that's great that you are discussing a deal with Chris. I have my money in my hand, and I'm waiting for it to show up in the store :). I'm thinking about some new contraction based on what i found here and some other info sources, and could definitely use your visual stabilisation system. Really impressed with your work!

Comment by Geoffrey L. Barrows on March 31, 2014 at 5:55am

@microuav- PM sent

@Jack- For us moving to a BGA version makes sense when we're ready to put more $ into the fabs. Personally I wish they'd just release a 28-pin or 40-pin QFN version of their STM32F4s- we don't necessarily need a lot of IO. As for making a super cam board- the technology would certainly support it but they need a market for millions first!

@Greg- Thank You for your kind words. I'm curious- is indoor flight something of interest to people here? Pretty much all drone applications are high-up. Flight indoors or in near-Earth environments is a very different beast- a lot more difficult (or else we'd have solved it years ago).


You need to be a member of DIY Drones to add comments!

Join DIY Drones


Season Two of the Trust Time Trial (T3) Contest 
A list of all T3 contests is here. The current round, the Vertical Horizontal one, is here

© 2018   Created by Chris Anderson.   Powered by

Badges  |  Report an Issue  |  Terms of Service