Finally the last missing board from the PX4 series has been released, the PX4FLOW smart camera module. It can replace GPS in indoor and outdoor applications and provides a metric position close to the ground with only very little drift. It is essentially a microcontroller hooked up to an automotive-grade machine vision sensor that can be freely programmed.
Given the wide distribution of the AR.Drone one might ask what the purpose is, and the answer is simple and well illustrated by the video: You can use this board with PX4FMU and the PX4 native autopilot stack, but you can also interface it with any other system, including Linux onboard computers running ROS. It works indoors and outdoors and brings the same stability the AR.Drone shows in flight to any aerial robot.
It has a native resolution of 752×480 pixels and calculates optical flow on a 4x binned and cropped area at 250 Hz (bright, outdoors), giving it a very high light sensitivy. Unlike many mouse sensors, it also works indoors and in low outdoor light conditions without the need for an illumination LED at 120 Hz (dark, indoors).
- 168 MHz Cortex M4F CPU (128 + 64 KB RAM)
- 752×480 MT9V034 image sensor
- L3GD20 3D Gyro
- 16 mm M12 lens (with IR block filter)
As the aerial image with overlaid trajectory shows, the position estimate is very accurate. This is without GPS, captured in flight at 1.6 m altitude and in one pass.
The work on this module has been accepted at the International Conference on Robotics and Automation (ICRA 2013) in Karlsruhe, Germany: Dominik Honegger, Lorenz Meier, Petri Tanskanen and Marc Pollefeys. An Open Source and Open Hardware Embedded Metric Optical Flow CMOS Camera for Indoor and Outdoor Applications, ICRA2013
The module has been developed by Samuel Zihlmann, Laurens Mackay, Dominik Honegger, Petri Transkanen and Lorenz Meier.
Frequently Asked Questions
- When will it be available? - It is available here and starts shipping next Monday, according to 3D Robotics
- Is the software available? - The software will be made available shortly open-source licensed. The module comes pre-flashed with the latest state.
- Is a ROS interface available? - Yes, here
- Will you offer a low-cost version? - The module has been designed to meet scientific standards and provide a baseline for what one can achieve with a reduced design. A low cost version is not planned, but we're interested to hear if someone is willing to contribute a cell-phone camera based design.
- Can I hook it up to robot XY? - Almost for sure, as it outputs the flow already in m/s and in MAVLink format
- Has it been used in scientific work yet? Yes, for example in this paper by Fraundorfer et al.
More information needed?
Comments
Is this product dead? I don't see where you can buy it on the 3DR store anymore. If it's out of the product line, was it killed from lack of interest or did it have problems and issues?
Could some one point me to a thread where this sensor is being used with a Pixhawk?
Can we expect the ability to use two of these camera systems in stereo to enhance their function?
Just picked mine up today! Looks great.
No GPS underwater, so position and velocity are a lot harder to determine. Camera needs a pressure housing(dive light), and the ultrasonic range finder speed of sound is going to be off. But air attenuates ultrasonic frequencies worse than water so the effective range may increase.
I'm going to observe concrete through a Pyrex dish full of water and see how well this might work in swimming pools.
Any spare ADC channels or clock cycles for an FFT on audio?
https://www.youtube.com/watch?v=mzj32w8mEWA&feature=youtu.be
APM / ACM on PX4 will certainly get an interface to this sensor. It gives you a full 3D position over the ground plus angular speed and a full state estimate together with the autopilot sensors. Its also not just useful close to the ground, its accuracy (in the mm / cm range at 2 m altitude) just scales linearly up with altitude. So whenever the system is close to obstacles in ground proximity you get substantially better accuracy than you would ever get from GPS, if you climb up higher at some point GPS gives you better results (but only at altitudes you would normally not fly a multicopter manually, since you have a hard time seeing how it is oriented). If you have seen a Parrot AR.Drone flying, you should have an idea how it performs and in which environment settings.
I was just thinking of this and I guess it would be completely possible to get roll from these sensors by watching the distortion of the image however and with a lot less noise than a IMU. the challenge is that it would be difficult to sense the difference between sloped earth and roll/pitch.
based on what I have read, It measures ground movement below the quad copter, which means that if you quad is moving slow enough or the camera is going fast enough you can get accurate, relative 3 axis navigation (x, y, yaw) - the sonar gives you a 4th axis (z). If you know your start position and can measure relative position accurately then you know where you are. I suppose it is possible to get roll and pitch within reason but I suspect that the system has been simplified to just give those 4 axes... I guess we see when the source is available.
Can someone say more about how this sensor is being used? "provides a metric position close to the ground" is not very clear. My best guess is it replaces a sonar sensor to determine position above the ground.
With several of these surouning the craft, maybe eight of them like a spider eye system. Sense and Avoid, the big kahuna, the holey cup.
yeesh... it has the maxb. sonar attached.... is this removable?
-
1
-
2
of 2 Next