Arducopter navigation with Wii camera

Hello All,

Using a Wii remote camera for Arducopter navigation is possible.

Following Leon's great blog post showing the use of a camera from a WII remote as a 3D position sensor I've been playing around and having a think about how we could use the Wii remote camera as a sensor on the Arducopter.

First some of the thinking.

Currently the only sensor we have that tells us where we are relative to something else in our environment is the sonar. Some of the other sensors tell us where we are relative to a theoretical datum (eg GPS), or a variable datum (eg air pressure) these could both see you trying to land your aircraft below ground or off in a tree somewhere. The accelerometers tell us how we are oriented relative to gravity but nothing about where we are. The gyros tell us how we are moving. The new work on optical flow which senses the surroundings also tells us how the aircraft is moving but not where it is.

My suggestion is that a combination of an optical sensor such as the wii camera and ground based beacons could be used for relative position determination to allow for precision landings, take offs and even indoor navigation and loitering.

The Wii Camera can track up to four infra red blobs. By sensing the location of a set of predefined IR beacons we can, through simple trigonometry, determine our height above the beacons, our displacement from the beacons and our rotation (yaw) relative to the beacons. This information could then be used to control the navigation algorithms to steer the aircraft to, for example, a precise landing on a 'helipad'.

Work so far

I have procured a broken Wii remote from ebay (only $1.99 yay!) and built up a simple interface circuit as per the examples here. An arduino, a little bit of software and a couple of IR LEDs later and I now have a bench test setup that is working well. The video below shows the bench test at work. The infra red beacons are red circles. The camera is the blue dot. The front elevation is derived solely from the wii camera data. In this video I am flying the camera above two IR beacons 130 mm apart. The only thing the software knows is the distance between the beacons. The accuracy is astounding - less than a centimetre resolution is easily possible at 20hz.

Currently with only two beacons we can't tell if we are rotated 180 degrees. With three beacons in an asymmetric line we could remove any ambiguity about where we are.

Even if we had no knowledge of the layout of the beacons in the aircraft but had sonar for height we could the use this and the camera data to calculate their layout in the real word and use them for navigation or loitering.

Thinking outside the box we could get all Hansel and Gretal and use the arducopter to drop beacons along its flight path to find its way back home. (a simple beacon can be made for less than a dollar).

So could this work?

Pros

  • Works on the bench
  • Doesn't cost much
  • Simple algorithms would not be to costly to run on the APM CPU

Cons

  • The Field of view of the camera is too narrow (+/- 20 deg) to be practical right now. Need to find and test a few wider angle lens types. there are examples of others doing this
  • You can't buy the cameras anywhere - you have to dismantle a Wii remote
  • Need to modify / interface with the Arducopter software

What next?

I need some help to integrate the camera into the Arducopter code. I can just about follow the optical flow code but when it gets into matrix rotations etc my feeble brain hurts. I imagine that this camera navigation could be used in Loiter mode to supplement or replace the GPS data, and a new Auto mode that included specific landing and take off sequences and down the track indoor navigation etc

What do you think?

 

regards

Andrew

You need to be a member of diydrones to add comments!

Join diydrones

Email me when people reply –

Replies

  • This is impressive! I just registered and my plan is to build a copter that could follow our rescue boat (inland water rescue) in some altitude and provide an bird's eye view in missing person searches. Using an IR beacon would make the whole thing MUCH easier than using a GCS.

  • Excellent seeing some work done in this area! Have been thinking of exactly this for quite some time, but never got around to/mastered the technical bits. (ever since I saw Johnny Lees TED Talk on WiiMote hacks http://johnnylee.net/projects/wii/).

    Keep up the good work!

    Johnny Chung Lee - Projects - Wii
  • Developer

    Very interesting and nice write up.  Didn't take you long to get it working!

     

    I think it's really neat/useful functionality but I think the issue with taking it "main stream" is Con #2 - the inability to buy the wii remote chip in quantity...

     

    I wonder if there aren't other image processing chips on the market that could do something similar.  Lots of cameras and mobile phones need to do this sort of thing..surely they can't all be writing their own algorithsm.  I'll bet some company has made a chip to do this kind of thing and it's probably somewhere on digikey.com.

  • It's funny when you upload a video to You tube what it suggests you watch next.

    Looks like University of Tubingen has done similar work.

    See their video on youtube

This reply was deleted.

Activity

DIY Drones via Twitter
RT @chr1sa: My talk on PX4 and FAA certification is coming up at 1:45 PST today on the PX4 Dev Summit livestream. Includes some cool new st…
Tuesday
DIY Drones via Twitter
RT @seesharp: I'm tuned into the PX4 / Dronecode free live conference. Great stuff. Microsoft AirSim talk in 10 minutes. https://t.co/0zbZ2…
Monday
DIY Robocars via Twitter
RT @masato_ka: 距離センサを3つとESP32を付けたラジコンカーをDonkeyCarライクにNNで自動走行。3層FC極小モデルをTensorFlow Lite for microcontrollerで動かしてる。機体は借り物でRumiCarって言います。Tenso…
Monday
DIY Robocars via Twitter
RT @SmallpixelCar: My car was able to go all the way autonomously until the crosswalk. It was only 100 yards from the target. What should b…
Jul 4
Liam left a comment on Agricultural UAVs
"Hi
I'm Liam from T-MOTOR. I would like to reach out to see if there is any possibility for us to work together.
We are a propulsion system manufacturer who offers motors, propellers and ESCs for all kinds of drone applications which vary from secur…"
Jun 30
DIY Robocars via Twitter
RT @SmallpixelCar: Smart move. The car used the shadow to guide it through the bridge. This was never in the training samples. But it learn…
Jun 30
DIY Robocars via Twitter
RT @SmallpixelCar: Getting closer to the target. Single camera. Untrained road. https://t.co/Wsr7RwDamj
Jun 29
Richard Cox left a comment on Australia
"Anyone in the DIYDRONES Australian subgroup based in Alice Springs, NT?
I am experimenting with Ardupilot (standard Arduplane), Pixhawk 4 FC in a 4-ch
RC "AXN Floater Jet" foamy plane..."
Jun 29
Omar Sykes left a comment on Australia
"Hi everyone, I am looking for someone who is good at drone building, repair and software in Adelaide. Please give me a call on 0477 319 219."
Jun 29
DIY Robocars via Twitter
RT @RoboticMasters: #donkeycar https://t.co/czuLoVRcA4
Jun 29
DIY Robocars via Twitter
https://t.co/DKQbNTbz0g
Jun 29
DIY Robocars via Twitter
RT @RoboticMasters: Donkey car, car car car car car car; Donkey car, car car car car car car; Donkey Car. Anyone like our tiny tiny donkey…
Jun 29
DIY Robocars via Twitter
RT @SmallpixelCar: After improving DBSCAN speed, I can get 11 frame per second on @NVIDIAEmbedded Jerson Xavier MAXN mode and the autonomou…
Jun 26
DIY Robocars via Twitter
RT @Heavy02011: Join us at next Virtual Race League: ⁦@diyrobocars⁩ Race #4 - Parking Lot Nerds, August 1st https://t.co/5KUpu7VGaH
Jun 25
DIY Robocars via Twitter
RT @SmallpixelCar: #CUDA implementation should be easy, because most of the time is on distance calculations berween two points and if the…
Jun 25
DIY Robocars via Twitter
RT @SmallpixelCar: This is the view from the car. Even my lanenet inference is fast on @NVIDIAEmbedded Jerson Xavier, DBSCAN clustering tak…
Jun 25
More…