Arducopter navigation with Wii camera

Hello All,

Using a Wii remote camera for Arducopter navigation is possible.

Following Leon's great blog post showing the use of a camera from a WII remote as a 3D position sensor I've been playing around and having a think about how we could use the Wii remote camera as a sensor on the Arducopter.

First some of the thinking.

Currently the only sensor we have that tells us where we are relative to something else in our environment is the sonar. Some of the other sensors tell us where we are relative to a theoretical datum (eg GPS), or a variable datum (eg air pressure) these could both see you trying to land your aircraft below ground or off in a tree somewhere. The accelerometers tell us how we are oriented relative to gravity but nothing about where we are. The gyros tell us how we are moving. The new work on optical flow which senses the surroundings also tells us how the aircraft is moving but not where it is.

My suggestion is that a combination of an optical sensor such as the wii camera and ground based beacons could be used for relative position determination to allow for precision landings, take offs and even indoor navigation and loitering.

The Wii Camera can track up to four infra red blobs. By sensing the location of a set of predefined IR beacons we can, through simple trigonometry, determine our height above the beacons, our displacement from the beacons and our rotation (yaw) relative to the beacons. This information could then be used to control the navigation algorithms to steer the aircraft to, for example, a precise landing on a 'helipad'.

Work so far

I have procured a broken Wii remote from ebay (only $1.99 yay!) and built up a simple interface circuit as per the examples here. An arduino, a little bit of software and a couple of IR LEDs later and I now have a bench test setup that is working well. The video below shows the bench test at work. The infra red beacons are red circles. The camera is the blue dot. The front elevation is derived solely from the wii camera data. In this video I am flying the camera above two IR beacons 130 mm apart. The only thing the software knows is the distance between the beacons. The accuracy is astounding - less than a centimetre resolution is easily possible at 20hz.

Currently with only two beacons we can't tell if we are rotated 180 degrees. With three beacons in an asymmetric line we could remove any ambiguity about where we are.

Even if we had no knowledge of the layout of the beacons in the aircraft but had sonar for height we could the use this and the camera data to calculate their layout in the real word and use them for navigation or loitering.

Thinking outside the box we could get all Hansel and Gretal and use the arducopter to drop beacons along its flight path to find its way back home. (a simple beacon can be made for less than a dollar).

So could this work?

Pros

  • Works on the bench
  • Doesn't cost much
  • Simple algorithms would not be to costly to run on the APM CPU

Cons

  • The Field of view of the camera is too narrow (+/- 20 deg) to be practical right now. Need to find and test a few wider angle lens types. there are examples of others doing this
  • You can't buy the cameras anywhere - you have to dismantle a Wii remote
  • Need to modify / interface with the Arducopter software

What next?

I need some help to integrate the camera into the Arducopter code. I can just about follow the optical flow code but when it gets into matrix rotations etc my feeble brain hurts. I imagine that this camera navigation could be used in Loiter mode to supplement or replace the GPS data, and a new Auto mode that included specific landing and take off sequences and down the track indoor navigation etc

What do you think?

 

regards

Andrew

You need to be a member of diydrones to add comments!

Join diydrones

Email me when people reply –

Replies

  • This is impressive! I just registered and my plan is to build a copter that could follow our rescue boat (inland water rescue) in some altitude and provide an bird's eye view in missing person searches. Using an IR beacon would make the whole thing MUCH easier than using a GCS.

  • Excellent seeing some work done in this area! Have been thinking of exactly this for quite some time, but never got around to/mastered the technical bits. (ever since I saw Johnny Lees TED Talk on WiiMote hacks http://johnnylee.net/projects/wii/).

    Keep up the good work!

    Johnny Chung Lee - Projects - Wii
  • Developer

    Very interesting and nice write up.  Didn't take you long to get it working!

     

    I think it's really neat/useful functionality but I think the issue with taking it "main stream" is Con #2 - the inability to buy the wii remote chip in quantity...

     

    I wonder if there aren't other image processing chips on the market that could do something similar.  Lots of cameras and mobile phones need to do this sort of thing..surely they can't all be writing their own algorithsm.  I'll bet some company has made a chip to do this kind of thing and it's probably somewhere on digikey.com.

  • It's funny when you upload a video to You tube what it suggests you watch next.

    Looks like University of Tubingen has done similar work.

    See their video on youtube

This reply was deleted.

Activity

DIY Robocars via Twitter
RT @knightsautoteam: Hi @diyrobocars, we are Orlando's first Autonomous racing club and would love your support. We are hosting our first K…
Jan 20
DIY Robocars via Twitter
RT @Heavy02011: #VirtualRaceLeague: @DIYRobocars Race #14 - #ParkingLotNerds join us January 15th for #AutonomousRacing #RoboRace ⁦@DAVGtec…
Jan 16
DIY Robocars via Twitter
RT @chr1sa: And after that came our races, 50 in all. This battle between these two Russians was the best we've ever seen -- incredible fig…
Jan 16
DIY Robocars via Twitter
RT @chr1sa: Before our @DIYRobocars virtual race this weekend, we had a presentation from the team that won the Indy Autonomous Challenge i…
Jan 16
DIY Drones via Twitter
Dec 12, 2021
DIY Robocars via Twitter
Dec 12, 2021
DIY Robocars via Twitter
RT @chr1sa: Just a week to go before our next @DIYRobocars race at @circuitlaunch, complete with famous Brazilian BBQ. It's free, fun for k…
Dec 4, 2021
DIY Robocars via Twitter
How to use the new @donkey_car graphical UI to edit driving data for better training https://www.youtube.com/watch?v=J5-zHNeNebQ
Nov 28, 2021
DIY Robocars via Twitter
RT @SmallpixelCar: Wrote a program to find the light positions at @circuitlaunch. Here is the hypothesis of the light locations updating ba…
Nov 26, 2021
DIY Robocars via Twitter
RT @SmallpixelCar: Broke my @HokuyoUsa Lidar today. Luckily the non-cone localization, based on @a1k0n LightSLAM idea, works. It will help…
Nov 25, 2021
DIY Robocars via Twitter
@gclue_akira CC @NVIDIAEmbedded
Nov 23, 2021
DIY Robocars via Twitter
RT @luxonis: OAK-D PoE Autonomous Vehicle (Courtesy of zonyl in our Discord: https://discord.gg/EPsZHkg9Nx) https://t.co/PNDewvJdrb
Nov 23, 2021
DIY Robocars via Twitter
RT @f1tenth: It is getting dark and rainy on the F1TENTH racetrack in the @LGSVLSimulator. Testing out the new flood lights for the racetra…
Nov 23, 2021
DIY Robocars via Twitter
RT @JoeSpeeds: Live Now! Alex of @IndyAChallenge winning @TU_Muenchen team talking about their racing strategy and open source @OpenRobotic…
Nov 20, 2021
DIY Robocars via Twitter
RT @DAVGtech: Live NOW! Alexander Wischnewski of Indy Autonomous Challenge winning TUM team talking racing @diyrobocars @Heavy02011 @Ottawa…
Nov 20, 2021
DIY Robocars via Twitter
Incredible training performance with Donkeycar https://www.youtube.com/watch?v=9yy7ASttw04
Nov 9, 2021
More…