Running an autopilot on Android

Here's a fun little project I've been working on for ~6 months now. It started when I looked at my phone and thought this has everything needed for an autopilot - 3-axis gyros, accelerometers, magnetometers, GPS, fast CPU, plus a nice big touch screen and more wireless modems than you can point a stick at. The only problems are (1) it's a multitasking operating system, not real-time; and (2) needs extra hardware to drive servos and switch between manual and auto control.

I solved the first problem by buying a used Samsung Nexus S with a faulty screen (discolouration) on ebay for $80, installing a stable, bare-bones version of android 2.3, rooting it, and disabling CPU throttling. 50Hz servo update loops and 110Hz sampling loops are no problem - timing is reliable and it only uses ~5% CPU (using floating point everywhere and some bloaty object-oriented code).

Second problem was solved like this:

The Ardupilot legacy board doesn't actually run any ardupilot code - it's just a convenient and cheap way to get an Arduino and a multiplexer. It just takes servo commands from the phone and uses the Arduino servo library code to drive the servos. Most of its time is spent sampling the baro sensor. It samples at around 20kHz, and using 4096x oversampling I get 4 16-bit samples per second with an altitude resolution of around 15cm and noise of about +/- 30cm. The bluetooth link looks scary but works fine. It's done about 10 hours of autonomous flight with no issues, and only drops out if I take the phone at least 10 meters away from the plane.

Waypoints are selected on a google map interface:

Gains etc can be easily adjusted:

And, of course, a flight video:

Behind the scenes it runs the DCM + complimentary filter code. I've added some extra features over the last few months:

Autonomous thermaling (works well in warmer weather)

Hardware in loop simulation with X-Plane and MS FSX over WiFi

Wind and airspeed estimation

Dead reckoning (using gyros/magnetometer and estimated speed - can work through short GPS dropouts)

Android-to-Android Telemetry over WiFi or mobile internet (later requires sim with public IP - hard to find)

Features I plan to add:

Waypoint addition/modification from Android groundstation (if/when I can convince a telco to give me public IPs)

Fly-by-wire(less) over mobile internet (ditto)

Photo/video taking at specified locations with onboard camera (will require mounting under wing)

I'll post about some of these over the next few weeks.

Hi John, very interesting project. I signed up just to comment XD. Using the mobile network is definitely the way to go in order to perform long range missions. Here are my two cents as Android programmer:

The phone doesn't need to be the "server" in terms of opening connections. Don't be picky here, the code design doesn't need to suffer. You can open a connection to the control station server, then "serve" your telemetry info as well as consume commands. It is a two way communication channel. I'm working in a very similar remote desktop project to control a PC from a phone and vice versa. Since the phone can drop connections due to lack of coverage, it makes sense for the phone to reconnect to ground station.
As for problem #1, it is extremely rare to see SIMs with public IP, but you don't need it either. I can think of two workarounds: 1) Configure a VPN in your phone and stablish a secure tunnel with ground station; and 2) Let the phone be the "client" in terms of opening connections. VPN is interesting since it provides a security layer and that way your phone will have a local IP you could use.
Android has a built in push mechanism based on google accounts. It might be useful to push some data to the phone, but I'd definitely go for raw TCP sockets. Run a SocketServer on the ground station PC, then open a Socket in the Android app, and keep it open. If it drops, let the phone handle reconnections.
You could have a second cheaper phone onboard streaming video (you could use an out-of-the-box app for that purpose, like USTREAM), or taking HQ snapshots.
Compression could be useful for sending telemetry back to ground station. Also for commands, but since we're not talking about sending real time FPV commands but just a few waypoints, it wouldn't be a problem.

Finally, provided you could find a platform with enough endurance, you should think of using the phone just as 3G modem to handle communications, and connect the phone to a better suited hardware (like APM2) to get the sensors info. Altitude shouldn't be a limitation, you'd be surprised of how high do mobile networks reach, despite being theoretically intended for the ground.

You need to be a member of diydrones to add comments!

Join diydrones

Comments

Karl Busenknecht April 20, 2015 at 12:12pm

Hi John,

Do you have any in depth Documentation about this? I mean HOW-TO to make it work on android?
ali March 22, 2015 at 12:15am

Any update on open-sourcing the code? Thanks!
Marek Suma December 18, 2014 at 12:32pm

I was planning to build something similar. I've recently bought FT311D Development Board that works as Android Accessory (no need for expensive USB OTG enabled phone, no Bluetooth latency). It has 4 PWM channels and costs about 35$.

FT311D Development Board - Google Search
Hernan Gabriel July 13, 2014 at 8:36pm

That is a awesome project.. I would like to contribute, since I consider that the way to go in term of Autopilots would be work on high level programming instead of that low level that APM offers... How can I help?
Lowkey Lyesmithe May 22, 2014 at 10:03pm

Same as the guy above, I'm of a mind to try and make work something very similar to what you've described 2 years ago! Do you still have plans to share the code?
Thatcher Chamberlin December 1, 2013 at 3:35pm

Have you made any progress on this? I'm looking into making my first drone and I'd love to have it running Android because of all the extra computational power it has. You said you might release you code - have you done that yet? Also, how did you like your choice of hardware?
segf4ult July 22, 2012 at 2:00pm

An interesting link about altitude and mobile networks:

http://translate.google.com/translate?sl=es&tl=en&js=n&...

These guys reportedly sent an Android phone up to 17.5 kM (57k ft) and have it sent back the location each 90 seconds.

NearSpace. Descubriendo la frontera del Espacio - Proyecto I+D cobertura móvil
segf4ult July 22, 2012 at 1:52pm
Hi John, very interesting project. I signed up just to comment XD. Using the mobile network is definitely the way to go in order to perform long range missions. Here are my two cents as Android programmer:
- The phone doesn't need to be the "server" in terms of opening connections. Don't be picky here, the code design doesn't need to suffer. You can open a connection to the control station server, then "serve" your telemetry info as well as consume commands. It is a two way communication channel. I'm working in a very similar remote desktop project to control a PC from a phone and vice versa. Since the phone can drop connections due to lack of coverage, it makes sense for the phone to reconnect to ground station.
- As for problem #1, it is extremely rare to see SIMs with public IP, but you don't need it either. I can think of two workarounds: 1) Configure a VPN in your phone and stablish a secure tunnel with ground station; and 2) Let the phone be the "client" in terms of opening connections. VPN is interesting since it provides a security layer and that way your phone will have a local IP you could use.
- Android has a built in push mechanism based on google accounts. It might be useful to push some data to the phone, but I'd definitely go for raw TCP sockets. Run a SocketServer on the ground station PC, then open a Socket in the Android app, and keep it open. If it drops, let the phone handle reconnections.
- You could have a second cheaper phone onboard streaming video (you could use an out-of-the-box app for that purpose, like USTREAM), or taking HQ snapshots.
- Compression could be useful for sending telemetry back to ground station. Also for commands, but since we're not talking about sending real time FPV commands but just a few waypoints, it wouldn't be a problem.
Finally, provided you could find a platform with enough endurance, you should think of using the phone just as 3G modem to handle communications, and connect the phone to a better suited hardware (like APM2) to get the sensors info. Altitude shouldn't be a limitation, you'd be surprised of how high do mobile networks reach, despite being theoretically intended for the ground.
johnm545 July 20, 2012 at 9:23pm

@Stefan: Plane is a Skywalker Sirius from bevrc. Another Easy Star clone, but a little bit bigger.

All the code is in an activity. Lack of real time doesn't seem to matter, it's got plenty of CPU grunt to stay on top of things. Linux kernals have 1ms timer resolution unlike windows (16ms). The Bluetooth adds about 15ms of latency which is less than a typical RC 2.4ghz system, and it's only sending 12 bytes each cycle (600 bytes/sec).
exporulez July 19, 2012 at 3:37am

Nice work :D Have you made a simple Activity running all the code? Some Android phones have more than one core. Have you decomposed the system to use parallelism? I know that linux kernel offer RT call for the scheduler. Maybe there is a way to use it in Android that is based on linux kernel. Try to search this class: javax.realtime.RealtimeThread. If you haven't you can find the implementations here: http://java.sun.com/javase/technologies/realtime/.This could fix the real time problem.

of 3

This reply was deleted.