Using smartphone for the brain of a drone

Hi DIY Drones, we are happy to write here about our drone that use smartphone as the brain, and phone's display for its face! We are trying to build an API set that runs on Android, which send commands in MAVLINK over USB serial to Pixhawk.

We were researchers of optimization problems at Stanford univ. and the Univ. of Tokyo, and we study route scheduling and optimizations for robotics, using simulated annealing of Ising model. As its application, we aim to implement our algorithm onto drones. APM is our base (Thank every contributors for the amazing work!), and we need a more powerful processor and also network connectivity, instead of using a Raspberry pi, we implemented an Android phone.

In our campus test flights, nobody cared about what we study, people just think it looks cool! Then we came to the idea, why not make it a robot. The phone display and speaker are there, which can also provide visual assistance in remote rescue assistance mission, or visual alert in aerial patrol.

After trials and error we made the prototype takes off successfully. Meanwhile we continue to implement our research algorithm, we are also looking into OpenCV. Everything of this project is Open Source, we are also exciting to see how people utilize the advanced functionalities on a phone to make drones intelligent. We will release the hardware CAD and software source on GitHub. We setup a start-up in Palo Alto, CA and to support our research we launched the project on Kickstarter. We will incorporate 3DR Pixhawk as the main flight controller, and Android as the co-processor that provide visual look and programming platform.

Details on our Kickstarter page:

https://www.kickstarter.com/projects/labromance/lab-the-living-aerial-bot

and our website:

http://www.labromance.com/

Cheers!

Kai

You need to be a member of diydrones to add comments!

Join diydrones

Comments

technicus acityone February 25, 2015 at 7:07am

Hmmmm... Where is there a smartphone does not matter.
In fact, he serves as the Mavlink GCS, but if you organize an exchange of information between them then there is the prospect for swarming solutions )
Kai Yan February 25, 2015 at 5:11am

@Jerry Giant, No, voice control is covered my motors, it doesn't work after motor spinning. Image could work but currently it is a development platform so RC controller is needed.
Kai Yan February 25, 2015 at 5:05am

@Joe, it is, we focused our initial milestone to be detecting and avoid oblivious objects using OpenCV, running on Android and by sending Mavlink rc channel override commands to pixhawk for motion. We have several code base from academic research, and we will l gradually include them into our library. It's a long run project rather than a single campaign.
Kai Yan February 25, 2015 at 5:00am

@mP1, one problem is route optimization, given multiple points on a map and to find the shortest path to visit every point at least once (Aka. traveling salesman problem). A good algorithm could save battery for drones.
Jerry Giant February 25, 2015 at 4:49am

sticking a android device to a apm based quadcopter, i don't know what the app or SDK can do. maybe a flying siri could at least humor me.

when props are spinning, could mic array filter off the noise?

maybe it's best use is to replace the stick, and take a photo when i smile, where many places are banning...
Joe Renteria February 25, 2015 at 4:47am

@Kai, I am not a big telemetry and video guy yet, but I know it can be quite a project getting everything up and running, getting decent range out of your equipment and possibly building a tracker. Then there is the extra weight and gadgets hanging everywhere. I think you should put the fact that you have everything you need already built in to good use, and focus on that as well as the development platform...
mP1 February 25, 2015 at 4:23am

What do these optimisation algorithms do ? I keep reading the words but i have no idea how they add features to a basic pixhawk.
Kai Yan February 25, 2015 at 4:18am

@Jerry, no, anyone can easily mount a phone on quads, that's not the point. What we do is to develop an API layer that include basic functions such as image/voice recognition etc.
Jerry Hatrick February 25, 2015 at 4:14am

So basically you stuck an android phone on the front of a regular quad.... and are asking people to pay you for it?
Kai Yan February 25, 2015 at 4:14am

@Joe, this design may not be suitable for video shooting, instead it is supposed as a platform to develop smart drones. The usage we come up now is for visual assistance in remote rescue, or visual alert in patrol mission, etc.

of 3

This reply was deleted.