Use speech recognition to control a flight

In this video I present the voice recognition feature in my GCS/Companion-app-combo FlightZoomer. The feature allows to control the flight of an RC aircraft (or multicopter) using simple voice commands. In order to get an additional safety level against wrongly recognized commands, the engine does read back whatever sentence it understood. By affirming the readback, the command is then activated. Just check out the video to see how this works.

On this image you can see the flown route. At point 1 I activated the autopilot (which b.t.w. is a copy of the Boeing 787 autopilot), then I commanded some left turns: first to a western course of 270° (point 2), then south 180° (point 3), then 50° (point 4), 270° again (point 5) followed by a right turn to 50° into the downwind (point 6). At point 7 a descend to 470 meters is initiated. Finally there is a turn to 150° into the base (point 8) and at point 9 the ILS approach is activated. The turn into the final approach (point 10) and the subsequent landing (point 11) was then performed fully automatic.

As you can see on the video, the voice engine really performs nicely and understands my commands well. You can also see how it misunderstands me in once case (and how the misunderstanding is resolved) when I pronounce a number wrongly (I am not a native English speaker).


FlightZoomer is a software solution, that runs on off-the-shelf devices and offers all you need for instrument flight and much more (e.g. a cellular network databus between gcs and companion computer, the companion computer is a smartphone, optional usage of 3rd party radio telemetry, embedding of the FPV camera feed in the cockpit app, synthetic FPV view based on telemetry alone).


The solution runs on top of the Ardupilot stack and will be released shortly. More information on https://flightzoomer.com/

Views: 428

Comment by Auturgy on June 15, 2018 at 10:09pm
Nice! Perhaps cross post to discuss.ardupilot.org too?
Comment by Martin Rüedi on June 15, 2018 at 10:27pm

Thanks, I did already make a blogpost which is visible directly on http://www.ardupilot.org.

Comment by iSkyMaster on June 16, 2018 at 10:24pm

Fantastic job Martin.  I am looking forward to seeing this solution and investigate more. Keep up the good work.

Comment by Tommy Larsen on June 18, 2018 at 4:27am

Where did you buy the shoulder mounted "GCS" ? :)

Comment by Richard Joy on June 18, 2018 at 6:53am

Really impressive. 

Comment by Martin Rüedi on June 18, 2018 at 2:01pm

@all thanks for the feedback!

@Tommy: I call it my wearable groundstation. It is my own design and I built it from scratch because there was nothing comparable on the market. I introduced it under this link and I believe I have even posted construction drawings somewhere on rcgroups ...

Comment

You need to be a member of DIY Drones to add comments!

Join DIY Drones

Groups

Season Two of the Trust Time Trial (T3) Contest 
A list of all T3 contests is here. The current round, the Vertical Horizontal one, is here

© 2018   Created by Chris Anderson.   Powered by

Badges  |  Report an Issue  |  Terms of Service