Use speech recognition to control a flight

In this video I present the voice recognition feature in my GCS/Companion-app-combo FlightZoomer. The feature allows to control the flight of an RC aircraft (or multicopter) using simple voice commands. In order to get an additional safety level against wrongly recognized commands, the engine does read back whatever sentence it understood. By affirming the readback, the command is then activated. Just check out the video to see how this works.

VoiceFlight.jpg?width=700

On this image you can see the flown route. At point 1 I activated the autopilot (which b.t.w. is a copy of the Boeing 787 autopilot), then I commanded some left turns: first to a western course of 270° (point 2), then south 180° (point 3), then 50° (point 4), 270° again (point 5) followed by a right turn to 50° into the downwind (point 6). At point 7 a descend to 470 meters is initiated. Finally there is a turn to 150° into the base (point 8) and at point 9 the ILS approach is activated. The turn into the final approach (point 10) and the subsequent landing (point 11) was then performed fully automatic.

As you can see on the video, the voice engine really performs nicely and understands my commands well. You can also see how it misunderstands me in once case (and how the misunderstanding is resolved) when I pronounce a number wrongly (I am not a native English speaker).


FlightZoomer is a software solution, that runs on off-the-shelf devices and offers all you need for instrument flight and much more (e.g. a cellular network databus between gcs and companion computer, the companion computer is a smartphone, optional usage of 3rd party radio telemetry, embedding of the FPV camera feed in the cockpit app, synthetic FPV view based on telemetry alone).


The solution runs on top of the Ardupilot stack and will be released shortly. More information on https://flightzoomer.com/

E-mail me when people leave their comments –

You need to be a member of diydrones to add comments!

Join diydrones

Comments

  • @all thanks for the feedback!

    @Tommy: I call it my wearable groundstation. It is my own design and I built it from scratch because there was nothing comparable on the market. I introduced it under this link and I believe I have even posted construction drawings somewhere on rcgroups ...

    The Wearable Groundstation
    Have a look at the wearable groundstation, that I have built as cockpit for my Boeing 787-like drones avionics suite FlightZoomer... The wearable…
  • Really impressive. 

  • Where did you buy the shoulder mounted "GCS" ? :)

  • Fantastic job Martin.  I am looking forward to seeing this solution and investigate more. Keep up the good work.

  • Thanks, I did already make a blogpost which is visible directly on http://www.ardupilot.org.

    ArduPilot Open Source Autopilot
    The most advanced open source autopilot for use by both professionals and hobbyist. Supports multi-copters, planes, rovers, boats, helicopters, ante…
  • Nice! Perhaps cross post to discuss.ardupilot.org too?
This reply was deleted.