How to setup a Mission Planner HD-Video HUD using GStreamer and Mission Planner plugins

This blog is a continuation of my previous post.

How to build a High-Definition FPV UAV using a Rasperry PI with HD ...

This post will discuss how to use GStreamer and Mission Planner together to display the HD video with a HUD (Head-Up-Display).

Note: I have only tested this feature on Windows so the instructions given here are for Windows only. 

To give proper credit, the HUD created here was borrowed from APM Planner, a Qt-Based app similar to Mission Planner. The HUD part was created from the Qt codebase QML HUD created by Bill Bonney who is on the APM Planner development team. To make the HUD work with the background video, I used a GStreamer library called "QtGStreamer" which integrates GStreamer plugins with painting on a Qt widget.  This library is available on the GStreamer website.

The end-result is dynamically added to Mission Planner using the plug-in architecture. 

In the previous posts I discussed used a Raspberry PI and a High-speed WiFi link using GStreamer on the PI and the ground station PC.  To get the HUD to work, you need to already have a successful link with the video on your ground station. 

Here are the steps to follow to install the plugin:

1) Install Mission Planner.

2) Download and install GStreamer from this link.  Use the x86 version, the x86_64 version will NOT work. (Use the default path 'C:\GStreamer' when installing). When installing GStreamer, select 'Custom' install and select ALL plugins to be installed.

3) Follow the steps in the previous blog noted above to get your video stream working.

4) Download and the MSI installer from this link. and run the installer.

If all went well, you should have the plugin installed.

Open Mission Planner and navigate to the "Flight Data" page and right-click on the map. You should see a menu item called "GStreamer HUD" as shown below:

Select this menu item and the following screen should appear:

In the upper-left corner is a context menu. Here is where you enter your GStreamer Pipeline string. If you had the video displaying without the HUD using a valid pipeline, enter it here.

Note: The GStreamer Pipeline string should be exactly the same as the string you used before, but WITHOUT the final video sink element. The video sink is the QtGStreamer element which will be added automatically by the plugin. The GStreamer pipe should therefore be the same, except remove the last element for the video sink.

Here is an example string I used on my setup:

udpsrc port=9000  buffer-size=60000 ! application/x-rtp,encoding-name=H264,payload=96 ! rtph264depay ! h264parse ! queue ! avdec_h264

If all is well, you can connect to your UAV and see the HUD elements moving.  To change the HUD, right click on the display and select which elements you want to display. The default is to display everything shown here. 

If anybody has problems, please post back and I'll update the blog in case I missed something, and you cannot get it to work.

Happy Flying!

Views: 47470

Comment by Bernt Christian Egeland on November 6, 2014 at 9:58am

@Patrick D.  Example ;

bitrate=900000  /  h264    / framerate=5/1  /  360 x 288.

Latency approx. 2-300ms for 3g video stream.

Comment by Steve on November 6, 2014 at 10:16am

@Patrick

I checked MP and it is installed as stated above and in the plugins directory I see "QtGstreamerHUD.dll" but I do not get option for Gstreamer.

This is the gstreamer that I downloaded with all plugins: http://gstreamer.freedesktop.org/data/pkg/windows/1.4.3/

I downloaded: gstreamer-1.0-devel-x86-1.4.3

Comment by Patrick Duffy on November 6, 2014 at 10:38am

@Steve, If that's the case, turn on your console log and see if you get any error messages. Also check your 'PATH' environment variable and see if there is an entry for "C:\Program Files (x86)\QtGStreamer\bin", and check if the files in that path got installed by the installer.  The installer also adds a variable named "GST_PLUGIN_PATH", which should be set to "C:\gstreamer\1.0\x86\lib\gstreamer-1.0;C:\Program Files (x86)\QtGStreamer\plugins"   

Let me know what the console log says.  It's possible I left out a file on some other dependency.  Which version of windows are you using? I have tested this on Windows 7 and Windows 8.1 and it seemed to work fine.

Comment by Patrick Duffy on November 6, 2014 at 3:21pm

@Steve, one other thing I noticed is that you used the development version of gstreamer. I am not sure the paths are the same. I would try the non-'devel' version instead unless you plan to compile/link with the gstreamer libraries.

Comment by Steve on November 6, 2014 at 4:04pm

I have Windows 7 Professional 64-bit. I checked path variables and they are correct. I re-installed gstreamer without dev and that did not change anything either...Qt path is correct as well.

The console in MP also does not run into any errors

Comment by Patrick Duffy on November 6, 2014 at 4:38pm

@Steve,  What version of Mission Planner are you using? Mine is 1.3.10 build 1.1.5369.11976.    When you right-click in the 'Flight Data" area (not Flight Plan), you don't see the option correct?

It's possible I left out a dependency. The way to check this is to use the tool 'depends.exe' which you can download here:

http://www.dependencywalker.com/

Download the "x86" version, then run 'depends.exe' and open the file "C:\Program Files (x86)\Mission Planner\plugins\MissionPlanner.HUD.CLI.dll".

The tool should give a report on missing dependencies.  You can ignore any missing ones which start with "API-MS",  "EXT-MS", or "IESHIMS.dll", as these are normal. They are not really missing.  If you see anything like a Qt dll missing, or anything else, then send me the list.

Comment by Tommy Larsen on November 7, 2014 at 4:06am

@Bernt, Good to see you back in business :)

Comment by Patrick Duffy on November 7, 2014 at 6:22am
@Steve, did you make any progress? Just wondering if anybody else can get it to work.
Comment by Dan Murray on November 7, 2014 at 7:04am

Working here Patrick - awesome!

FWIW, I installed it on a new computer and it wasn't working at first - could get the HUD up and running but never any data. Had to run gst-launch separately and get windows to prompt to allow it to "monitor networks" or something similar. Just a heads up in case anyone else runs into it.

Can't wait to test it in the air! Thanks again.

Comment by Patrick Duffy on November 7, 2014 at 7:30am

@Dan,  glad to hear you got it to work.  I think that message you got was for the firewall. Please let me know if you have any issues, or suggestions. In the next release, I will be adding more stuff to optionally display from the MavLink.  The current list was what I considered 'critical (battery status, flight mode), so if you have specific items you want first, let me know. A few items, like GPS status, arm state, and critical popup messages will be in the next version.

Comment

You need to be a member of DIY Drones to add comments!

Join DIY Drones

© 2019   Created by Chris Anderson.   Powered by

Badges  |  Report an Issue  |  Terms of Service