How to setup a Mission Planner HD-Video HUD using GStreamer and Mission Planner plugins

This blog is a continuation of my previous post.

How to build a High-Definition FPV UAV using a Rasperry PI with HD ...

This post will discuss how to use GStreamer and Mission Planner together to display the HD video with a HUD (Head-Up-Display).

Note: I have only tested this feature on Windows so the instructions given here are for Windows only. 

To give proper credit, the HUD created here was borrowed from APM Planner, a Qt-Based app similar to Mission Planner. The HUD part was created from the Qt codebase QML HUD created by Bill Bonney who is on the APM Planner development team. To make the HUD work with the background video, I used a GStreamer library called "QtGStreamer" which integrates GStreamer plugins with painting on a Qt widget.  This library is available on the GStreamer website.

The end-result is dynamically added to Mission Planner using the plug-in architecture. 

In the previous posts I discussed used a Raspberry PI and a High-speed WiFi link using GStreamer on the PI and the ground station PC.  To get the HUD to work, you need to already have a successful link with the video on your ground station. 

Here are the steps to follow to install the plugin:

1) Install Mission Planner.

2) Download and install GStreamer from this link.  Use the x86 version, the x86_64 version will NOT work. (Use the default path 'C:\GStreamer' when installing). When installing GStreamer, select 'Custom' install and select ALL plugins to be installed.

3) Follow the steps in the previous blog noted above to get your video stream working.

4) Download and the MSI installer from this link. and run the installer.

If all went well, you should have the plugin installed.

Open Mission Planner and navigate to the "Flight Data" page and right-click on the map. You should see a menu item called "GStreamer HUD" as shown below:

Select this menu item and the following screen should appear:

In the upper-left corner is a context menu. Here is where you enter your GStreamer Pipeline string. If you had the video displaying without the HUD using a valid pipeline, enter it here.

Note: The GStreamer Pipeline string should be exactly the same as the string you used before, but WITHOUT the final video sink element. The video sink is the QtGStreamer element which will be added automatically by the plugin. The GStreamer pipe should therefore be the same, except remove the last element for the video sink.

Here is an example string I used on my setup:

udpsrc port=9000  buffer-size=60000 ! application/x-rtp,encoding-name=H264,payload=96 ! rtph264depay ! h264parse ! queue ! avdec_h264

If all is well, you can connect to your UAV and see the HUD elements moving.  To change the HUD, right click on the display and select which elements you want to display. The default is to display everything shown here. 

If anybody has problems, please post back and I'll update the blog in case I missed something, and you cannot get it to work.

Happy Flying!

Views: 47972

Comment by Steve on November 7, 2014 at 7:48am

@Patrick I have same version of MP as you. I am clicking on flight data and no right click option for it. I'll try different computer when I have time

Comment by Dan Murray on November 7, 2014 at 7:49am
@Patrick - awesome! How hard is it to customize the HUD? I always thought something like the G1000 would be awesome:

https://www8.garmin.com/HiRes/aviation/g1000/g1000-svt-runway-ident...
Comment by Patrick Duffy on November 7, 2014 at 8:32am

@Dan,  It's not hard to add new items. Graphical items can take longer, but text elements are easy. The table shown in your garmin example shouldn't be too difficult.  It would be nice to have a specification to work from so I can understand which MavLink items correspond to an element on the HUD.   If you have a personal list feel free to post them. I like to use the 'agile' method and only add things as they get requested by users. That way I avoid doing work on things nobody is going to use.

Comment by Andrew Girgen on November 7, 2014 at 11:28am

@Patrick, Thanks for posting this.  I have been looking for a tutorial like this for a while.  Like a few others, I'm not getting the GStreamer HUD option when I rt click on flight data map.  I downloaded the dependancy viewer and came up with the following as missing:  MSVCP110.DLL, MSVCR110.DLL, GPSVC.DLL, IESHIMS.DLL

I didn't get any errors in console when loading mission planner, I am running the same version of mission planner as you and I made sure I didn't download the 64 bit version of gstreamer (gstreamer-1.0-x86-1.4.3.msi).  Thanks for any info you can provide!

Comment by Patrick Duffy on November 7, 2014 at 12:04pm

Hello Andrew.  That's a clue. Thanks for checking this.  Those are the Visual C++ 2012 dlls, and apparently on some systems, they are already there. I'll re-build and send another release which includes them.

Comment by Andrew Girgen on November 7, 2014 at 12:07pm

great, thanks for looking into this!

Comment by Patrick Duffy on November 7, 2014 at 1:15pm

@Andrew / Steve.   I have uploaded a new version to dropbox with the missing DLLs. Please try again. You may have to uninstall the old one first. Let me know if you still have issues.

Comment by Andrew Girgen on November 7, 2014 at 1:23pm

It works!

Comment by Patrick Duffy on November 7, 2014 at 1:53pm

@Andrew,  glad to hear it works.  Please let me know if you have any other issues / suggestions for improvement.

Comment by Patrick Duffy on November 7, 2014 at 2:02pm

FYI, if you want to record the video, you have to 'tee' the pipeline. Here is an example pipeline string:

udpsrc port=9000  buffer-size=60000 ! application/x-rtp,encoding-name=H264,payload=96 ! rtph264depay ! h264parse ! tee name=record ! queue ! avdec_h264 !  record. ! queue ! mp4mux ! filesink location = /temp/fpv.mp4

the "avdec_h264" element should be the last element followed by the record tee pipeline elements. It's the same as the non-record pipeline without the video sink element, as the sink is added by the HUD automatically.

Important!  To get the recording to actually save, you have to hit the "stop" button on popup menu located in the upper-left corner of the HUD. Otherwise the EOS signal is never sent to the pipeline and the MP4 header is not written. You will have raw data, but the header is missing so you won't be able to load the file into a viewer.  Once you hit 'start' again, the file you have specified will get clobbered, so rename it if you want to save it.

Comment

You need to be a member of DIY Drones to add comments!

Join DIY Drones

© 2019   Created by Chris Anderson.   Powered by

Badges  |  Report an Issue  |  Terms of Service