How to setup a Mission Planner HD-Video HUD using GStreamer and Mission Planner plugins

This blog is a continuation of my previous post.

How to build a High-Definition FPV UAV using a Rasperry PI with HD ...

This post will discuss how to use GStreamer and Mission Planner together to display the HD video with a HUD (Head-Up-Display).

Note: I have only tested this feature on Windows so the instructions given here are for Windows only. 

To give proper credit, the HUD created here was borrowed from APM Planner, a Qt-Based app similar to Mission Planner. The HUD part was created from the Qt codebase QML HUD created by Bill Bonney who is on the APM Planner development team. To make the HUD work with the background video, I used a GStreamer library called "QtGStreamer" which integrates GStreamer plugins with painting on a Qt widget.  This library is available on the GStreamer website.

The end-result is dynamically added to Mission Planner using the plug-in architecture. 

In the previous posts I discussed used a Raspberry PI and a High-speed WiFi link using GStreamer on the PI and the ground station PC.  To get the HUD to work, you need to already have a successful link with the video on your ground station. 

Here are the steps to follow to install the plugin:

1) Install Mission Planner.

2) Download and install GStreamer from this link.  Use the x86 version, the x86_64 version will NOT work. (Use the default path 'C:\GStreamer' when installing). When installing GStreamer, select 'Custom' install and select ALL plugins to be installed.

3) Follow the steps in the previous blog noted above to get your video stream working.

4) Download and the MSI installer from this link. and run the installer.

If all went well, you should have the plugin installed.

Open Mission Planner and navigate to the "Flight Data" page and right-click on the map. You should see a menu item called "GStreamer HUD" as shown below:

Select this menu item and the following screen should appear:

In the upper-left corner is a context menu. Here is where you enter your GStreamer Pipeline string. If you had the video displaying without the HUD using a valid pipeline, enter it here.

Note: The GStreamer Pipeline string should be exactly the same as the string you used before, but WITHOUT the final video sink element. The video sink is the QtGStreamer element which will be added automatically by the plugin. The GStreamer pipe should therefore be the same, except remove the last element for the video sink.

Here is an example string I used on my setup:

udpsrc port=9000  buffer-size=60000 ! application/x-rtp,encoding-name=H264,payload=96 ! rtph264depay ! h264parse ! queue ! avdec_h264

If all is well, you can connect to your UAV and see the HUD elements moving.  To change the HUD, right click on the display and select which elements you want to display. The default is to display everything shown here. 

If anybody has problems, please post back and I'll update the blog in case I missed something, and you cannot get it to work.

Happy Flying!

Views: 48030

Comment by Andrew Girgen on November 7, 2014 at 4:54pm

So does anyone have a flight video with this setup??

Comment by Patrick Duffy on November 9, 2014 at 5:04pm
@Andrew, I setup today to record a flight, but just before launch noticed a servo problem and had to abort. Possibly tomorrow I'll have some flight video.

Also, when you record using a tee in the pipeline as in the example I posted before, you won't see the HUD in the video. I think this is the preferred record mode, and currently there is no method to record both the video and the HUD overlay. If someone wants that, I will have to add a new feature.
Comment by ap105 on November 17, 2014 at 4:41am

Patrick, awesome work!  The color effects with different hue/saturation levels are great, but I would like to get as close as possible to the original colors of the video which does not seem to work right now. Is there a working slider combination to do that?

Comment by Patrick Duffy on November 17, 2014 at 7:27am

@ap105,  this is an issue with QtGstreamer when rendering the video.  There are some other options in the gstreamer plugin that may fix that issue but I have not had the time to play with the settings to figure it out. I may have to contact the original developer to understand the issue, but I think it has something to do with the auto-color balance which appears not to work correctly. When I render using the native video sink, it's fine so it's something in the gstreamer plugin that is changing the color balance. 

Comment by ap105 on November 19, 2014 at 9:57am

@Patrick: ok. Are you still planning to post the source up to github? I am using this with the C920 and would love to put some camera control into the plugin instead of writing a separate app/plugin for that.

Comment by Patrick Duffy on November 19, 2014 at 10:17am

@ap105,  I'll post the source as soon as I get some time, possible later today. The current configuration is part of Mission Planner because of the plugin architecture so you will also need to get Mission Planner, and the dependencies for the plugin. This needs a bit of documentation, and also a 'cmake' file that checks for the needed dependencies which does not exist yet.  To build the plugin, you'll need MissionPlanner source, GStreamer (x86 dev) , QtGStreamer (from freedesktop.org), and Qt installed on your system.  The plugin is actually Qt-based, but the plugin DLL is .net (C#) and a proxy dll  written C++/CLI that actually connects MissionPlanner to the Qt HUD module.   My plan is to make the HUD a stand-alone app that can parse the Mavlink stream directly rather than use Mission Planner, but this is a bigger project.

Comment by Tommy Larsen on November 19, 2014 at 10:32am

@Patric: It shouldn't be a big thing to make it a stand-alone. In MP just press ctrl-f and then click "mavlink". Then you can forward all mavlink data to your hud via TCP. Or am i missing something here?

Comment by Patrick Duffy on November 19, 2014 at 12:06pm

@Tommy,  Getting the mavlink data is easy, but currently I don't have the parser or the framework in the plugin code. It will take some work to duplicate what is in MissionPlanner without pulling the entire app with it.  The plugin currently uses a class in MissionPlanner that already holds the current state of the UAV. I don't have this in the plugin yet so this will require some development. If someone else wants to contribute time to get this done, by all means, any help would be appreciated. 

Comment by Dan Murray on November 19, 2014 at 12:24pm

@All, can we get a group together of interested parties? I think if we all pitch in, it should be pretty darn easy to get a QML-based, pymavlink/proxy-backed standalone HUD together. I am just finishing up my G1000 HUD QML, would be happy to contribute that and some GStreamer/networking knowledge, but will need some assistance with the glue between pymavlink and Qt.

Comment by Patrick Duffy on November 20, 2014 at 10:31pm

FYI,  I have posted the source on github.  https://github.com/pnduffy/GStreamerHUD

This repository is a fork of Mission Planner, but with 3 projects added to support the HUD:  QtGStreamerHUD,  MissionPlanner.HUD, and MissionPlanner.HUD.CLI.   I did not include any of the dependencies, Qt, GStreamer, or QtGStreamer. 

The goal here is to create a new repository from this one, which will be the 'stand-alone' app, but I have posted the 'plug-in' version for anybody interested in looking at the code.

Comment

You need to be a member of DIY Drones to add comments!

Join DIY Drones

© 2019   Created by Chris Anderson.   Powered by

Badges  |  Report an Issue  |  Terms of Service