How to setup a Mission Planner HD-Video HUD using GStreamer and Mission Planner plugins

This blog is a continuation of my previous post.

How to build a High-Definition FPV UAV using a Rasperry PI with HD camera, using a high speed WiFi link

This post will discuss how to use GStreamer and Mission Planner together to display the HD video with a HUD (Head-Up-Display).

Note: I have only tested this feature on Windows so the instructions given here are for Windows only.

To give proper credit, the HUD created here was borrowed from APM Planner, a Qt-Based app similar to Mission Planner. The HUD part was created from the Qt codebase QML HUD created by Bill Bonney who is on the APM Planner development team. To make the HUD work with the background video, I used a GStreamer library called "QtGStreamer" which integrates GStreamer plugins with painting on a Qt widget. This library is available on the GStreamer website.

The end-result is dynamically added to Mission Planner using the plug-in architecture.

In the previous posts I discussed used a Raspberry PI and a High-speed WiFi link using GStreamer on the PI and the ground station PC. To get the HUD to work, you need to already have a successful link with the video on your ground station.

Here are the steps to follow to install the plugin:

1) Install Mission Planner.

2) Download and install GStreamer from this link. Use the x86 version, the x86_64 version will NOT work. (Use the default path 'C:\GStreamer' when installing). When installing GStreamer, select 'Custom' install and select ALL plugins to be installed.

3) Follow the steps in the previous blog noted above to get your video stream working.

4) Download and the MSI installer from this link. and run the installer.

If all went well, you should have the plugin installed.

Open Mission Planner and navigate to the "Flight Data" page and right-click on the map. You should see a menu item called "GStreamer HUD" as shown below:

Select this menu item and the following screen should appear:

In the upper-left corner is a context menu. Here is where you enter your GStreamer Pipeline string. If you had the video displaying without the HUD using a valid pipeline, enter it here.

Note: The GStreamer Pipeline string should be exactly the same as the string you used before, but WITHOUT the final video sink element. The video sink is the QtGStreamer element which will be added automatically by the plugin. The GStreamer pipe should therefore be the same, except remove the last element for the video sink.

Here is an example string I used on my setup:

udpsrc port=9000 buffer-size=60000 ! application/x-rtp,encoding-name=H264,payload=96 ! rtph264depay ! h264parse ! queue ! avdec_h264

If all is well, you can connect to your UAV and see the HUD elements moving. To change the HUD, right click on the display and select which elements you want to display. The default is to display everything shown here.

If anybody has problems, please post back and I'll update the blog in case I missed something, and you cannot get it to work.

Happy Flying!

You need to be a member of diydrones to add comments!

Join diydrones

Comments

FlyingW February 12, 2017 at 8:49am

Can this same video stream be displayed on the HUD that is native to Mission Planner?

There is an option to tell the native HUD to get video from a source, and I've seen folks on the web putting their FPV streams on the native HUD.

I'm not heartbroken if it can't as I am very satisfied with Mr. Duffy's version. Just wondering about the difference.

Thanks,

Paul
Robert Mapes February 9, 2017 at 9:29am

The Brilliance lies within Patrick's innovation. It is a work of art :)
FlyingW February 9, 2017 at 9:19am

Brilliant - got it working. Thanks Robert and Patrick.
Robert Mapes February 9, 2017 at 9:03am

After you enter it you need to press enter to save the string
FlyingW February 9, 2017 at 9:01am

Ok, I got a good baseline using UDP connections between ArduCopter and Mission Planner and for the streaming video.

Changing over to TCP now. I seem to be having a problem with entering the pipeline string into the HUD. I paste it in the form, but it reverts back to a previous entry - in the case shown the test pattern.

How do I make sure that the HUD takes the pipeline string?

Thanks again,

Paul
Robert Mapes February 9, 2017 at 7:53am

i meant to say I use -A and -C
FlyingW February 9, 2017 at 7:51am

Robert,

I was editing my words while you answered and I do not know to which command you are referring. Thanks for describing the syntax again: I don't know whether to use -A and -C, or -A and -A.

Thanks,

Paul
FlyingW February 9, 2017 at 7:50am

Patrick,

I got it working with UDP so it must be something about my setup not letting TCP through.

Do I need two separate telemetry channels - one to feed Mission Planner on port 14550, and another one to feed the HUD on port 14551? When I connect Mission Planner the HUD no longer updates but still receives the video.

I was using this command to send data to both the HUD and mission planner:

...ArduCopter-quad -A udp:192.168.1.103:14550 -C udp:192.168.1.103:14551

Can I use this command to place both the data and the video over the same link?

...ArduCopter-quad -A udp:192.168.1.103:14550 -A udp:192.168.1.103:14551

Thanks again,

Paul
Robert Mapes February 9, 2017 at 7:45am

That is how I do it, this command places both the data and the video over the same link but over different ports. Also, if there is video already coming over in a separate window (the D3D video window), you have to close it first so that the video can appear in the HUD (the two video streams are contending for the same port).
Patrick Duffy February 8, 2017 at 8:53pm

@Paul, the pipeline should be:

tcpclientsrc host=192.168.1.105 port=5000 ! gdpdepay ! rtph264depay ! avdec_h264

No need for the videoconvert or videosink elements. Let me know if that works for you. Also, I would suggest using UDP instead of TCP. It will reduce your latency.

of 39

This reply was deleted.