3689624585?profile=original

This blog is a continuation of my previous post.

How to build a High-Definition FPV UAV using a Rasperry PI with HD camera, using a high speed WiFi link

This post will discuss how to use GStreamer and Mission Planner together to display the HD video with a HUD (Head-Up-Display).

Note: I have only tested this feature on Windows so the instructions given here are for Windows only. 

To give proper credit, the HUD created here was borrowed from APM Planner, a Qt-Based app similar to Mission Planner. The HUD part was created from the Qt codebase QML HUD created by Bill Bonney who is on the APM Planner development team. To make the HUD work with the background video, I used a GStreamer library called "QtGStreamer" which integrates GStreamer plugins with painting on a Qt widget.  This library is available on the GStreamer website.

The end-result is dynamically added to Mission Planner using the plug-in architecture. 

In the previous posts I discussed used a Raspberry PI and a High-speed WiFi link using GStreamer on the PI and the ground station PC.  To get the HUD to work, you need to already have a successful link with the video on your ground station. 

Here are the steps to follow to install the plugin:

1) Install Mission Planner.

2) Download and install GStreamer from this link.  Use the x86 version, the x86_64 version will NOT work. (Use the default path 'C:\GStreamer' when installing). When installing GStreamer, select 'Custom' install and select ALL plugins to be installed.

3) Follow the steps in the previous blog noted above to get your video stream working.

4) Download and the MSI installer from this link. and run the installer.

If all went well, you should have the plugin installed.

Open Mission Planner and navigate to the "Flight Data" page and right-click on the map. You should see a menu item called "GStreamer HUD" as shown below:

3689624376?profile=original

Select this menu item and the following screen should appear:

3689624624?profile=original

In the upper-left corner is a context menu. Here is where you enter your GStreamer Pipeline string. If you had the video displaying without the HUD using a valid pipeline, enter it here.

Note: The GStreamer Pipeline string should be exactly the same as the string you used before, but WITHOUT the final video sink element. The video sink is the QtGStreamer element which will be added automatically by the plugin. The GStreamer pipe should therefore be the same, except remove the last element for the video sink.

Here is an example string I used on my setup:

udpsrc port=9000  buffer-size=60000 ! application/x-rtp,encoding-name=H264,payload=96 ! rtph264depay ! h264parse ! queue ! avdec_h264

If all is well, you can connect to your UAV and see the HUD elements moving.  To change the HUD, right click on the display and select which elements you want to display. The default is to display everything shown here. 

If anybody has problems, please post back and I'll update the blog in case I missed something, and you cannot get it to work.

Happy Flying!

E-mail me when people leave their comments –

You need to be a member of diydrones to add comments!

Join diydrones

Comments

  • @Rogue, 

    Try this string: 

    rtspsrc location=rtspu://admin:admin@192.168.100.103:554/cam1/h264 latency=50 ! decodebin3

    BTW,  I have posted a new version of the app in the play store, and also on the dropbox that has many enhancements and bug fixes.  If you want to use your phone with VR case, this is the version you should use.

    https://play.google.com/store/apps/details?id=org.qtproject.qtgstre...

    https://www.dropbox.com/s/qaijfei0zthje5l/QtApp-release-signed.apk?...

    VR Drone Mavlink Virtual Reality HUD - Apps on Google Play
    Monitor your Drone video and overlay a HUD using this App on your phone with Google Cardboard or other VR device that can hold your phone. Display vi…
  • Hello Patrick I test your android App "QTGstreamer HUD" And I can't view the video that work on my windows laptop following gstreamer pipeline below:

    gst-launch-1.0.exe rtspsrc location=rtspu://admin:admin@192.168.100.103:554/cam1/h264 latency=50 ! decodebin3 ! autovideosink

    Could you please help me?

    Thanks you!

  • @Roque,  yes you install MavProxy on the drone PI and configure it to connect to the serial port connected to your Pixhawk/APM.  MavProxy should be configured to forward the data to a UDP port on your GCS. 

  • I use ezwifibroadcast, TX side pi3 and RX side pi3. I connected on TX side telem to rxtx of pi3.

    So you tell me to install mavproxy on RX side Pi3?

  • @Roque,  Please explain your configuration so I can comment on how to setup telemetry. I assume you are using a Raspberry PI with a PixHawk/APM connected to the serial port?  To get telemetry, you will need to install MavProxy on the PI and have it connect to the serial port on the PI and forward UDP to your GCS.  

  • @Patrick, Very good Job! I use ezwifibroadcast 1.4 and arducopter, but there's no explained how to use bidirectional telemetry and RX side UDP server for telem as you show on you video, could you show us how to?

    On the wiki ezwifi there's no explanation to do this... it'd be cool that you watch us the how to ;)

  • Eureka!  Thanks Patrick - I just inserted your addition to the section in the rtph264pay command - and it now comes up independent of startup order.

    Thank you so much; I've been at this on and off for a few weeks and beginning to lose sleep on it.  Good to have made progress and I appreciate your patient help.

    Be well,

    Paul

  • Patrick,

    Do you mean adding another 'config-interval=1' within the area of the rtph264pay command enclosed by thje exclamation marks?

    Can you show me the exact syntax?

    Thanks,

    Paul

  • @Paul,  it should not be a problem starting from rc.local.  You need to make sure you fork the process by adding an ampersand character (&) after the command so the process forks otherwise it dies at the end of the script.

    You should not have to worry about sequences when starting the HUD.  Add 'config-interval=1' after the rtph264pay element on the PI and it should solve your problem.

  • I found that I can start the stream by manually running the raspivid command from the Pi CLI.  It will not work when I set up the raspivid command to start automatically on boot-up with a script that runs from rc.local.

    Is there a better or at least a different way that works to automatically start the streaming at boot up? 

This reply was deleted.