This blog is a continuation of my previous post.
How to build a High-Definition FPV UAV using a Rasperry PI with HD camera, using a high speed WiFi link
This post will discuss how to use GStreamer and Mission Planner together to display the HD video with a HUD (Head-Up-Display).
Note: I have only tested this feature on Windows so the instructions given here are for Windows only.
To give proper credit, the HUD created here was borrowed from APM Planner, a Qt-Based app similar to Mission Planner. The HUD part was created from the Qt codebase QML HUD created by Bill Bonney who is on the APM Planner development team. To make the HUD work with the background video, I used a GStreamer library called "QtGStreamer" which integrates GStreamer plugins with painting on a Qt widget. This library is available on the GStreamer website.
The end-result is dynamically added to Mission Planner using the plug-in architecture.
In the previous posts I discussed used a Raspberry PI and a High-speed WiFi link using GStreamer on the PI and the ground station PC. To get the HUD to work, you need to already have a successful link with the video on your ground station.
Here are the steps to follow to install the plugin:
1) Install Mission Planner.
2) Download and install GStreamer from this link. Use the x86 version, the x86_64 version will NOT work. (Use the default path 'C:\GStreamer' when installing). When installing GStreamer, select 'Custom' install and select ALL plugins to be installed.
3) Follow the steps in the previous blog noted above to get your video stream working.
4) Download and the MSI installer from this link. and run the installer.
If all went well, you should have the plugin installed.
Open Mission Planner and navigate to the "Flight Data" page and right-click on the map. You should see a menu item called "GStreamer HUD" as shown below:
Select this menu item and the following screen should appear:
In the upper-left corner is a context menu. Here is where you enter your GStreamer Pipeline string. If you had the video displaying without the HUD using a valid pipeline, enter it here.
Note: The GStreamer Pipeline string should be exactly the same as the string you used before, but WITHOUT the final video sink element. The video sink is the QtGStreamer element which will be added automatically by the plugin. The GStreamer pipe should therefore be the same, except remove the last element for the video sink.
Here is an example string I used on my setup:
udpsrc port=9000 buffer-size=60000 ! application/x-rtp,encoding-name=H264,payload=96 ! rtph264depay ! h264parse ! queue ! avdec_h264
If all is well, you can connect to your UAV and see the HUD elements moving. To change the HUD, right click on the display and select which elements you want to display. The default is to display everything shown here.
If anybody has problems, please post back and I'll update the blog in case I missed something, and you cannot get it to work.
Happy Flying!
Comments
@Alp, here's a sample gstreamer pipeline that splits to record:
gst-launch-1.0 udpsrc port=9000 buffer-size=60000 ! application/x-rtp,encoding-name=H264,payload=96 ! rtph264depay ! h264parse ! tee name=record ! queue ! avdec_h264 ! video/x-raw, format=I420 ! d3dvideosink sync=false record. ! queue ! mp4mux ! filesink location = /temp/fpv.mp4
If you are using the Qt-based HUD, use this one:
udpsrc port=9000 buffer-size=60000 ! application/x-rtp,encoding-name=H264,payload=96 ! rtph264depay ! h264parse ! tee name=record ! queue ! avdec_h264 record. ! queue ! mp4mux ! filesink location = /temp/fpv.mp4
@Alp, if you want to add multiple video streams, you can start multiple processes of gstreamer, The Rpi only supports one camera, but you could use multiple webcams or a 3d cam that has multiple streams. The PI2 may have enough performance. I have not heard of anybody doing this yet, so you would be blazing a new trail.
@Aytek
Actually I just want to use my phone's cool AMOLED screen for FPV via google cardboard clone :)
ALP,
Non directional 3D flight most likely cause vertigo... I have tired and due to lack of frame rate and motion blur, it is cause sickness...
By the way, GStreamer support 3D split video mode or 2D /3D conversion?
I mean, can I use google cardboard with Rasspberry's video streams?
Hmmm.. Ok, thanks. I will try it with Core i5. How can I use "tee" element? Can you say me an examp. ?
raspivid -ih -md 5 -t 999999 -vf -hf -w 1296 -h 730 -fps 30 -b 2500000 -n -g 60 -pf high -o - | gst-launch-1.0 -v fdsrc ! h264parse ! rtph264pay pt=96 config-interval=1 ! udpsink host=xxx.xxx.xxx.xxx port=5000
@Alp, in your GStreamer pipeline, use a 'tee' element to split the stream to a file. Make sure you have a PC with enough horsepower to keep up writing the stream and display at the same time. I tried this on a cheap laptop and the display kept freezing.
@Patrick,
How can I send the video stream data to file in the same time?
@Erich, it appears that there is an issue with the installation. Which version of the HUD are you using? Mission Planner version or stand-alone? The stand-alone version will install gstreamer in a separate folder and it could conflict with a previous version, especially if you have the 64 bit version. The HUD will only work with the 32 bit version. The Mission-planner version assumes you have the 32 bit gstreamer installed.
You might try uninstalling, and re-install, and then check you path variable that it is set to include the location where the 32 bit gstreamer is installed.
@Patrick,
no matter what string is entered into the "Pipeline" field the same error is being showed. I tried following:
1. videotestsrc
2.udpsrc port=5000 buffer-size=60000 ! application/x-rtp,encoding-name=H264,payload=96 ! rtph264depay ! h264parse ! queue ! avdec_h264
3. udpsrc address=192.168.42.11 port=5000 caps="application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96" ! rtph264depay ! avdec_h264
port 5000 is also configured at RPi
All I get is the same error all the time. As you can see on the picture (link), gstreamer works okay when running from command line (cannot show you the real stream because the gear is at home currently).
Error image no.1
-
12
-
13
-
14
-
15
-
16
of 39 Next