This blog is a continuation of my previous post.
How to build a High-Definition FPV UAV using a Rasperry PI with HD camera, using a high speed WiFi link
This post will discuss how to use GStreamer and Mission Planner together to display the HD video with a HUD (Head-Up-Display).
Note: I have only tested this feature on Windows so the instructions given here are for Windows only.
To give proper credit, the HUD created here was borrowed from APM Planner, a Qt-Based app similar to Mission Planner. The HUD part was created from the Qt codebase QML HUD created by Bill Bonney who is on the APM Planner development team. To make the HUD work with the background video, I used a GStreamer library called "QtGStreamer" which integrates GStreamer plugins with painting on a Qt widget. This library is available on the GStreamer website.
The end-result is dynamically added to Mission Planner using the plug-in architecture.
In the previous posts I discussed used a Raspberry PI and a High-speed WiFi link using GStreamer on the PI and the ground station PC. To get the HUD to work, you need to already have a successful link with the video on your ground station.
Here are the steps to follow to install the plugin:
1) Install Mission Planner.
2) Download and install GStreamer from this link. Use the x86 version, the x86_64 version will NOT work. (Use the default path 'C:\GStreamer' when installing). When installing GStreamer, select 'Custom' install and select ALL plugins to be installed.
3) Follow the steps in the previous blog noted above to get your video stream working.
4) Download and the MSI installer from this link. and run the installer.
If all went well, you should have the plugin installed.
Open Mission Planner and navigate to the "Flight Data" page and right-click on the map. You should see a menu item called "GStreamer HUD" as shown below:
Select this menu item and the following screen should appear:
In the upper-left corner is a context menu. Here is where you enter your GStreamer Pipeline string. If you had the video displaying without the HUD using a valid pipeline, enter it here.
Note: The GStreamer Pipeline string should be exactly the same as the string you used before, but WITHOUT the final video sink element. The video sink is the QtGStreamer element which will be added automatically by the plugin. The GStreamer pipe should therefore be the same, except remove the last element for the video sink.
Here is an example string I used on my setup:
udpsrc port=9000 buffer-size=60000 ! application/x-rtp,encoding-name=H264,payload=96 ! rtph264depay ! h264parse ! queue ! avdec_h264
If all is well, you can connect to your UAV and see the HUD elements moving. To change the HUD, right click on the display and select which elements you want to display. The default is to display everything shown here.
If anybody has problems, please post back and I'll update the blog in case I missed something, and you cannot get it to work.
Happy Flying!
Comments
Hello Baz,
What's the pipeline you are using on the PI and Windows machine?
This is the string I use on Windows from the command prompt:
C:\gstreamer\1.0\x86\bin\gst-launch-1.0 udpsrc port=9000 buffer-size=60000 ! application/x-rtp,encoding-name=H264,payload=96 ! rtph264depay ! h264parse ! queue ! avdec_h264 ! video/x-raw, format=I420 ! d3dvideosink sync=false
In the HUD it looks like this:
udpsrc port=9000 buffer-size=60000 ! application/x-rtp,encoding-name=H264,payload=96 ! rtph264depay ! h264parse ! queue ! avdec_h264
Let me know your setup and I'll see if I can reproduce your problem.
@Patrick, Great setup and and walk through. I seem to be having the same problem as Ratfeet. It seems the pipeline is pushing from the Pi through to the windows machine, but I'm getting a freeze frame in the mission planner HUD rather than streaming video. Whenever I stop and restart the service on the Pi it takes a new image and refreshes it into the HUD, but won't stream it as a video as such. It's done this today for me on two different windows 7 machines, I'm wondering if it could be something on the Pi? Also yesterday I was able to get Gstreamer working in a popup window on the windows machine from command prompt, but today it sits stuck on GstSystemClock. Could it possibly be something to do with the clocks on the machines? Would they need to be in sync as such or it shouldn't matter?
@Patrick, If you are referring another Windows machine: No. I have updated my VGA drivers though. I will borrow another Windows machine, install gstreamer and test.
@RatFeet, I suspect you have a driver issue on windows. Have you tried on different hardware?
@Patrick, I got it to work between the Pi and my main workstation, MacOS.
Raspberry Pi:
gst-launch-1.0 -v fdsrc ! h264parse ! rtph264pay config-interval=1 pt=96 \
! gdppay ! tcpserversink host=serverIp port=5000
Mac:
! gdpdepay ! rtph264depay ! avdec_h264 ! videoconvert ! autovideosink sync=
@Patrick, If I start the Pi before the Windows (GCS), I get no freeze. I get nothing. I only get the freeze if I start Windows, GCS, before the Pi. The message I sent you was when I enabled debugging, other than that all is working correctly.
I have just updated Nvidia drivers for my Ienovo T410 laptop and am getting same result !!!! :(
Again, when I run the script on Pi, the Pi Camera starts immediately without any problem and would continue have the video feed till I interrupt it. I am hopeless as I tried everything I could.
@RatFeet, is that the last message before it freezes? One possible cause is that GLib is loading from a different location than where GStreamer is installed. If you have cygwin or minGW installed, it's possible that a different version the GLib dll is getting loaded. You may have to modify your 'path' environment variable. This is just a guess.
@ Patrick, does this make any sense?
(gst-launch-1.0:2649): GLib-CRITICAL **: Source ID 18 was not found when attempting to remove it
@Patrick, Will do that and update the blog. Much appreciated Patrick.
@RatFeet, I would turn on GStreamer debugging and see if you get any error messages when the stream starts.
Check out this link: http://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer/...
Set GST_DEBUG to something like 6 and watch the console window.
Also, I would check that you have the latest version of the video driver for your system. Don't rely on windows update, go directly to the manufacturer's website.
-
32
-
33
-
34
-
35
-
36
of 39 Next