This blog is a continuation of my previous post.
How to build a High-Definition FPV UAV using a Rasperry PI with HD camera, using a high speed WiFi link
This post will discuss how to use GStreamer and Mission Planner together to display the HD video with a HUD (Head-Up-Display).
Note: I have only tested this feature on Windows so the instructions given here are for Windows only.
To give proper credit, the HUD created here was borrowed from APM Planner, a Qt-Based app similar to Mission Planner. The HUD part was created from the Qt codebase QML HUD created by Bill Bonney who is on the APM Planner development team. To make the HUD work with the background video, I used a GStreamer library called "QtGStreamer" which integrates GStreamer plugins with painting on a Qt widget. This library is available on the GStreamer website.
The end-result is dynamically added to Mission Planner using the plug-in architecture.
In the previous posts I discussed used a Raspberry PI and a High-speed WiFi link using GStreamer on the PI and the ground station PC. To get the HUD to work, you need to already have a successful link with the video on your ground station.
Here are the steps to follow to install the plugin:
1) Install Mission Planner.
2) Download and install GStreamer from this link. Use the x86 version, the x86_64 version will NOT work. (Use the default path 'C:\GStreamer' when installing). When installing GStreamer, select 'Custom' install and select ALL plugins to be installed.
3) Follow the steps in the previous blog noted above to get your video stream working.
4) Download and the MSI installer from this link. and run the installer.
If all went well, you should have the plugin installed.
Open Mission Planner and navigate to the "Flight Data" page and right-click on the map. You should see a menu item called "GStreamer HUD" as shown below:
Select this menu item and the following screen should appear:
In the upper-left corner is a context menu. Here is where you enter your GStreamer Pipeline string. If you had the video displaying without the HUD using a valid pipeline, enter it here.
Note: The GStreamer Pipeline string should be exactly the same as the string you used before, but WITHOUT the final video sink element. The video sink is the QtGStreamer element which will be added automatically by the plugin. The GStreamer pipe should therefore be the same, except remove the last element for the video sink.
Here is an example string I used on my setup:
udpsrc port=9000 buffer-size=60000 ! application/x-rtp,encoding-name=H264,payload=96 ! rtph264depay ! h264parse ! queue ! avdec_h264
If all is well, you can connect to your UAV and see the HUD elements moving. To change the HUD, right click on the display and select which elements you want to display. The default is to display everything shown here.
If anybody has problems, please post back and I'll update the blog in case I missed something, and you cannot get it to work.
Happy Flying!
Comments
@Patrick! You legend. That additional exclamation point caused the problems. Working on this end now. Thanks again.
@ap105, Building QtGStreamer is a bit tricky. Do you have CMake installed on your system? Running CMake will help find the missing modules and if your path is setup correctly. I can upload the build directory but it will only work on Windows. Are you building for Windows or ??
@Baz, If you have a working stream without the HUD, it should work in the HUD, but you have to modify the string to leave off the last element (video sink). The last ! should be removed. You are showing this string:
udpsrc port=5000 ! application/x-rtp, payload=96 ! rtpjitterbuffer ! rtph264depay ! avdec_h264 !
The last element in the pipeline should be 'avdec_h264' (without extra exclamation point).
@ Patrick and anyone else who has this working... Which version of Gstreamer on the windows machine did you use for the windows install?
I'm onto the 3rd Windows machine having the same problem, in that I can get one frame into MP HUD but then it stops. Can't get a stream in a popup window from the settings @Patrick is using, but can get a stream using a different string as posted above.
Two machines are windows 7, and one is windows 8.
This is the install file I'm using for Gstreamer -
Seeing as I'm seeing it on 3 machines, I'm thinking it's something common to my install, or the Pi. All three are company computers as well. Would any administrative polices affect the dependencies or libraries installed?
@Patrick
Some progress tonight!! So I can now load a stream on the Pc, just not in the mission planner window using these strings.
On the Pi - raspivid -n -t 0 -rot 270 -w 960 -h 720 -fps 30 -b 6000000 -o - | gst-launch-1.0 -e -vvvv fdsrc ! h264parse ! rtph264pay pt=96 config-interval=5 ! udpsink host=PC_IP port=5000
And on the Windows machine -C:\gstreamer\1.0\x86\bin gst-launch-1.0 -e -v udpsrc port=5000 ! application/x-rtp, payload=96 ! rtpjitterbuffer ! rtph264depay ! avdec_h264 ! fpsdisplaysink sync=false text-overlay=false
Tried this string in mission planner but it wouldn't come up = udpsrc port=5000 ! application/x-rtp, payload=96 ! rtpjitterbuffer ! rtph264depay ! avdec_h264 !
Does this give any additional clues?
Thanks again for your help.
@Patrick. Thanks for releasing the source. I think I am close to a complete build setup, but I still fail to compile qtgstreamer. Do you have any guide how to compile it, or would you share your c:\qt-gstreamer-1.2.0\build\ directory? This would fix the remaining linker dependency issue I have.
@Baz, there is a suspicious message, "Error, Empty pipeline not allowed". Since you are specifying a valid pipeline string, I would guess that one of the plugins is not loading correctly. Try running gst-inspect on each element of your pipeline to see if there are any load issues. It could also be an issue of your 'PATH' variable. I have had issues with different version of GLIB being loaded because windows finds the glib dll in a different folder, and it causes gstreamer to choke.
@ Patrick, just turned on the Gstreamer debugging. Went through all the levels, started at 6 (Too much information to catch anything) and stepped it down. Getting this error at levels 4-1. Make any sense?
C:\gstreamer\1.0\x86\bin\gst-launch-1.0 udpsrc port=9000 buffer-size=60000 ! application/x-rtp,encoding-name=H264,payload=96 ! rtph264depay ! h264parse ! queue ! avdec_h264 ! video/x-raw, format=I420 ! d3dvideosink sync=false
Can get a still image into the HUD in MP - and the image refreshes if I interrupt the stream on the pi and restart it, but it won't give me a constant feed. Anything in here give any clues? Thanks for your help.
@Baz, Did you try turning on gstreamer debugging? If it was working, and then stopped, try uninstalling everything on the windows pc (including wiping the folder), and re-install gstreamer fresh. You can also delete the plugin cache to force it to re-create the plugin database. Did you install anything after you had it working? Could be installing the Mission Planner plugin caused something to break. Also, if you had the 64 bit version of gstreamer, make sure you remove that too.
Thanks for this
I tried both of those pipelines, one in the command prompt, and the second one in the HUD. Same problem again. It's strange as I could get the stream in a pop up window two days ago but now that won't even come up. I do get a still image in the mission planner, and when I stop and restart the streamer service on the pi the image refreshes in Mission planner.
I'm using this command on the PI end - raspivid -t 0 -w 1280 -h 720 -fps 30 -b 1700000 -o - | gst-launch1.0 -v fdsrc ! h264parse config-interval=1 ! rtph264pay ! udpsink host = "Windows Machine Ip Address" port= 9000
I've got a second Pi which I'll setup with Gstreamer to see if I can at least get the pop up working again. Could it be a port issue with port 9000? Firewall?
Also, I had the same as Steve did on the 6th of Novemember where everything seemed to install correctly but I had no option for the Gstreamer HUD in Mission Planner. When I right click MP and run as administrator that problem is solved though. Not at home at present but I'll do some more testing today and report back. Very keen to get this working!
-
31
-
32
-
33
-
34
-
35
of 39 Next