This blog is a continuation of my previous post.
How to build a High-Definition FPV UAV using a Rasperry PI with HD camera, using a high speed WiFi link
This post will discuss how to use GStreamer and Mission Planner together to display the HD video with a HUD (Head-Up-Display).
Note: I have only tested this feature on Windows so the instructions given here are for Windows only.
To give proper credit, the HUD created here was borrowed from APM Planner, a Qt-Based app similar to Mission Planner. The HUD part was created from the Qt codebase QML HUD created by Bill Bonney who is on the APM Planner development team. To make the HUD work with the background video, I used a GStreamer library called "QtGStreamer" which integrates GStreamer plugins with painting on a Qt widget. This library is available on the GStreamer website.
The end-result is dynamically added to Mission Planner using the plug-in architecture.
In the previous posts I discussed used a Raspberry PI and a High-speed WiFi link using GStreamer on the PI and the ground station PC. To get the HUD to work, you need to already have a successful link with the video on your ground station.
Here are the steps to follow to install the plugin:
1) Install Mission Planner.
2) Download and install GStreamer from this link. Use the x86 version, the x86_64 version will NOT work. (Use the default path 'C:\GStreamer' when installing). When installing GStreamer, select 'Custom' install and select ALL plugins to be installed.
3) Follow the steps in the previous blog noted above to get your video stream working.
4) Download and the MSI installer from this link. and run the installer.
If all went well, you should have the plugin installed.
Open Mission Planner and navigate to the "Flight Data" page and right-click on the map. You should see a menu item called "GStreamer HUD" as shown below:
Select this menu item and the following screen should appear:
In the upper-left corner is a context menu. Here is where you enter your GStreamer Pipeline string. If you had the video displaying without the HUD using a valid pipeline, enter it here.
Note: The GStreamer Pipeline string should be exactly the same as the string you used before, but WITHOUT the final video sink element. The video sink is the QtGStreamer element which will be added automatically by the plugin. The GStreamer pipe should therefore be the same, except remove the last element for the video sink.
Here is an example string I used on my setup:
udpsrc port=9000 buffer-size=60000 ! application/x-rtp,encoding-name=H264,payload=96 ! rtph264depay ! h264parse ! queue ! avdec_h264
If all is well, you can connect to your UAV and see the HUD elements moving. To change the HUD, right click on the display and select which elements you want to display. The default is to display everything shown here.
If anybody has problems, please post back and I'll update the blog in case I missed something, and you cannot get it to work.
Happy Flying!
Comments
Patrick,
Thanks for your work in this area of real time streaming video.
I am running Mission Planner and also your stand-alone HUD application. I started ArduCopter in the Pi to send telemetry to Mission Planner using port 14550 and added port 14551 to the command to also send telemetry to the HUD application:
ArduCopter-quad -A tcp:0.0.0.0:14550 -C tcp:0.0.0.0:14551
Both Mission Planner and the HUD application update as expected.
I was able to stream video to the PC using this command in the pi:
$ sudo raspivid -t 0 -h 1080 -w 1920 -fps 30 -b 2000000 -o - | gst-launch-1.0 -v fdsrc ! h264parse ! rtph264pay config-interval=1 pt=96 ! gdppay ! tcpserversink host=192.168.1.105 port=5000
And I used this command on the PC to open a gstreamer window:
C:\>gstreamer/1.0/x86/bin/gst-launch-1.0 -v tcpclientsrc host=192.168.1.105 port=5000 ! gdpdepay ! rtph264depay ! avdec_h264 ! videoconvert ! autovideosink sync=false
It worked fine, however; I cannot get a video to come up in the HUD. What pipeline can I put into the HUD application so that it displays the same stream? (Note that I have since removed the gstreamer application from the PC, since I understand (perhaps incorrectly) that the HUD comes with its own gstreamer parts.)
Thanks for your advice and be well,
Paul
@Patrick, I had to uninstall, delete the folders and reinstall both programs, but now I don't have the error anymore. In the next days I will try to stream, but I believe there will not be problems. Thanks!
Can you explain me what is the "Cam Adapter:IP" under setting?
And to receive data, can I use the same UDP port that Mission Planner is using or have I to use another one?
@Mattia, you have a conflict with the old version of the HUD. To solve this problem, you have to manually remove the files from the old version and make sure your 'path' variable does not include any other gstreamer folders on your system. The HUD uses it's own local copy of gstreamer so if you have gstreamer DLLs in another folder and the path is set to search these folders, you will get a conflict.
@Patrick, thanks. I have installed the stand-alone plugin but when I execute the program, I got this error:
The program anyway starts, but when I start the streaming from my RPI, the HUD immediately crashes...
Any idea?
@Mattia, If you run the stand-alone version of the HUD, you still run Mission Planner as normal, however you must split the MavLink data so that both applications get the stream from your UAV. You can use MavProxy to do this.
Hi, the stand alone version works fine but does not have all the options of the Mission Planner. You can make them work at the same time without any trouble. However it would be so convenient to have it in an integrated window in mission planner.
you can find a download link in the comments (page 30 for me)
Hi, I had the same problem than Tommy_768 (Gstreamer HUD disappear from the options) after updating my MissionPlanner. I tried all the previous versions and the last that works fine is the 1.39.
Robert Mapes suggests to use the stand alone version of the Gstreamer HUD, but where can I download it?
Or did anybody find a solution?
@Robert, Thank you for the feedback. I had switched the pipeline to videotesrc just to be sure that there was activity. I was aware that would only give me the test screen. When you mention the D3D window, does that open with QT HUD or just when I launch Gstreamer? And by close do you mean to "X" out or minimize the screen? Also when I copy the pipeline form the command line, which command line are you referring to? Is it the command to start Raspivid or what?
The good news is that it is working again, but I was making so many changes I need to figure out what I did. I did try other pipeline strings and I hope they save. So my workflow is log in to Pi/Navio > start Raspivid > Start Ardu Rover > Start QT HUD > Start Mission Planner > Connect Mission Planner > Go to QT HUD and connect telemetry on 14551. (Your Help is really appreciated here).
It looks like a video window. if it is the D3D video window that opens first, you must then close it before running the QT HUD app. The QT HUD app video window does not work, doesn't overlay inside the HUD until you do this. Only one video can stream on the 9000 port at one time. Also, your pipeline in the HUD is videotestsrc which is what you are seeing, just the test screen. It should match what shows up in your command line. It doesn't match. copy what is in your command line and paste it into the pipeline and hit enter. It will save in the dropdown and should appear in your command line.
I am not sure but would think it is a Bitvise SSH window starting the Raspid
-
4
-
5
-
6
-
7
-
8
of 39 Next