This is a new Android App for running a Heads Up Display on a SmartPhone or table computer or Windows. It's available on the Google Play store at https://play.google.com/store/apps/details?id=org.qtproject.qtgstreamer
If you want to side-load, I have posted a free version on dropbox at https://www.dropbox.com/s/g38tv79z3ey8yks/QtGStreamerHUD.apk?dl=0
There is also a windows version available at https://www.dropbox.com/s/km52szl7ghdo15a/GStreamerHUDApp.msi?dl=0
This app will overlay the HUD on a video stream using GStreamer, or you can run the app without video with a normal HUD.
The App will automatically listen for a MAV-Link data stream on UDP port 14550, the default UDP port for MAV-Link. You can also configure the app to use TCP, and on windows you can connect directly to a serial port.
If you want to display data on a ground station PC and the App at the same time, you can use MavProxy to split the UDP stream, and then connect to your UAV using Mission Planner.
Here is a sample command that I use to split the data stream with MavProxy:
mavproxy --master=COM5,115200 --out=192.168.0.1:14550 --out=192.168.0.126:14550
The ground station PC is at 192.168.0.1, and the SmartPhone is at 192.168.0.126 (on my wifi network). You will have to configure the IP address according to your own WiFi network. My WiFi network in the field is a Ubiquity Rocket M5 configured as an access point, and a second M5 is on the UAV, so I can get live video from the UAV over the same network that connects the smartphone.
For the windows version, you can run the HUD on the same PC as MissionPlanner (or APM Planner), and use MavProxy with a different UDP port for the HUD.
If you want to use the video overlay, you will need to have a valid gstreamer data stream running, either over UDP or TCP. For my setup, I am using a Raspberry PI streaming H264 over UDP using this command on the PI:
raspivid -t 0 -w 1280 -h 720 -fps 40 -b 4000000 -o - | gst-launch-1.0 -v fdsrc ! h264parse config-interval=1 ! rtph264pay ! udpsink host= $1 port=9000
The "$1" argument is the IP address of your GCS or SmartPhone/Tablet PC. This command is for 720p, but you should consider the resolution of the device and adjust accordingly. It makes no sense to send a 720p video to a phone with a 800x600 display.
In the HUD, configure the gstreamer pipeline with this string:
udpsrc port=9000 buffer-size=60000 ! application/x-rtp,encoding-name=H264,payload=96 ! rtph264depay ! h264parse ! queue ! avdec_h264
Do not include a video sink element, as the App will add this dynamically. You can change the port to whatever you wish as long as your PCs do not have a firewall blocking the port.
Another thing to consider is the processing power of your device. If you see a lot of pixelation, your device is too slow, so you will need to change the resolution of the transmission and/or the bitrate. Some smartphones are just too slow to display the video at full resolution. I have tested 720p on my Google Nexus 6 and it works fine, but on a cheap tablet PC, I had to slow it down and switch to 800x600.
The App is easy to setup, however if you have any questions or issues, please send me feedback.
Happy Flying!
Comments
@RogerCon, I have posted the XP version on the dropbox. I think it works, but I would like somebody to verify it.
@bocorps, You should connect the phone via WiFi. But the trick is to configure your ground M5 to be a normal access point (not Ubiquity 'Airmax' TDMA/MIMO). Set the bandwidth to 20mhz or your phone won't see the access point. For 85% of the folks out there, you can disable AirMax and still get the range you are interested in.
You have to configure your remote M5 to also use standard WiFi with a 20mhz bandwidth, and then you can connect both the remote M5 and the phone to the ground M5, and the M5 acts as a wireless router. You can connect your ground PC also using WiFi to the M5, but you still need a POE to power it.
Once you have both your phone and GCS connected to the ground M5, you can run MavProxy on the GCS to send the MavLink data to your phone and the GCS. The video can also be sent to the phone is the same way. On your remote, just use the IP address of the phone when you start the stream. I'll be posting a blog with a video later on how to do this.
@Patrick, Can you explain how do you plug your external antenna to your smartphone ?
I tried to plug my a nano M5 through an etho-usb adapter + OTG cable but the phone cannot detect antenna. I guess it is from the usb adapter that would need a powered usb hub but it is just an assumption.
I will try the above when I get back to our shop. Will try the XP if its ready too. thx!
@RogerCon, So you want to start the HUD with the pipeline string on the command line? That's easy to add.
I have added a feature where it remembers any new strings and adds them to a drop list. Currently it only has a place for one string, so in case you have a 'record' and 'display' version, you can easily switch between them, or any version you want.
I have the XP version compiled, but I have not tested yet. I'll add that to the dropbox after I blow the dust off my XP machine and give it a try.
That should work.
I use the following batch file:
set hr=%time:~0,2%
if "%hr:~0,1%" equ " " set hr=0%hr:~1,1%
gst-launch-1.0.exe -e -v udpsrc port=9006 buffer-size=60000 ! application/x-rtp,encoding-name=H264,payload=96 ! rtph264depay ! h264parse ! tee name=record ! queue ! avdec_h264 ! video/x-raw, format=I420 ! timeoverlay ! d3dvideosink sync=false record. ! queue ! mp4mux faststart=true ! filesink location="./Videolog/Video_%date:~-4,4%%date:~-10,2%%date:~7,2%_%hr%%time:~3,2%%time:~6,2%.mp4"
Using faststart=true puts the header at the beginning of the file as I could not get around the EOS usage.
The reason for the "set" parameters is for the time format showing XXX_09:00 instead of XXX_ 9:00
Any thoughts on how to incorporate the above file format on the hud command line?
@RogerCon, I use a tee to record the video, so splitting the stream should work. Here is the pipeline I use:
udpsrc port=9000 buffer-size=60000 ! application/x-rtp,encoding-name=H264,payload=96 ! rtph264depay ! h264parse ! tee name=record ! queue ! avdec_h264 record. ! queue ! mp4mux ! filesink location = /temp/fpv.mp4
To get it to record, you have to hit the 'Stop' button on the toolbar, otherwise, the stream never gets the EOS signal and the MP4 headers are not written to the file and you cannot open them. Give this a try and let me know if it works. It works for me.
I have posted the source for Android, and I will be posting the Windows version in a couple of days.
When you say make the screen smaller, do you mean the HUD elements?
Some messages should be displayed, like alarms, there is a level check before they get displayed, and I think the default is 3. I'll add an option for changing it.
As for the customization of the UI, that's the plan, but the current API does not capture all the Mavlink items into the data structure so more items need to be added in code. The classes were borrowed from QGroundControl, but eventually, I want to make a user-selectable way of adding items dynamically with a 'drag and drop' mechanism.
Works like a charm on my w7 machine. Well done!
We log every flight so we use tee and filesink while watching the stream, ive tried to add filesink but it didnt work
Would it be hard to add ?.
Are you going to release the source?
if not here is a wishlist.
-Be able to make the screen even smaller
-Include warnings/failsafe messages.
-Add more selectable mavlink data
@Bill, here's the link to the source: https://github.com/pnduffy/QtGStreamerHudAndroid
I don't have the build instructions yet, or a CMake configuration that sets up all the dependencies. The dependencies are: GStreamer for Android, (GLib), QtGStreamer (I had to build this custom for Android) , the Android NDK, Boost for Android, Qt for Android ARMv7. You can get the GStreamer and QtGStreamer libraries from gstreamer.freedesktop.org.
@Bill, I'll post it later today and send you a link.
-
4
-
5
-
6
-
7
-
8
of 8 Next