Google Cardboard your Drone!
This is a new Android App for running a Heads Up Display on a SmartPhone using Google Cardboard glasses to hold the smart phone.
I have posted a free version on dropbox at https://www.dropbox.com/s/iysadmr511nvwlr/QtGstreamerGoogleCardboar...
Your phone will need to have side-loading enabled to install the app. Please google on how to side-load an app.
This app will overlay the HUD on a video stream using GStreamer in two distinct panes for viewing in the Google Cardboard viewer. The app supports both a single and dual video stream, and depending on the processing power of you phone, it' possible to have a stereo view using multiple cameras.
The App will automatically listen for a MAV-Link data stream on UDP port 14550, the default UDP port for MAV-Link for the HUD telemetry items. To support both the video and telemetry, you will need to supply both data streams to the phone, or for a simple WiFi camera without telemetry, you can connect directly to the camera for live video.
Scenario #1: Simple connection to WiFi Camera
In the App video setup menu, (top-left toolbar) supply this string for the GStreamer pipeline:
souphttpsrc location=http://10.0.0.1:60152/liveview.JPG?%211234%21http%2dget%3a%2a%3aimage%2fjpeg%3a%2a%21%21%21%21%21 ! jpegdec
This string is for the Sony QX10. For other cameras, you will need to know how to connect and get the live stream using a standard GStreamer pipeline. I would suggest experimenting with GStreamer on a PC first to discover the correct string, then use it in the Android app.
For a Kodak PixPro SL10 camera, I found this string to work:
souphttpsrc location=http://172.16.0.254:9176 ! multipartdemux ! jpegdec
In the UI configuration menu (top-left toolbar) select, "Split Image" to get the same image on both viewing panes. This is for a single camera configuration.
Scenario #2: Companion PC (Raspberry PI) with camera and PixHawk/APM Telemetry
There is a blog post currently running about using the Raspberry PI as a Companion Pi 2/3/Zero, so I will not give details here on how to setup the PI to get telemetry via the PixHawk/APM telemetry port, please refer to those pages for instructions.
If you have a Raspberry Pi with a camera, you can use this setup:
On the PI:
raspivid -t 0 -w 1280 -h 720 -fps 40 -b 4000000 -o - | gst-launch-1.0 -v fdsrc ! h264parse config-interval=1 ! rtph264pay ! udpsink host= $1 port=9000
The "$1" argument is the IP address of your SmartPhone. This command is for 720p, but you should consider the resolution of the device and adjust accordingly. It makes no sense to send a 720p video to a phone with a 800x600 display.
In the Android App, configure the gstreamer pipeline with this string:
udpsrc port=9000 ! application/x-rtp,encoding-name=H264,payload=96 ! rtph264depay ! h264parse ! queue ! avdec_h264
For telemetry, I suggest using MAVProxy on the Raspberry Pi, and send the UDP stream to the phone's IP address and UDP port 14550. The Raspberry PI should be acting as a WiFi access point, and the phone configured with a static ip address.
If you want to display data on a ground station PC and the App at the same time, you can use MavProxy to split the UDP stream, and then connect to your UAV using Mission Planner.
Here is a sample command that I use to split the data stream with MavProxy on the Raspberry Pi.
mavproxy --master=/dev/ttyAMA0 --baudrate=115200 --out=192.168.1.1:14550 --out=192.168.0.3:14550
In this case, the phone is at IP address 192.168.0.3 (static IP address) and Mission Planner is running on a PC at 192.168.1.1. Check your phone's documentation on how to setup a static IP for WiFi. It's usually in the 'Advanced' section of the settings in your phone. You will also need to configure your Raspberry PI to have a static IP address for it's WiFi access point. The actual MAVProxy settings will depend on how you setup your PixHawk/APM, so choose the correct baud rate you have configured in the flight controller.
Another thing to consider is the processing power of your device. If you see a lot of pixelation, your device is too slow, so you will need to change the resolution of the transmission and/or the bitrate. Some smartphones are just too slow to display the video at full resolution. I have tested 720p on my Google Nexus 6 and it works fine, but on a cheap tablet PC, I had to slow it down and switch to 800x600.
In part two of this blog, I will explain how to setup the dual video stream and more advanced configurations for high-power/long range FPV using an antenna tracker and the Ubiquity Nano/Rocket M5 ground station access points.