Part 2: How to build a High-Definition FPV UAV using a Rasperry PI with HD camera, using a high speed WiFi link

Part Two: Here is the original picture of the finished product:

This is the second part of a 2-part series on 'How to build a High-Definition FPV UAV using a Raspberry PI with HD camera, using a high speed WiFi link.

In my first post on the subject (located here), I discussed the parts I used, and how to install them into a Hobby King Go-Discover FPV model. 

In this post, I will discuss installing the Raspberry PI and the PI camera in the Go-Discover gimbals, and the software configuration for both the Raspberry PI and the ground station PC.

From the previous post, step 3 was completed by installing the Ubiquity Rocket M5 in the model.  Now onto step 4:

Step 4: Install the Raspberry PI and PI Camera

Here is a photo of the position of the PI in the Go-Discover model:

The PI fits nicely just behind the camera gimbals, with the USB and HDMI ports on top. In the right side you can see the Cat5 network cable attached. This cable connects to the ethernet switch, which is also connected to the Rocket M5 input port.  

The two cables shown on top are the servo control wires for the gimbals, which I have directly connected to channel 4 and 5 on my radio.  I am using channel 4 (normally the rudder stick on my radio. Since there is no rudder on a flying wing, this is a convenient channel to use to move left and right with the camera. I have not (yet) moved to a head tracker, but if you already have that setup, just assign the channels accordingly.

To install the PI camera, remove the stock plate from the gimbals (for a GoPro), and mount the PI camera as shown in this photo:

The PI camera case fits very nicely into the slot, and again I used a small piece of velcro to hold it down. You could use a couple of small screws instead if you want a more secure hold.  The two gimbals servos are also shown here. They are simple to install, just follow the Go-Discover instructions.

Here is a front view of the PI camera installed:

Here is the block diagram describing all the connections:

Some comments on my previous post suggested that it is possible to eliminate the ethernet switch and serial-to-ethernet converter using the Raspberry PI and a serial port on the PI. I believe this post describes how to talk to the PI via the NavLink, but in this case, I want to use the PI to bridge the connection from the ground station to the APM/PixHawk. Somebody please comment on this if you know more about it.   I believe it would require a TCP/IP to serial link from the PI to the telemetry port on the APM, and some software on the PI to act as the bridge.  The main connection to the ground station is via the Rocket M5 and TCP/IP, not through a telemetry link (900 Mhz or Zigbee like I used on my other models).

Step 5: Getting it all to work with software configuration (the really fun part starts now).

Check out this post on what others have done with streaming and the PI.  My experiments showed that using GStreamer on both the PI and on Windows gives really good results with very low latency, if you use the right parameters. 

Get GStreamer on the PI by following this blog.   This is the same version of GStreamer that I am using on my setup. 

Make sure your PI camera works ok by plugging in the PI to a standard monitor using the HDMI port and follow the instructions on the Raspberry PI website on how to get the camera up and running (without GStreamer).  Once you have a working PI and camera, you can then proceed to stream things over the network.  

Note: It is suggested that you first get the PI streaming video by plugging it directly into your local network where you can also connect your ground station PC with the correct IP addresses (without the Rocket M5).   For my PI, I picked 192.168.1.2,  and for the ground station, 192.168.1.1.    Make sure you can ping the PI from your PC and the PC from the PI.  

For streaming, you will also have to make sure all the ports you intent to use are open on the firewall (described later).

For the ground station PC,  you can download GStreamer here.  Make sure when you install, select to install everything , or full installation (not the default). 

Here is the command I use for the PI to pipe the camera output to GStreamer:

raspivid -t 0 -w 1280 -h 720 -fps 30 -b 1700000 -o - | gst-launch1.0 -v fdsrc ! h264parse config-interval=1 ! rtph264pay ! udpsink host = 192.168.1.1 port= 9000

The command is explained as follows:

raspivid is the command to start the camera capture on the PI.  The -w switch is for the width in pixels, and the -h switch is for the height.  In this case, I am using 1280 X 720, but you can try any combination that fits your needs. 

The -b switch is the bit rate for the sampling. In this case I chose 1.7mbs to send over the stream. Again you can experiment with higher or lower values. This settings seems to work good for me, and the latency is almost unnoticeable.  

the "-o - |" is piping the output to gstreamer.  Make sure you include the dash before the pipe "|" symbol. 

For the GStreamer command, all the filters are separated with an exclamation point "!", as these are individual drivers that are part of GStreamer.  Since the PI has hardware accelerated video, the output is in a format called "H264", which is a highly-compressed stream. The GStreamer filters are configured to transport the output via a UDP socket connection to the target PC. Notice the 'udpsink' element which specifies the host - in this case your ground station, and the UDP port.  I am using port 9000, but you can use any open port on your system, but be sure to open the firewall or it won't work!  You can also use TCP instead of UDP, but for such a data stream, I chose to use UDP since dropouts are certainly possible, and with UDP this is ok, but with TCP, you could have socket problems and higher latency. 

Note: to get the PI to execute this command on boot, make a shell script with the above command and add it to your local.rc boot sequence. That way when the PI boots, you get the stream without having to log into the PI remotely. 

For the ground station PC, once you have installed GStreamer and opened the correct ports, use this command (from the command prompt) to view the stream:

c:\gstreamer\1.0\x86_64\bin\gst-launch-1.0 udpsrc port=9000 ! application/x-rtp,encoding-name=H264,payload=96 ! rtph264depay ! avdec_h264 ! videoconvert ! autovideosink

If all goes well, you should see the PI camera output on your PC screen in a popup window.  For those of you what want to use FPV goggles, you can connect to the HDMI port on your PC to display the output if your goggles support HDMI. 

I have this command in a batch file (with a PAUSE) statement at the end to keep the window open.

WHEW!  If you got this far, you are amazing. 

The last step to complete the build is to connect to the APM from mission planner.  The method I used to connect was to install a utility that converts a TCP connection to a virtual serial port, but I also think that directly connecting the mission planner to the TCP port will also work, however I have not tried it. I will post back later after trying it.

Here is the link to setup the serial to ethernet device to have an IP address and port.

Here is the link to the configuration utility for installing the virtual serial port.   

Once you have a serial connection over TCP/IP working to the APM, you should be able to connect with Mission Planner. On the maiden flight, it worked perfectly, and I didn't see a single drop in the telemetry data or anything noticeable in the video transmission, however my first flight was limited to 2km.

The last step is to connect the Rocket M5 to the Nano M5 and test everything using the OTA (over the air) connection. If all is well, you are ready to fly!  But be careful on your maiden, you just spent $700. 

Finally, here is a photo of my Antenna Tracker with the Nano M5 attached. My next update will include a video of a longer flight.  

Happy Flying!

Views: 95727

Comment by Ashley Webb on March 13, 2017 at 3:39pm

@Patrick, Hi, thanks so much for your previous help. I have a quick question and will then hopefully be up to speed with you guys. Im not getting a response when I try the ARMING_CHECK

I installed Mavproxy and Mavlink over ssh after setting up a static ip for the pi. I have telem 2 connected to the pixhawk (5v, gnd, tx, rx) 

I ran these 4, no problem: 
sudo apt-get update
sudo apt-get install screen python-wxgtk2.8 python-matplotlib python-opencv python-pip python-numpy python-dev libxml2-dev libxslt-dev
sudo pip install pymavlink
sudo pip install mavproxy
I disabled os control of serial through the advanced options then tested the connection:
sudo -s
mavproxy.py --master=/dev/ttyAMA0 --baudrate 57600 --aircraft MyCopterConnect /dev/ttyAMA0 source_system=255
no script MyCopter/mavinit.scr
Log Directory: MyCopter/logs/2017-03-13/flight2
Telemetry log: MyCopter/logs/2017-03-13/flight2/flight.tlog
MAV> sudo -s
MAV> mavproxy.py --master=/dev/ttyAMA0 --baudrate 57600 --aircraft MyCopter

Heres where I'm having issue. I'm not getting a response and its not moving into STABILIZE>

MAV> param show ARMING_CHECK
MAV> 

Any idea what Im missing? Thanks so much!

Comment by babak_ea on April 24, 2017 at 1:32am

@Patrick,Is this Possible that ,Replace Rpi Instead of laptop on Ground station and using the Gstreamer for FPV and recording? If yes, what is the command that I use for ground station's RPi?

Comment by Patrick Duffy on April 24, 2017 at 8:36am

@babak_ea,  I use this pipeline on windows to record, but you could modify for linux:

udpsrc port=9000  buffer-size=60000 ! application/x-rtp,encoding-name=H264,payload=96 ! rtph264depay ! h264parse ! tee name=record ! queue ! avdec_h264 ! video/x-raw, format=I420 ! d3dvideosink sync=false record. ! queue ! mp4mux ! filesink location = /temp/fpv.mp4

Replace the sink that works for the PI.  

Comment by Ashley Webb on April 24, 2017 at 9:10am

Hi @Patrick, I've installed mavlink and mavproxy on my rpi 3 but I cannot get a heartbeat. I've changed the serial port to uart=1 in config, used S0 to access the digital serial because Im on the rpi3, I've changed the baud rate, reinstalled on an rpi2, no heartbeat.

I'd love to use the image you're using. Thanks again!  

Comment by Patrick Duffy on April 25, 2017 at 8:50am

@Ashley,  I need to find some cloud storage that I can upload it. My free dropbox is full.  Let me see what I can do.

Comment by Luke on June 17, 2017 at 6:56am
I highly recommend you check out ez-wifibroadcast. It uses raspberry pi to transmit low latency (below 175ms) hd video and telemetry over wifi in monitor mode. You can just download the images- about as plug n play as it gets with diy stuff like this.

https://github.com/bortek/EZ-WifiBroadcast
Comment by Patrick Duffy on June 17, 2017 at 6:21pm

@Luke,  thanks for the tip.  I have looked into WiFi broadcast but have opted to use Ubiquity hardware instead to get the use of commercial high-gain antennas.  I can get 15km with ordinary wifi using a NanoBeam M5 and a 2 watt transmitter on the UAV.  Both are available to consumers and are FCC legal.  You do need a tracker for the dish though. My experience is that it works very well and there is no need to use wifi broadcast.   If there was a way to get Ubiquity hardware to work using WiFi broadcast, I would switch.  If you know of a setup that can get that kind of range using WiFi broadcast with commercially available hardware, please let me know.

Comment by Rana on June 17, 2017 at 10:22pm

Hi Patrik, Thanks for your wonderful work.

In EZ_WifiBroadcast, what is done is the transmission band is shifted away from the regular wifi band by making modification in the linux driver /firmware for USB-WiFi dongle so that it does not interfere with the other wifi systems in the area.

Also RF power of upto +30dBm is achieved with the same way increasing the range, secondary the video downlink is made uni-directional using UDP so that the video does not stall if case of break for a very short duration.

Comment by Ashley Webb on July 21, 2017 at 12:07am

Hey @Patrick! I've got my ubiquiti set up working with my rpi running apsync. I'm now trying to get my antenna tracker running with my ubiquiti antenna on it. Are you able to bypass the 3dr radio? Ive tried a pixhawk on the tracker with antenna tracker installed and I've tried a servo controller plugged directly into mission planner. No response from the tracker but I tested the servos seperately and they are working. Can you share your set up for your ubiquiti antenna tracker? Thanks for your help!   

Comment by Patrick Duffy on July 21, 2017 at 4:12am

@Ashley, I am using the MissionPlanner tracker, not ArduTracker.  I was using the ArduTracker setup with a Pixhawk and a Raspberry Pi to feed the Pixhawk telemetry from the Ubiquity connection.  With continuous rotation servos and it was basically working, but it was unstable and would often go wild and start rotating randomly and the limits settings did not work so it ended up causing damage because servos didn't stop when they should have.  I struggled to get it stable.

As as result, I abandoned the ArduTracker GPS solution and went back to Mission Planner's simple tracker.  I wrote a blog about how to setup a tracker using an Arduino.   This setup is stable and works better than ArduTracker, with the limitation of non-continuous rotation.  I would suggest that you use this setup rather than ArduTracker.  The firmware for ArduTracker is not stable in my opinion.  It's actually over complex to use this setup rather than MissionPlanner to drive a simple servo.

One note on the servos not responding.  ArduTracker will not turn on the servos until you arm your copter.  A blue light on the tracker will light up when it thinks you have armed.

Comment

You need to be a member of DIY Drones to add comments!

Join DIY Drones

Groups

Season Two of the Trust Time Trial (T3) Contest 
A list of all T3 contests is here. The current round, the Vertical Horizontal one, is here

© 2019   Created by Chris Anderson.   Powered by

Badges  |  Report an Issue  |  Terms of Service