Over the course of the last two weeks, I built a fully digital FPV setup.
Previously, I used an analog camera, Eagle Tree OSD, 1.3GHz Video Link and a handmade FPV ground station + FatShark goggles. But well... the video quality arriving on the ground was pretty bad when compared to current video standards, even when I used a GoPro as the video source.
This led to my current setup: Digital IP camera (really cool, it can be removed by screwing it out of the black holder) and a WLAN bridge. As the FatSharks do not support digital video they had to be replaced... by a 40" flatscreen hanging in my car (pics after this weekend). It's awesome!
Naturally, raw video is not enough: I need an OSD. First version used my laptop's crappy camera, read out using OpenCV:
The data will be read out from a ArduPilot (should they ever be available again here in Europe... still waiting for mine) connected over the same WLAN. The serial connection will be emulated over the LAN using a WIZNet RS232 gateway. I already tried out the thing, works without a flaw! When the ArduPilot arrives I will also be able to use my antenna tracking code. Currently, without tracking, I get about 2km of range over something in the range of 100° in front of the receiver antenna, outside of that angle the framerate drops.
When the IP cam finally arrived there was a huge drawback: OpenCV cannot access it! The built-in Live555 server just quits with an error (something like "461 Unsupported Transport" if I recall it correctly). I tried many solutions, but the only solution that fully supported my IP cam (which is only streaming h.264 over RTSP) was embedding VLC using libVLC. This works really good and as a convenient side-effect my analog grabber is now supported without modification, too, as are all possible devices that VLC can access out of the box. This solution however has two serious problems: First, the video image is rendered onto my widget (I am programming in C++ with Qt) using direct render which means my pretty overlays shown above did not work so well. This one I managed to bypass (please ignore all the stuff lying around... plus big black thing for privacy reasons):
The second problem is the following: VLC enforces a minimum network buffer latency of approximately 215ms. I can fly that thing, but it really is not pretty. So heres the question to the programmers: Is there a better way to grab h.264 encoded frames over RTSP? If your first thought is "Just drop VLC and use ffmpeg and live555 without the VLC overhead" I kindly invite you to implement exactly that after I sent you the source code.
Conclusion:
The video quality is awesome, flying around with the X8 really is fun. No interference with my 2.4GHz radio setup. The image lag is a problem, I would currently not try to land using the video strea, but once that is solved, this is the future!
Finally, two more pics of the hardware setup:
Camera with protection cover, WLAN access point.
Retractable landing gear!
If anyone of you knows more about or has a solution to the streaming lag problem, every answer is greatly apprectiated!
Nicolas
Comments
Can you give me any details on that gimbled camera system you have installed? That would work perfectly for a project I'm working on.
-Topher
Nicholas,
On the ground I have a B2HP powered by a 11.1V LiPo and a single ethernet cable running back to my GCS laptop. I'm using a virtual serial port as you say - using Paparazzi autopilot on Ubuntu it is quite easy to script a virtual serial port without a need for windows shareware etc. So my GCS software thinks it is connected to a USB port but the virtual serial port redirects this traffic via the ethernet link using UDP.
Of course the B2HP ground radio also has a serial port and I have the option of setting up a second virtual serial port and connecting this to a serial servo controller to drive a tracking antenna. All with a single ethernet cable between the GCS laptop and ground radio/antenna unit.This is still on my todo list but I'm certain it can be done.
Overall, a high powered digital WIFI link is a very capable system but it requires a little more research and understanding then your average off-the-shelf product. Still well within the reach of amateurs who are prepared to put in the extra work.
Hi,
@RH1N0: "Won't work" just makes me want it more ;-)
How are you accessing the serial stream on the ground side? The WIZNet solution has the advantage of having a driver which is providing a virtual serial port on windows.
@Dr Mike Black: At the current time I would not really recommend using this setup instead of an analog one. It certainly is a fascinating flight experience completely in HD but the lag is noticeable. As stated above, I can fly it, but I try to stay at least 50..70m up high so my safety pilot can recover the aircraft when I do something stupid. However, I am using a full Ubiquity setup: AirCam Dome + PicoStation in the air, Bullet on the ground. The software is custom built. If you need help once you have the equipment just ask!
Unfortunately, there does not seem much concern from a programmers side to take on the problem. There always is the possibility of buying an industrial-grade camera with an SDK available, but, apart from being expensive, that would limit the possibilities to couple the viewing program with other cameras - and I would really like to release the program for everyone to use.
If the lag problems can be solved, I think that the benefits of having a network (even if it is not capable of realtime) far outweigh the costs, not only for FPV but for autonomous flying in general. Switching to digital only for the APM and my other equipment whilst having an entire analog transmission path besides that results in massive increase in weight which I do not want. So this problem has to be solved :-)
Regards,
Nicolas
Very nice! I was considering trying this with the new 802.11ac hardware. Not sure how many antennas it uses but I imagine the fewer the better so 802.11G may be better and seems to have enough throughput. I wonder if it would be better to use a non-IP camera and convert the analog (or digital) output directly to another transport format to try to do away with the latency? I would rather have a dropped frame or pixels rather than buffering at the beginning. Hmmm, seems all we need is someone to code a protocol that has no error checking/buffering that we can push video through.
Hi all,
I have to say, im dribbling here!
not over familiar with the concept of wifi up and down, and chance you can post you airside setup and ground side?
I HAVE to give this a go...
kind regards.
Mike
Hi Nicolas,
This is a great setup for aerial surveilance but not so good for FPV due to the lag as you have discovered. You can configure the buffer time in VLC but I found that anything less then 100 mS will start to drop frames.
I am using a P2HP and I have removed the PCB from its case and modified the firmware to allow me to use the internal serial port for my telemetry. I have achieved a 10 Km secure digital video/data link with this setup.
@Jaan: I've got a Bullet M2HP on the ground and a Flat Plane Antenna, both hooked up to a laptop which does the processing. Connection to the TV via VGA.
@Mike: Glad you like it!
@Martin: That's the idea :-) As I said, the RS232 link over wifi works, I'm just waiting for my APM2.
@ Not Sure: Thanks for the suggestions! I'm going to let the picostation untouched though, the current setup allows me to just slide it off the model using the builtin connector.
@Rana: I'm going to search for it, but the idea for the plane was to have an entire network up there - I've got a network switch in the plane to connect the Picostation, camera and APM. Direct connection via UFL spoils that a bit.
@Hai Train: The problem is, as Mike says in the comment directly under yours, the network buffer. Compression and decompression doesn't take that much time - however I didn't measure it exactly as I am currently working on solving the VLC issues. I looked at the Teradek equipment, too, but to be honest: As long as I don't do this professionally, they are a little bit pricey. In the non-hobby department they are the way to go, though I would couple one of their wired encoders with my access point to get more range and better signal.
@Mike de Landgraaf: Yep, it's the rtsp buffer. VLC has an option to adjust it but I can't drop it below the said 215ms - there is not much more delay besides that. Currently I'm flying unencrypted, too, but as long as the other guys at the airstrip consider everything I do "magic" I don't think they will do more than tap into the video feed.
Keep the comments coming!
Hi Nicolas,
I am currently doing the same, however i came across the lag just like you. It's the rtsp buffer thats giving you problems. However, as i'm not flying fpv, it really isnt a big problem for me. I have a 100Mbit link between every helicopter i have, with a 1080P HD stream at 30fps (and enough bandwith for 4 more streams). . There was rather a bit tweaking, hardware and software, but it now works fine.. You can change your wpa encryption to AES, that speeds up things for sure. However, it does not solve your 200ms lag problem....
@Nicolas. In relation to latency, I think that has to do with the video codec compression. I use a Teradek HDMI to h264 over 802.11n, and with default settings its latency is noticeable at 300ms, but with a Teradek decoder I have the latency down to < 100ms which isn't noticeable.
Nicolas, my wi-fi camera is a regular wi-fi video camera, non HD, purchased from ebay, an year back, don't remember the link, you can google it.
This camera has UFL connector from where you can feed rf to additional 1W wi-fi amplifier but note that you require 1W amplified wi-fi at ground.
-
1
-
2
-
3
-
4
of 4 Next