http://intuitiveaerial.com/home/2013/7/14/oculus-fpv-first-flight
First flight of our Oculus Rift FPV system!
This system is completely digital. A laptop in the air compresses the video from the two cameras in realtime and sends them over 802.11n (2.4 GHz) down to the laptop on the ground.
Latency is around 120 ms - fully usable for flying!
Range is 50-100 m using built-in wifi-cards in laptops. We will increase this by using external APs. Ranges for e.g. Ubiquiti-systems are quoted as several km - using legal frequencies and power levels. When out of range and then back the reconnection is instant (no "connection broken, reconnecting...").
There is a huge difference between this and an analogue FPV system. The all-digital design means realtime HD video from multiple (not limited to 2) cameras can be broadcasted and received. The received images are then combined into a view-sphere which can then be viewed in a below-25-ms latency loop between ground computer and the Rift. Jonathan is working on the code to let the imu of the Rift determine what is seen.
This project is an R&D effort from Intuitive Aerial. We have dreamed of this system since before we heard of the Oculus Rift. It is like a piece of science fiction that just came true. We have plans for how we can go from software running on laptops to small specific-purpose units so no computer is needed (FPGA-based compression and encode, lower-latency camera connection, possibly integrate cameras on pcb). Such units would also be useful for regular FPV and would offer HD resolutions and wifi interoperability. Let us know if you want this to happen - right now Jonathan is the only one working on this, as his master's thesis project :)
Comments
The digital downlink we are using could of course be used just for the digital video. Since it is WiFi based, a laptop can receive the signal. With directional antennas and/or higher power APs the distance is ok. Flying a laptop is not so convenient though.
Therefore to proceed further with this project there is needed to be developed some form of embedded system doing the video encode in the air and interfacing with the cameras. Developing that is not too easy..
Low latency digital video is hard because there are so many steps adding latency. Just getting an image from the cameras can take 30 ms, and that's on a good camera over usb2. Teradek Cube is an example of a digital link that is good but too laggy. Also, the Cube doesn't offer multiple camera streams or suiting camera interfaces for what we want to do here. (The Bolt is low-latency but also low-range, there are rumors of new chips from Amimon that might be better..)
The wooden box is needed because we're prototyping a fully digital system and rely on a computer. Se earlier comments for why one would want a digital system.
As cool as the oculus rift is I think it would be more useful to concentrate on finding a way to transmit low latency digital video in a small package that doesnt interfere with 2.4 GHz RC equipment. Once that is available you could use any number of already available devices to manipulate and view the video. Just being able to view FPV video using the same computer or tablet that my ground control station is on would be huge.
I would like this to happen for sure!!
I for one am very interested. I am surprised you got the latency that low. Do you plan on releasing the code?
Next goal for the human race: brushless gimbal + oculus rift + uploaded to goo tube in 3D
A digital FPV system can (when it is more developed) offer:
- HD resolutions
- Stereo imaging / Multi camera support
- Stitch surroundings into "viewing spere", which can then be viewed in e.g. Oculus Rift for super-immersive experiences
- Legal spectrum downlink with good range
cf. Jamie Hyneman here: https://www.youtube.com/watch?v=EKhp4H3DUwE
Yes in some sense we are flying a wooden box with a laptop - but look at what this has the potential to be. There is more sophistication here than might be obvious. The video shows:
1. A brief overview of the system (a more in depth technical info can be found here: http://intuitiveaerial.com/home/2013/6/8/oculus-rift-fpv)
2. Flying the system, demonstrating control using video from the WiFi downlink
3. Showing the actual footage from the system
4. A brief outline of how we could develop this further
WiFi interference can be an issue with some Tx/Rx equipment. Given how much there is on 2.4 GHz these days it is always a good idea to test range before flight. We have good experiences with stock Futaba (not so good with FrSky), and if you want to go across town there are LRS options for that.
The video is actually very funny. It is just a big wooden box with a laptop and a webcam inside that is lifted by a multi-rotor. Not very sophisticated yet :)
Issue may be with the 2.4 GHz band conflicting with the radio controls.
http://diydrones.com/profiles/blogs/flying-with-new-gopro-3-wifi-tu...
But is a good start and surely it will be more compacted in the future.