Our all-star interns have been brewing up some cool devices this summer at 3DR. Three words: Oculus Rift FPV.
We have the Oculus communicating through the Mission Planner as a joystick input, driving the pan/tilt servo gimbal on our Skyhunter fixed-wing aircraft. Video is piped to the ground via standard 5.8GHz TX/RX gear, with the 3DR OSD kit in line. The end result is one for which you should stay firmly seated; even without the video stretched all the way to the periphery, the experience is incredibly immersive.
Our current setup is based on monocular video, but stayed tuned for more from Project Warg, including stereoscopic video, new airframes, and closed-loop aircraft control with the Oculus!
Comments
Thanks for the info Alex.
I was afraid code was involved. Im trying a hardware solution.
I'm in the process of splitting the image and displaying two camera images and streaming it via 5.8ghz transmitter. The image splitter is from a 2 camera security monitor box.
Using a composite to HDMI converter to the OR.I know the image will be crap but its one step closer for mankind!
If I may ask, how did you get the AMP mission planner to recognize the head tracking via the usb? When I bring up the config page it shows a blank field for the joystick.
Cheers.
@John; I am using the rift to control an Axis Q6035-E PTZ IP camera.
This camera has absolute positioning so it is fairly easy to use with the rift.
It's a proof of concept for my boss who is showing it a a trade show in Canberra tomorrow.
So far I have it working well enough for him to demo it at the show.
The code I wrote for the rift sends PTZ commands via HTTP calls to the NVR (which I also wrote).
The code displays the live camera feed from the NVR in the rift headset and as you turn your head around the camera follows accordingly. The latency is not too bad but can certainly be improved upon.
So far it is good enough for my boss to demo at the show tomorrow.
Lloyd
@John; Actually I found some example code from the snes emulator that I could use.
I am displaying video from an IP camera in the Oculus and using the sensor to control the PTZ on the camera.
Would be interesting to see you code anyway if you can find it.
Thanks
Any chance of sharing the Oculus rift code?
That video doesn't seem to show what should normally be seen on a rift display.
Are you just doing head tracking or is the video display correct in the rift?
Thanks
Jeroen--The video latency is pretty low, I couldn't put a number to it, but i'd say it's not noticeable.
Mark--The Oculus resolution is quite low, but the new HD Oculus should hopefully look a bit better. I agree, stitching the images together in the air makes a lot of sense if you're interested in doing 3D. I think that makes a lot of sense for copter, but I don't think you'll see too much of a 3D effect from a plane at 100m. We'll definitely going to give it a shot though!
More video is in the works! We're currently testing out a few new features and working on reducing latency so that you can actually steer a plane or copter rather than just a gimbal.
So the laptop does the translation from RCA video to HMDI stereo video for the rift, how about the latency of the video signal?
I see a little latency from the mavlink connection. That will tend to give you motion-sickness. Head tracking needs to be on the 50 Hz RC link. I used a 3dr mpu3. I should put the code up. It works quite well. It handles all the ppm in/out while it's doing the DCM stuff, & a cli for setup/cal and some mode buttons. You need to hack into a 5 v ppm signal if your trainer port is full voltage as my Hitec is. It adds 3 extra channels to a cheap 6 ch Tx.
Great! That brings me to another idea! The days of 8 Bit arduino FC like apm 2.x are counted. But they could be re-used on the ground as drift free headtracker - antenna movement steering and supplying the possibility of differential GPS and airpressure (for planes). Cables on the ground could be reduced by telemetry (Blutooth etc)