It's been a long while since I posted my last blog post about this project and there have been some progress though veeery slow. The last two months I have very little time to work on this.

Anyway, thought I should at least post the latest vid.

This was filmed a couple of weeks ago, unfortunately I can't record on my laptop when using the Oculus so I had to place a camera on a tripod in front of it :)

As you can see in the video there is a significant lag (about 400ms) on the video, I have to work on that as well as the h264 encoding.

Thanks to Patric Duffys excellent posts about "How to build a High-Definition FPV UAV using a Rasperry PI with HD camera, using a high speed WiFi link" here and here I found a really nice way to transfer HD video down to my GCS which can then show it on my Oculus Rift. On the picture below you can see it mounted on my quad.

The setup I'm using here consists of:

* Odroid-w (Raspberry PI)

* Raspicam clone with M12 lens mount and a 160 fov lens

* Fatshark servo gimbal

* Ubiquiti Wisp-station M5 for 5.8 GHz wifi in air

* Ubiquiti Rocket-M5 for 5.8GHz wifi on ground

Here is the raspicam, gimbal and Odroid-w with docking board for ethernet.

Here is the Ubiquiti Wisp-station M5 mounted on top of my battery plate.

On the Raspberry I'm using FFMPEG to stream 1280x960 h264 video to the GCS and also in my application FFMPEG is used. Right now I have around 400 ms latency on the video link. Though using gstreamer I can get it down to about 100ms,

Its been a while since I posted part 1 on this project. Since then I have been working on the HUD as well as building myself a test copter. Oh, and I've also finally got my 3dr pixhawk,

There is nothing much to show today except the HUD. I did not use the camera here to get a better view of what it looks like.

As you can see it is very like the HUD in mission planner. I have looked a lot at the Mission Planner code when creating this to get it as similar as possible without overdoing it at this stage.

The basic idea here is that the HUD should always be straight ahead so when I tilt my head the HUD stays in its original position. This way I always know what is straight and hopefully also get a better sense of immersion. If I turn off the head tracking the gimbal is put in neutral position and the HUD will also lock its position in the center of my view.

At this stage there's really not much more to say about it. It is drawn using Direct2d to a texture that is then placed on a polygon and rendered in Direct3d. There is still a lot of work to do on it but the main parts are in place.

Hopefully I will be able to do some test flights this week if the weather allows, will post an update then.

. 

This is the first post in what I hope to be a series of posts on exploring the possibilities in using the Oculus Rift alongside a UAV. The plan in the long run is to create a GCS for the Oculus Rift or other HMDs that gives you a feeling of actually sitting up in the air being able to look around by just turning your head. Either as a pilot, copilot or passenger. Add to that the ability to control your UAV and interact with an augmented reality like maps for planning missions or guided mode and overlays giving information about your surrondings etc using for example a Sixense STEM or a Razer Hydra (Sixense TrueMotion) system. 

Anyway, here are the first trembling steps.

For some reason the embedded link doesent seem to work so here is a direct link

This is a really dodgy setup and the code right now is a mess. But here you can see my laptop connected to a PixHawk via mavlink running arducopter 3.2rc9 (yeah its a clone, but there wasn't any originals available in Sweden/europe when I needed one). Received MavLink data is displayed in the Oculus, though for now I only have GPS status, sats, hdop and attitude roll displayed in the Oculus so there is a lot of work to do. To know which way is straight ahead a green cross is displayed in oculus. The application then takes tracking data (roll, pitch, yaw) from the Oculus and assembles MavLink messages at a 50Hz loop that are being sent to the PixHawk to control the gimbal. The gimbal is controlled with an Alexmos 8bit SBGC. As you can see the gimbal is not very well tuned. The cables from the webcams are really stiff making it very hard to tune. Also a smaller gimbal with less inertia on the roll and yaw axis will probably be more suited to the task but this is what I had at home. 

Image is delivered from a Microsoft LifeCam studio webcam captured with direct show. Only one camera is used since my laptop refuses to capture two at the same time. Though on my workstation I can use both cameras along with the Oculus which is really cool. I can also use a Hauppahuge USB Live2 capture card to be able to use a analog camera with a wireless link. The image is really crappy though. My plan here has been to use two webcameras and a Odroid and stream a stereo video over 4G/LTE network using h264 low latency encoding. So far though I have not had much time to look into this. Maybe a DTV link using a Hides DVB-T dongle or a Aires Pro 5.8GHz link would better.

Right now I will continue to work on cleaning up the code and the HUD and hopefully I'll manage to do a test in the air soon.

Since I couldn't find anyone who done this before I thought I should write a guide on how to do it.

As I understand it some of the Arduflyers delivered by RCTimer in 2012 and early 2013 are missing the bootloader on the ppm encoder chip, the Atmega 32u2. This makes it impossible to upgrade the firmware on the ppm encoder using USB and Flip as instructed on the wiki. You will know if the bootloader is missing if it doesn't say Atmega 2560 dfu in device manager after you have reset the 32u2 chip.

I was about to upgrade the firmware for the PPM encoder when I noticed that one my APM boards where missing the bootloader. While I was searching to buy a programmer to use to restore the bootloader I discovered that it was possible to use an Arduino as an avrisp programmer together with avrdude. I decided to give this a shot and as it turned out it worked really well. So if you are missing the bootloader on your 32u2 chip and have an Arduino laying around you don't have to buy a programmer to restore the bootloader. I used an Arduino UNO but it should work with any Arduino like the Duemilanove, Mega, Micro etc.

Download and install winavr from Sourceforge.
It will install to C:\WinAVR-20100110 and avrdude.exe will be found under the bin catalog.
Backup your avrdude.conf file and replace it with this one avrdude.conf.
The configuration file will be found under the same catalog as the avrdude.exe file.

Download the Arduino-usbdfu.hex bootloader file or build it. I downloaded it from the 2.9 branch. Place the file in the winavr bin directory.
If you don't already have the Arduino IDE installed, download and install it.
Load the ArduinoISP sketch to your Arduino.
Wire up the Arduino as instructed in the sketch. If you have a 6 pin ICSP-cable you can use the ICSP connector on the Arduino. I also added 3 leds to pins 7, 8, and 9 with a 1k resistor, for heartbeat, error and programming.
Add a 10uF capacitor between reset and GND after you have downloaded the sketch to your arduino. This is to disable the autoreset function.

Ok, your arduino is now an ISP programmer and you are now ready to flash your 32u2 chip on your APM or your 2560 chip if you would like. The manual for avrdude can be found here. I strongly suggest you read it to understand what your doing before trying this.

Connect your arduino and your APM to the computer and then connect your ICSP cable to the APM. Readup on Jabrams instructions where applicable here.

Read the fuses a couple of times to make sure you have a good connection and get the same results back every time, especially the device signature. 

Then flash the Arduino-usbdfu.hex to the 32u2 chip using this command:

Note that my arduino is on com6 and you have to change that part to whatever port yours is on.

After you have flashed the dfu hex, recycled power and use filp to flash the latest ArduPPM.hex to the 32u2 following the instructions on the wiki.