Camera Targeting with MatrixPilot and the UAV DevBoard

Matrix Pilot Camera Targeting: Tree Test - Software Stabilized

Strategy: Have MatrixPilot / UAV DevBoard (UDB) point camera roughly in the right direction.

Take out the jitter movement using software stabilization.

The above flight was flown on a day with 10mph of wind, with a lot of turbulence (I included the landing in the video to show the turbulence).

1. The UDB is calculating its orientation 40 times / second using a 16 bit Direction Cosine Matrix

2. I use se Bill Premerlani's "High Bandwidth Dead Reckoning", which means MatrixPilot knows it's

absolute position 40 times / second, by integrating the accelerometers. The accelerometer positions are

corrected by allowing for the GPS delay and some of the GPS dynamics (GPS info arrives at least 1

seconds after the real event).

3. The camera code computes the target location from the above 40 times / second.

The main issue at the moment is that I'm using a camera with progressive scan. This causes each frame of the image to be distorted when the camera rotates (accelerates) in a new direction.

Pitch Servo resolution is 0.2 degrees which translates into moving the picture 7 pixels. Ideally it would be at least 1/10th of that, e.g. 0.02 degrees.

Photos of the build of this project are here.
Main Wiki for MatrixPilot is here.

For reference the flight path, in autonomous mode, is shown below.

An interesting point about the above waypoint route, is that the plane is skewed into wind on the top leg of the waypoint route (W4 to W5). (see picture above). The plane has estimated the wind vector on each curved section of flight, is allowing for the wind in it's flight path, but even more importantly is calibrating the yaw gyro .... allowing for the fact that the plane's heading is not the same as the received GPS heading (it is using the GPS heading as the source data for yaw gyro calibration, but also allows for the wind to derive true heading). So I'm postulating that, without the wind estimation, navigation, and yaw gyro correction (wind corrected), the Camera would actually be looking way too far to the right of the above picture during W4 to W5. (because the yaw gyro would less accurately aligned with the true heading of the plane).

Having said all that, looking at the video again, I can now see that there is still a small residual amount by which the camera is looking slightly left of target (to the right of the picture) during W4 to W5. I know that the wind estimation is correct, because it flies a straight line between W4 to W5, and Cross Tracking was turned off for this flight (so the plane estimated the wind exactly correctly). So there may still be some inaccuracies lurking in that aspect of the code.

You need to be a member of diydrones to add comments!

Join diydrones

Comments

Jack Crossfire October 21, 2010 at 10:55am

The rolling shutter artifacts make you wish we could go back to CCD cameras.
Carlito Oliveira October 21, 2010 at 10:40am

very nice man! now I understand why your video had that high quality, ur not transmitting it real time right?
I made my self a plane very similar to the twinstar but it has only one motor. I'm also courious about what's the maximum flight time u have already achieved?

Thanks again.
Developer

Pete Hollands October 21, 2010 at 10:24am

@carlito. Here is a picture. And a demo on the ground of the plane's camera stabilization.
Carlito Oliveira October 21, 2010 at 9:50am

Hello Peter,

Do u mind sharing with us some pictures of your plane? I'm curious about where u mounted your camera.
Thanks.
T3

William Premerlani October 21, 2010 at 9:37am

Hi Peter,

Congratulations! I know that you put a lot of work into this, the camera targeting code is more intricate than the core direction cosine matrix algorithm. A real work of art.

Also, thanks for providing a genuine application for the "dead-reckoning" algorithm.

Best regards,
Bill
Developer

Pete Hollands October 21, 2010 at 9:11am

@Loti, Yes, MatrixPilot sends a new command to the servos 40 times / second. Of course, the servo command may be the same as the previous servo command if the target is still in the same place relative to the orientation of the plane. So, in practice, the servos may not be trying to move to new locations 40 times / second.

I use Adam Barrow's web based waypoint editor to plan the flight in Google Maps. It is working well for me now. (simple but very effective). I set the target altitude of the camera target to be approximately ground level at take off. The tree was in a slight dip, so in reality, the target was probably about 5 meters up the tree from ground level.
Developer

Pete Hollands October 21, 2010 at 8:59am

An interesting point about the above waypoint route, is that the plane is skewed into wind on the top leg of the waypoint route (W4 to W5). (see picture above). The plane has estimated the wind vector on each curved section of flight, is allowing for the wind in it's flight path, but even more importantly is calibrating the yaw gyro .... allowing for the fact that the plane's heading is not the same as the received GPS heading (it is using the GPS heading as the source data for yaw gyro calibration, but also allows for the wind to derive true heading). So I'm postulating that, without the wind estimation, navigation, and yaw gyro correction (wind corrected), the Camera would actually be looking way too far to the right of the above picture during W4 to W5. (because the yaw gyro would less accurately aligned with the true heading of the plane).

Having said all that, looking at the video again, I can now see that there is still a small residual amount by which the camera is looking slightly left of target (to the right of the picture) during W4 to W5. I know that the wind estimation is correct, because it flies a straight line between W4 to W5, and Cross Tracking was turned off for this flight (so the plane estimated the wind exactly correctly). So there may still be some inaccuracies lurking in that aspect of the code.
Loti October 21, 2010 at 8:50am

Thank you for sharing.
When you say The camera code computes the target location from the above 40 times / second. Do you mena that you are paning/tiling the camera 40 times / second ? that's a lot !!
You give the target before starting the fly ? what altitude for the target ?
Bye
Developer

Pete Hollands October 21, 2010 at 8:45am

Loti, It's a python script that I wrote called flan.pyw. It primarily reads SERIAL_UDB telemetry from MatrixPilot, and then spits out a spreadsheet CSV file, and also a Google Earth KMZ file. It plots the aircraft orientation, the wind vectors (not shown above), and the route of the air plane, along with the waypoints positioned in 3D in the sky so you can see whether the plane if flying accurately. All the data is contained within Google Earth's time animation slide bars. If flying with a magnetometer, flan.pyw will show the earth's magnetic vector throughout the flight. The colour coding of the routes shows which waypoint the plane desires to reach next. Souce code here.
Loti October 21, 2010 at 8:34am

Hi pete,
nice work, how do you generate the above picture for the flight path ?
Would be nice to have camera targeting in APM
Loti

of 2

This reply was deleted.