Replies to This Discussion

Permalink Reply by automatik on March 24, 2010 at 9:09pm

Hi Daniel,
image processing is tough but interesting discipline in itself. To get you started look at some of the books on the subject

image processing books on Amazon

Also take a look at OpenCV. It's an open source image processing library - take a look at examples they have. Good companion book for OpenCV can be found here

• ▶ Reply

Permalink Reply by Daniel Nugent on March 24, 2010 at 9:21pm

Ok thanks. I read the discription and that book sounds great! Do you personally have any experiance with image processing? Do you know if most colleges(like the u of m) have classes on that kind of programming. Because I would love to have a professor accesible, but I was unsure if image processing classes are commonly taught.

• ▶ Reply

Permalink Reply by Diarmuid Wrenne on March 25, 2010 at 1:33am

Try OpenCV ˇhttp://opencv.willowgarage.com/wiki/

AVRcam

Look for optical flow, blob tracking, that sort of thing. Automatic tracking an object the size of a car at distance is non trivial unless you tell it it's a car first. Or else use optical flow to see what points of interest are moving over the background.

Cheers

Diarmuid

• ▶ Reply

Permalink Reply by Diarmuid Wrenne on March 25, 2010 at 3:02am

Hi Daniel,

Try http://code.google.com/p/aforge/ . c# and microsoft friendly. Good for prototyping and for developing algorithms and allows for a more interactive development process.

Look at PIL for python. Does not do what aforge does (hough lines, etc) but is multiplatform.

Imagemagick might be a starting point.

Image processing consists of going through the image, pixel by pixel and doing something based on the location and properties of the pixel. So blob tracking. Take a picture image1, take another picture Image 2 . Create a new blank image3 Transform both pictures to gray scale. Subtract the images. Take pixel x,y in Image1 and compare it to pixel x,y in image 2 . Now determine if it is the same or similar shade. If it is , make Image3 pixelx,y blank. If not make image3 pixel x,y some value (255,255,255) . At the end you have a new picture with the difference between the 2 . Now do that 30 times per seconds and you have the motion.
Now you need to write something to identify a shape or a corner, etc and track that. Different ball game if the camera is moving too.

Aforge is a very good tool for learning how this stuff works.

Look at http://letsmakerobots.com/node/13354 as the starting point for image processing with micros.

http://www.davidchatting.com/arduinoeyeshield for use of video with arduino.

The thing with the arduino is that with only one serial it is hard to interface with a video and still see the output. The new Mega will make a big difference, but likely an ST32 or similiar would be needed (AVRCam used a Mega)

Good Luck

Diarmuid

• ▶ Reply

Permalink Reply by Martin Seven on March 25, 2010 at 3:47am

> Different ball game if the camera is moving too.

There's no such thing as a non-moving camera when talking UAVs and aircraft in general, so sadly, naive blob tracking is not gonna work. Ever.

• ▶ Reply

Permalink Reply by Daniel Nugent on March 25, 2010 at 3:12pm

Ok thanks. I don't think I'm going to be using the Arduino to do the processing. Just because it would be easier to use the ground station because it has the speed. I'm definitely thinking about buying that openCV book though. This is how the program would work: the object(moving or stationary) would be selected manually and then it would be tracked by the pan and tilt on the camera and the plane itself. I get how it maybe a problem with the plane moving.I thought about what you said with picture1 and then picture 2 and then it makes a picture 3. what if it took 3 pictures instead of 2 and the third image would compensate for the moving background. I haven't really thought it through that much because I'm still kind of confused on how it all works but i was wondering if that may work( if you get what I'm saying).
example:
pic1- first pic
pic2 second pic
pic3 find the pixel change
pic4 take another pic
pic 5 it will compare pic4 with the pic3 and see the change and see that a group of pixels changed more than others. (I Don't know, I'm just hoping i will spark an idea in someone.)
Explanation
What it would do is first see how fast the pixels are moving across the screen. There are going to be two different speeds for a moving object, so there will be to different pixel speeds. It will obviously go through an algorithm and calculate the actual speeds of the two moving objects(car and background) on screen. Next it will compare the objects actual speeds to the velocity of the plane. and the object that has a similar speed as the plane is the background. So then the plane will then know to track the 2nd object. And for a stationary object the pixel speeds will be the same so the actual speed will be the same. the computer will then know infer that the object is stationary and run a different set of commands. After explaining all that I'm not even sure if you would need a third image(maybe for accuracy) but right I just confused myself so I'm going to need to think about this so I can explain it better.

sorry this was a really confusing post. Just try to understand it as best as you can, because im really bad at expressing my ideas.

• ▶ Reply

Permalink Reply by Diarmuid Wrenne on March 25, 2010 at 3:43pm

I used aforge today to do some simple motion capture at my desk. It does a great job of finding blobs. Tracking then is a bit of a black art in that if you have the centroid of a blob in frame 1 and then you have a set of blobs in frame 2, you need to determine whcih of the frame 2 bobs is the same as your first one.

Really, I would suggest you head out , take a video of a car moving ( find a tall building) and then run it through some image manipulation code and see what you can see.

Then, try a simple algorithm of keeping the car in the center of the frame so that the code outputs a motion order to you the operator, . Then when that is done, head back up with the webcam on the computer and see if it generates the right set of commands when you try to track a car. When that works you can see if you can interface with a servo system to move the camera.

Cheers

Diarmuid

• ▶ Reply

Permalink Reply by automatik on March 25, 2010 at 2:54pm

@Daniel
"Do you personally have any experiance with image processing?" - yes. Not for UAV (yet) but different robotics platforms ( moving and stationary) I used bunch of different tools including openCV, matlab, LabVIEW IMAQ, etc.

"Do you know if most colleges(like the u of m) have classes on that kind of programming." - :)) I don't know. But someone in CS dept., or Engineering ( ME, aerospace) should have some knowledge. Just ask around.

Image processing is fun but not trivial....

• ▶ Reply

ModeratorPermalink Reply by Sgt Ric on March 25, 2010 at 3:09pm

I know the Kestral UAS works with object tracking... They state... "while tracking the target, our Localization algorithms provide the user with the moving target’s heading, velocity, and estimated GPS location.
Human size targets can be tracked as well."

I know the miltary has target GPS, velocity calculations, but then again, they have everything!

Here's a video off the Procerusuav site.

• ▶ Reply

Permalink Reply by Sami Finnila on March 25, 2010 at 3:44pm

@Daniel Nugent:

One of the best ways to learn about something new is to experiment with an existing system: Roborealm is a "Robotic Machine Vision Software" that I've used previously to create some UGVs with simple image processing capabilities with the aid of a base station. The software has a simple GUI so you really don't need to know the first thing about processing images on the programming level. This might be helpful for you too in understanding the basic concepts of image processing; all you need is a web camera.

Sami F.

• ▶ Reply

Permalink Reply by Joel Ryan on March 25, 2010 at 4:10pm

I know nothing of the image processing, but when it comes to the math of finding velocity and gps, I know a little bit. You would need to find a way to estimate the linear distance to the object you are tracking, and you can use this, along with the angle of declination from the plane to find the distance between your target and the spot on the ground below your plane.

To better visualize what I'm saying, you might want to draw the following:
A point labeled "A" which will be the target
A point 315 degrees from A (directly down and to the right) labeled B which is the point on the ground directly below your plane
A line between A and B labeled ground distance.
A line running horizontally from A to the right
A line running vertically from B going upwards.
A point at the intersection of those lines labeled C

Now Imagine your plane is directly over point B on your paper which we will call D. This creates an oblique triangular pyramid. You need to know the altitude of your plane (line BD), the distance from the plane to your target (line AD) and the angle of declination to the target. (angle ADB) This will allow you to solve for one face of the pyramid. Next, you need to know the compass heading of your plane, and the angle of the camera in relation to the nose of the plane. You can use these values to find the value of angle ABC. Now you can use more trig to solve for the triangle ABC. Now you can use the haversine formula (don't use the law of spherical cosines) to get GPS coordinates for all of these. Then you do it again, when things have moved and you compare the changes.

If you have any questions or want to know more, just pm me. I'm also working on formulas for aiming cameras and antennas at planes from ground stations, and at targets from planes using GPS. If you want to know about that I'd be happy to chat about it.

• ▶ Reply

Permalink Reply by Daniel Nugent on March 26, 2010 at 3:17pm

Oh wow! OK I don't know where to start. Thanks for all the replies. OK. Yeah, I got my main ideas from the Procerusuav website, they have some cool stuff. I have already figured out a method to do all the calculations. Its almost the same exact method as Joel said. Its just a matter of learning the programming environment and perfecting the algorithms(which I'm looking forward to because I love math). Thanks for all the great sources. I was really unsure where to start and how to learn but from what everyone is saying it sounds like the programming isn't going to be the hard part, its just going to be alot of testing and tweaking.
Yeah my Dad actually works at the U of M with the professors so I'm hoping I can get some good connections and help from them. I really don't have any questions right now because I'm still trying to process and research all this great info everyone just gave me. I'll reply back soon with questions and hopefully more knowledge....

Well thanks again for everyone's help,
Daniel

• ▶ Reply

• ‹ Previous
• 1
• 2
• Next ›
• Page