Discussions

image processing for precision landing

Posted by Anthony on August 21, 2016 at 4:45pm

Hi all,

I’m looking for a straightforward solution to have a quad land on a small outdoor target area of 1ft x 1ft. Fully autonomous. Acceptable accuracy would be +/- 2 inches of the target center.

We know the GPS location of the landing site, which the quad will use as a waypoint to get to the area and start hovering at altitude. Then for the actual precision landing and descent, a visual marker could be used, ideally I wouldn’t need anything electronic on the ground to make things simple (unless it's really cheap as in less than $50 and leads to a much simpler solution). Also all computing systems must be onboard the quad.

I was thinking of combining a Pixhawk, a Raspberry Pi 3 and its V2 camera module (8MP) to do computer vision with OpenCV. I would like to keep things simple and if possible limit the image recognition to basically a color mask + find contours. First the Pixhawk will take the quad to the GPS location. Then in “locate & descend” mode the RPi3 would start scanning and feed the Pixhawk with (x,y,z) velocity vectors to get closer to target and land.

Will this be good enough? Any potential roadblocks that I should anticipate?

While searching on the forum I found a UC Berkeley project [1] that seems related although it’s 2 years old. I also came across the PX4Flow work but I’m hoping I can do without.

Thanks!

[1] http://diydrones.com/profiles/blog/show?id=705844%3ABlogPost%3A1789944

You need to be a member of diydrones to add comments!

Join diydrones

Tags: image processing, landing, opencv, pixhawk, precision landing, quad, raspberry, recognition, vtol
Email me when people reply –
Follow

Replies

  • Dave Jacob September 19, 2017 at 12:35pm

    Seems like there are several ways to do precision landings. I like the solution Fnoop came up with.

    Here is another precision landing using computer vision (OpenCv) with a phantom 4 pro and interpreting the video stream to stick a landing:

    https://vimeo.com/230643360

  • Fnoop June 5, 2017 at 1:39pm

    Just a late-on follow-up, I have a little project that probably does what you were looking for.  It does precision landing to within a couple of cm on a printed target using on-board computer.  No rangefinder is necessary.

    https://github.com/fnoop/vision_landing

  • Thomas Stone October 24, 2016 at 7:00am

    Hi Anthony, 

    I think I also saw your post in the Ardupilot.org forum, but I will reply here as well. 

    Yes, it seems like 3.4-RC6 requires lidar/sonar for precision landing. 

    However, you can test the precision landing feature w/out lidar/sonar by using the previous firmware. 

    Arducopter_PL_Quad_3.3.3-rc2.px4

    For commercial projects that require a high level of landing accuracy, lidar is very importan (the baro alt can drift by quite a bit). Currently, we strongly recommend the SF10/A. 

    Best,

    Thomas


    Anthony said:

    @ Thomas Stone : It seems like a rangefinder is required for precision landing in the ardupilot code.  I was going through https://irlock.readme.io/docs/getting-started and only see it as optional.

    Have you had much success landing without a rangefinder?

    Regarding the slowdown feature, are you just adjusting LAND_SPEED?

    Does the slowdown help a lot?

    Thanks!


    Thomas Stone said:

    @Anthony E

    Great questions. I can only answer them from the IR-LOCK perspective. They may or may not be applicable to other sensor systems. ... (I will probably add more content later. I have been super-busy lately)

    John A.B. is correct when he says that reliable and accurate tracking does not always translate into accurate copter control.

    Multiple control strategies have been tested published. Here are two of them. 

    (1) Precision Landing with Accuracy Management (blog post link)

    (2) Precision Landing Slowdown Feature (video link)

     

    ArduPilot Open Source Autopilot
    The most advanced open source autopilot for use by both professionals and hobbyist. Supports multi-copters, planes, rovers, boats, helicopters, ante…
  • Anthony October 22, 2016 at 7:49pm

    @ Thomas Stone : It seems like a rangefinder is required for precision landing in the ardupilot code.  I was going through https://irlock.readme.io/docs/getting-started and only see it as optional.

    Have you had much success landing without a rangefinder?

    Regarding the slowdown feature, are you just adjusting LAND_SPEED?

    Does the slowdown help a lot?

    Thanks!


    Thomas Stone said:

    @Anthony E

    Great questions. I can only answer them from the IR-LOCK perspective. They may or may not be applicable to other sensor systems. ... (I will probably add more content later. I have been super-busy lately)

    John A.B. is correct when he says that reliable and accurate tracking does not always translate into accurate copter control.

    Multiple control strategies have been tested published. Here are two of them. 

    (1) Precision Landing with Accuracy Management (blog post link)

    (2) Precision Landing Slowdown Feature (video link)

     

  • Patrick Poirier September 3, 2016 at 6:18am

    Meanwhile you can use SmartCamera (original version) and just have to change PYTHON PATH and get your home address in the pl_sim.py

    Have fun :-)

  • Patrick Poirier August 31, 2016 at 7:12pm

    Update: the problem seem to be with his version of pymavlink, I just desinstalled and reinstalled using standard  sudo pip and I can regain control of the modes.

    But now I got an error with the VN_logger on  precisionland.py ... I opened a new issue. If I disable the logger ,I can see the target moving on the simulator screen, but it crashed after a while because of the logger.  To be followed.

  • Patrick Poirier August 31, 2016 at 6:06pm

    Hello Antony,

    I just tried it and it seem that the simulator get stuck in STABILIZE mode without any ways to switch to other modes like GUIDED. This is why you get disarmed because it does not start the motors. You can put rc 3 1450 right after arm throttle and you will see the copter climbing.

    I opened an issue on Daniel github about this problem.  Stay tuned :-)

  • Anthony August 31, 2016 at 12:20pm

    @Patrick Poirier - yes I tried, it doesn't help.

  • Patrick Poirier August 31, 2016 at 4:30am

    Hello Antony

    Have you applied throttle: rc 3 1500

  • Anthony August 31, 2016 at 2:11am

    @Patrick Poirier - I tried running the precision landing SITL code you pointed at.  Setting up the environment is OK, but I seem to be running into the following issue: the AP will disarm after 10s no matter what I do and nothing happens.

    Here's a copy of my sim_vehicle.sh terminal:

    STABILIZE> param show ARMING_CHECK

    STABILIZE> ARMING_CHECK     0.000000

    STABILIZE> arm throttle; mode GUIDED; takeoff 20; api start /Users/ant/visnav/PrecisionLand.py

    STABILIZE> setting mode

    (4, 0)

    Take Off started

    Connecting to vehicle...

    Vehicle connected!

    Running Precision_Land

    Using simulator

    Waiting for intial home lock: Requires armed status....

    Got valid intial home location

    STABILIZE> 

    And in the console window:

    Running script /Users/ant/.mavinit.scr

    -> module load droneapi.module.api

    fence breach

    online system 1

    Mode STABILIZE

    APM: APM:Copter V3.3 (d6053245)

    APM: Frame: QUAD

    Flight battery 90 percent

    GPS lock at 0 meters

    APM: ARMING MOTORS

    Got MAVLink msg: COMMAND_ACK {command : 176, result : 3}

    Got MAVLink msg: COMMAND_ACK {command : 22, result : 4}

    APM: Initialising APM...

    Got MAVLink msg: COMMAND_ACK {command : 400, result : 0}

    ARMED

    Requesting 0 waypoints t=Wed Aug 31 02:06:59 2016 now=Wed Aug 31 02:06:59 2016

    APM: DISARMING MOTORS

    DISARMED

    The AP automatically disarms after 10 seconds and prints the last two lines.  Any idea what is happening?

This reply was deleted.

Activity