Arducopter Erroneous Autopilot Causes Crash

All,

https://youtu.be/w5_E3-6p4s4

Had a flight yesterday with virtually no changes from the previous day (with successful flight) on the same flying field here in San Diego.  Notably this time I started receiving erroneous data on my OSD (mavlink messages); it was clearly causing some control issues.  I'm wondering if anyone has experienced this?  I'd really like to not hit the ground going 17mph again :).

Most of the reason for the crash is my fault in that I was flying LOS and didn't look at the FPV monitor and notice the bad data coming down until watching the video later.  I could definitely tell something was wrong while maneuvering around.  Here's the video (having a DVR is an AMAZING troubleshooting tool).  I've also included the log file, showing that at one point I try to activate an RTL, which essentially triggers a flyaway situation, at which point I switch back to STAB and try to recover.  Then erroneous data comes down (clearly visible 10000ft/min VS!! and crazy horizon line) and its too hard to recover, so I set it down.  Lucky I was flying in a big empty field!  

I've also got the logs: 2016-01-24%2012-49-42.zip.  I did an analysis but not really sure what to look for.  I'm thinking the Kalmann filter went nuts... any help?  Anyone seen this kind of behavior before?  Here's my build info:

Frame: QAV500 Frame w/540mm arms
Props: 12x5.5 Carbon Props
Motors: Tiger Motor MN3110-17 (700kv), 4mm shaft
Autopilot: Raspberry Pi 2 w/ Navio+ hat
ArduCopter Version: Copter V3.3
External Compass: HMC5883L https://www.adafruit.com/products/1746
GPS: Ublox M8N + Garmin GA25 MCX GPS Antenna
Battery: Turnigy MultiStar 4S 14.8V 16Ah
Internet Cam: Raspberry Pi Cam
FPV Cam: 1/3 Inch SONY CCD Board Camera 700TVL
ESCs: Emax 4 in 1 30A w/SimonK Firmware
Telemetry: RFD900+ Long Range Radio Modem
FPV Tx/Rx: Boscam Thunderbolt 2000mW 5.8GHz
FPV Antenna: Boscam 5.8GHz Cloud Spirit Circular Polarized
RC Tx: FrSky Taranis 9XD
RC Rx: Dragon Link V2, UHF

You need to be a member of diydrones to add comments!

Join diydrones

Email me when people reply –

Replies

  • All,

    So I took comments into consideration and did a couple tests.  I took the laptop out to the field and hooked up to telemetry via my RFD900+ serial modem.  First I tried the compass configuration noted above.  The main screen I was looking at was the EKF status screen (opens when you are in Mission Planner and you click on "EKF" on the HUD.  First thing I noticed was that when you rev the motors from 0 to 50% or so (while on the ground) you get EKF very Red for Compass.  So this definitely points to the compass as being part of the issue.  If anyone has had luck using this compass for APM, let me know.  Maybe its just too close to the battery wires (at least 200mm+ away though?).  It is not currently on a mast, but I felt that it was far enough away from DC noise that interference there wouldn't be an issue, while clearly it is if the EKF status is directly proportional to the throttle level!  

    The easiest thing to do was disconnect the external compass, use the internal compass, and calibrate.  So that's what I did.  I performed the same test, and while the EKF definitely was poorer the faster I increased the throttle, it was noticeably less than using the external compass.  So the sun setting, I take off and perform what I would consider a perfectly normal flight just using the internal compass.  However, upon post flight analysis I get:

    Log File C:/Program Files (x86)/Mission Planner/logs/QUADROTOR/1/2016-01-25 16-59-17.log
    Size (kb) 9707.171875
    No of lines 129429
    Duration 0:09:36
    Vehicletype ArduCopter
    Firmware Version V3.3
    Firmware Hash 340970fc
    Hardware Type
    Free Mem 0
    Skipped Lines 0

    Test: Autotune = NA -
    Test: Balance/Twist = NA -
    Test: Brownout = GOOD -
    Test: Compass = FAIL - Large change in mag_field (191.71%)
    Max mag field length (814.09) > recommended (550.00)

    Test: Dupe Log Data = GOOD -
    Test: Empty = GOOD -
    Test: Event/Failsafe = GOOD -
    Test: GPS = GOOD -
    Test: IMU Mismatch = NA -
    Test: Parameters = GOOD -
    Test: PM = NA -
    Test: Pitch/Roll = NA -
    Test: Thrust = NA -
    Test: VCC = FAIL - VCC below minimum of 4.6v (0.0v)

    The compass failed WORSE than before... but it always pointed in the correct direction and the video clearly shows no erroneous altitude measurements nor AHI issues like before.

    So the questions arise: 

    1) Has anyone had luck with the HMC5883L https://www.adafruit.com/products/1746 compass running APM Copter v3.3+?

    2) If so, where did you mount the compass?  Was there a certain orientation you used?  Did you experience any bad mag/ekf data at first and need to adjust it?

    I think that with the right placement and tuning I'll get my external compass to work properly and wont have a "Compass Fail" for variance post flight, but for now the Navio+ internal compass is doing the job until I can figure out what to do with the above external or replace it entirely.

    Also here are the logs from my "good" flight.  Also thanks again Craig/Kev for the log analyzer.  Here are its output:

    APM Log File Analiser v2.2.0.8

    Log FileName: A:\guest\Videos\GoPro\QuadCopter\Fiesta Island\20160125\Logs\2016-01-25 16-59-17.log
    Ardu Version: V3.3 Build: (340970fc)
    Ardu Type: APM:Copter
    Hardware: 259
    Frame Type: X
    No. Motors: 4

    Data Found in APM Log File:-
    IMU, GPS, CTUN, PM, CURR, MSG, MAG, CMD, ATT, MODE, STRT, EV, DU32, AHR2, EKF1, EKF2, EKF3, EKF4, UBX1, UBX2, RCIN, RCOU, BARO, RAD

    Data NOT Found in APM Log File:-
    NTUN, ATUN, ATDE, MOT, OF, INAV, D16, DU16, D32, DFLT, PID, CAM, ERR, GPS2, IMU2, IMU3, MAG2, MAG3, TERR, POWR, SIM

    FAILSAFE SETTINGS:
    Main Battery = Not Activated
    Ground Control = Activated, with always RTL
    Receiver = Activated at 975 PWM with always RTL
    GEO Fence = Not Activated


    PARAMETER ISSUES:-
    ARMING_CHECK = 0
    Warning: Pre-Arming Check is not Active (0=Off & 1=Fully On)


    TUNE = 0
    Information: CH6 Tuning is Disabled

    06/01/1980 - 000560: Mode Changed to Stabilize
    06/01/1980 - 000560: Mode Initialised at 0 0 Alt: 0 Spd:0
    06/01/1980 - 000561: Armed
    Limited Data Available - Mode Ignored

    06/01/1980 - 000562: Mode Changed to Stabilize
    06/01/1980 - 000562: Mode Initialised at 0 0 Alt: 0 Spd:0
    24/01/2016 00:00:17 - 000579: Take Off
    24/01/2016 00:00:17 - 000579: Mode Time Changed to 24/01/2016 00:00:17
    24/01/2016 00:00:17 - 000588: Simple Mode is Disabled
    24/01/2016 00:00:17 - 000588: APM Information: RC Pre-Arm checks have been completed.
    24/01/2016 00:00:17 - 000588: APM Information: All Pre-Arm checks have been completed.
    24/01/2016 00:00:17 - 000588: APM Information: USB is NOT Connected.
    24/01/2016 00:00:17 - 000588: APM Information: System Time has been set from the GPS.
    24/01/2016 00:00:17 - 000588: APM Information: Home is NOT Set
    24/01/2016 00:00:18 - 000821: Super Simple Mode is On
    24/01/2016 00:00:18 - 000821: APM Information: Home is Locked
    24/01/2016 00:00:19 - 000978: *** Update Program to include this Event *** : 28
    24/01/2016 00:00:19 - 001048: Auto Mode is waiting for Throttle.
    24/01/2016 00:00:29 - 003332: LAND_COMPLETE_MAYBE
    24/01/2016 00:00:30 - 003509: Landed
    Alt(m ~ ft) Spd(m/s ~ mph) Dist(km ~ mi) Launch(m ~ ft) GPS-Sats GPS-Hdop Eff(mA/min)
    Max 0 ~ 0 1 ~ 2 0.00 ~ 0.00 4.19 ~ 13.73 8 2.22 N/A
    Avg -2 ~ -7 0 ~ 0 N/A ~ N/A N/A ~ N/A 7 1.70 ~~~
    Min -6 ~ -20 0 ~ 0 N/A ~ N/A 0.00 ~ 0.00 6 1.23 N/A
    Stabilize Flight Time (Session)= 13 seconds, 0:13
    Stabilize Flight Time (Total)= 13 seconds, 0:13
    WARNING: Less than 9 Satellites was detected during this mode.

    24/01/2016 00:00:36 - 004853: Take Off
    24/01/2016 00:00:36 - 004860: Auto Mode is waiting for Throttle.
    24/01/2016 00:00:39 - 005396: SET_HOME
    24/01/2016 00:00:39 - 005397: Set Home as 32.7774900 -117.2217000 Alt:3.85
    24/01/2016 00:00:40 - 005732: *** Update Program to include this Event *** : 28
    24/01/2016 00:09:52 - 128890: LAND_COMPLETE_MAYBE
    24/01/2016 00:09:52 - 129070: Landed
    Alt(m ~ ft) Spd(m/s ~ mph) Dist(km ~ mi) Launch(m ~ ft) GPS-Sats GPS-Hdop Eff(mA/min)
    Max 91 ~ 298 17 ~ 38 3.04 ~ 1.89 202.73 ~ 665.11 18 1.8 N/A
    Avg 11 ~ 36 5 ~ 11 N/A ~ N/A N/A ~ N/A 16 0.81 554.57
    Min -6 ~ -20 0 ~ 0 N/A ~ N/A 2.70 ~ 8.85 9 0.71 N/A
    Stabilize Flight Time (Session)= 556 seconds, 9:16
    Stabilize Flight Time (Total)= 569 seconds, 9:29

    24/01/2016 00:09:54 - 129429: DisArmed

    Overall Flight Summary:
    Alt(m ~ ft) Spd(m/s ~ mph) Dist(km ~ mi) Launch(m ~ ft) GPS-Sats GPS-Hdop
    Max 91 ~ 298 17 ~ 38 3.04 ~ 1.89 202.73 ~ 665.11 18 2.22
    Avg 0 ~ 0 0 ~ 0 N/A ~ N/A N/A ~ N/A 15 0.83
    Min 0 ~ 0 0 ~ 0 N/A ~ N/A N/A ~ N/A 6 0.72
    WARNING: Less than 9 Satellites was detected during this mode.

    Power Summary:
    Battery(V) Vcc(V) Current(A) Used Cap(mAh) Eff(mA/mim)
    Max 16.15 0.00 56.86 5176 N/A
    Avg 0.00 0.00 32.71 N/A 545.80
    Min 13.60 0.00 0.62 N/A N/A
    Overall Flight Time = 569 seconds, 9:29

    *** Vibration Analysis has been removed from v2.2 onwards, you should now
    *** review the "Vibration Graph" instead to understand your levels of vibration.

    • have you run compass motor  calibration?

      http://copter.ardupilot.com/wiki/common-compass-setup-advanced/#com...

      • Ivan,

        I haven't tried that yet, but definitely sounds like I need to.  Have you had success using it?  If I have significant motor interference I would think it would be more necessary to move the compass away from the motors than to solve the problem by throwing a software setting fix at it, but maybe others have used it with good results?

  • you had tons of EKF variance errors.

    The barometer is most likely garbage, it jumps from 470 to 30m , and from 2m to 262m within one reading.

    Also, whenever you fly with an AHI , and see it's way off (like 50sec in the video), just land - it cannot go well.

    vibrations seem fine - I don't know what the cause to this problem is, but if barometer is behaving like that, there can be all kinds of problems with the IMU too (check for proper power to the sensor board/Raspberry, I don't mean voltage only check ripple too)

  • took a look at your file, you got some serious altimeter issues, also there seems to be a mechanical issue as roll/pitch dont follow the desired roll/pitch right before the crash

    I am not too knowledgeable with the Navio+, you might want to post on that thread thats around here somewhere

    2016-01-25_12-16-19.png

    2016-01-25_12-16-42.png

    2016-01-25_12-17-21.png

This reply was deleted.

Activity

DIY Robocars via Twitter
RT @TinkerGen_: "The Tinkergen MARK ($199) is my new favorite starter robocar. It’s got everything — computer vision, deep learning, sensor…
Monday
DIY Robocars via Twitter
Monday
DIY Robocars via Twitter
RT @roboton_io: Join our FREE Sumo Competition 🤖🏆 👉 https://roboton.io/ranking/vsc2020 #sumo #robot #edtech #competition #games4ed https://t.co/WOx…
Nov 16
DIY Drones via Twitter
First impressions of Tinkergen MARK robocar https://ift.tt/36IeZHc
Nov 16
DIY Robocars via Twitter
Our review of the @TinkerGen_ MARK robocar, which is the best on the market right now https://diyrobocars.com/2020/11/15/first-impressions-of-tinkergen-mark-robocar/ https://t.co/ENIlU5SfZ2
Nov 15
DIY Robocars via Twitter
RT @Ingmar_Stapel: I have now explained the OpenBot project in great detail on my blog with 12 articles step by step. I hope you enjoy read…
Nov 15
DIY Robocars via Twitter
RT @DAVGtech: This is a must attend. Click the link, follow link to read the story, sign up. #chaos2020 #digitalconnection #digitalworld ht…
Nov 15
DIY Robocars via Twitter
RT @a1k0n: Got a new chassis for outdoor races (hobbyking Quantum Vandal) but I totally didn't expect that it might cause problems for my g…
Nov 11
DIY Drones via Twitter
First impressions of the Intel OpenBot https://ift.tt/36qkVV4
Nov 10
DIY Robocars via Twitter
Nov 9
DIY Robocars via Twitter
Excellent use of cardboard instead of 3D printing! https://twitter.com/Ingmar_Stapel/status/1324960595318333441
Nov 7
DIY Robocars via Twitter
RT @chr1sa: We've got a record 50 teams competing in this month's @DIYRobocars @donkey_car virtual AI car race. Starting today at 10:00am…
Nov 7
DIY Robocars via Twitter
Nov 6
DIY Robocars via Twitter
RT @a1k0n: Car's view, using a fisheye camera. The ceiling light tracking algorithm gave me some ideas to improve ConeSLAM, and having grou…
Nov 5
DIY Robocars via Twitter
RT @a1k0n: To get ground truth I measured the rug, found the pixel coordinates of its corners, calibrated my phone camera with my standard…
Nov 5
DIY Robocars via Twitter
RT @a1k0n: @DIYRobocars is back in December, but outside. Time to reinvestigate ConeSLAM! I rigged up a quick and dirty ground-truth captur…
Nov 5
More…