Erroneous Mavlink Messages Cause Crash/Flyaway

All,

https://youtu.be/w5_E3-6p4s4

Had a flight yesterday with virtually no changes from the previous day (with successful flight) on the same flying field here in San Diego.  Notably this time I started receiving erroneous data on my OSD (mavlink messages); it was clearly causing some control issues.  I'm wondering if anyone has experienced this?  I'd really like to not hit the ground going 17mph again :).

Most of the reason for the crash is my fault in that I was flying LOS and didn't look at the FPV monitor and notice the bad data coming down until watching the video later.  I could definitely tell something was wrong while maneuvering around.  Here's the video (having a DVR is an AMAZING troubleshooting tool).  I've also included the log file, showing that at one point I try to activate an RTL, which essentially triggers a flyaway situation, at which point I switch back to STAB and try to recover.  Then erroneous data comes down (clearly visible 10000ft/min VS!! and crazy horizon line) and its too hard to recover, so I set it down.  Lucky I was flying in a big empty field!  

I've also got the logs: 2016-01-24%2012-49-42.zip.  I did an analysis but not really sure what to look for.  I'm thinking the Kalmann filter went nuts... any help?  Anyone seen this kind of behavior before?  Here's my build info:

Frame: QAV500 Frame w/540mm arms
Props: 12x5.5 Carbon Props
Motors: Tiger Motor MN3110-17 (700kv), 4mm shaft
Autopilot: Raspberry Pi 2 w/ Navio+ hat
ArduCopter Version: Copter V3.3
External Compass: HMC5883L https://www.adafruit.com/products/1746
GPS: Ublox M8N + Garmin GA25 MCX GPS Antenna
Battery: Turnigy MultiStar 4S 14.8V 16Ah
Internet Cam: Raspberry Pi Cam
FPV Cam: 1/3 Inch SONY CCD Board Camera 700TVL
ESCs: Emax 4 in 1 30A w/SimonK Firmware
Telemetry: RFD900+ Long Range Radio Modem
FPV Tx/Rx: Boscam Thunderbolt 2000mW 5.8GHz
FPV Antenna: Boscam 5.8GHz Cloud Spirit Circular Polarized
RC Tx: FrSky Taranis 9XD
RC Rx: Dragon Link V2, UHF

You need to be a member of diydrones to add comments!

Join diydrones

Email me when people reply –

Replies

  • you have fairly high vibs

    you have low VCC

    Test: Compass = FAIL - Large change in mag_field (198.77%)

    Test: VCC = FAIL - VCC below minimum of 4.6v (0.0v)

    the logs also show a TON of EKF errors before you even took off.

    24/01/2016 00:00:43 - 000926: APM Information: Home is Locked
    24/01/2016 00:00:46 - 001622: Auto Mode is waiting for Throttle.
    24/01/2016 00:00:51 - 002567: SET_HOME
    24/01/2016 00:02:26 - 024638: EKF/ Inertial Nav Check ERROR: Bad Variance.
    24/01/2016 00:02:52 - 030581: EKF/ Inertial Nav Check ERROR: Bad Variance cleared.
    24/01/2016 00:02:53 - 030859: EKF/ Inertial Nav Check ERROR: Bad Variance.
    24/01/2016 00:03:45 - 042845: EKF/ Inertial Nav Check ERROR: Bad Variance cleared.
    24/01/2016 00:03:46 - 043169: EKF/ Inertial Nav Check ERROR: Bad Variance.
    24/01/2016 00:03:49 - 043728: EKF/ Inertial Nav Check ERROR: Bad Variance cleared.
    24/01/2016 00:03:50 - 044098: EKF/ Inertial Nav Check ERROR: Bad Variance.
    24/01/2016 00:03:55 - 045166: EKF/ Inertial Nav Check ERROR: Bad Variance cleared.
    24/01/2016 00:03:57 - 045647: EKF/ Inertial Nav Check ERROR: Bad Variance.
    24/01/2016 00:04:26 - 052292: EKF/ Inertial Nav Check ERROR: Bad Variance cleared.
    24/01/2016 00:04:28 - 052786: EKF/ Inertial Nav Check ERROR: Bad Variance.
    24/01/2016 00:04:55 - 059060: EKF/ Inertial Nav Check ERROR: Bad Variance cleared.
    24/01/2016 00:04:57 - 059637: EKF/ Inertial Nav Check ERROR: Bad Variance.
    24/01/2016 00:05:26 - 066122: Landed

    And then again

    24/01/2016 00:06:30 - 067044: Take Off
    24/01/2016 00:06:30 - 067051: Auto Mode is waiting for Throttle.
    24/01/2016 00:07:33 - 081525: EKF/ Inertial Nav Check ERROR: Bad Variance.
    24/01/2016 00:07:47 - 084628: EKF/ Inertial Nav Check ERROR: Bad Variance cleared.
    24/01/2016 00:07:48 - 084907: EKF/ Inertial Nav Check ERROR: Bad Variance.
    24/01/2016 00:07:53 - 085979: EKF/ Inertial Nav Check ERROR: Bad Variance cleared.

    and then again

    24/01/2016 00:08:28 - 094200: Mode Changed to Stabilize
    24/01/2016 00:08:28 - 094200: Mode Initialised at 32.7775258 -117.2212037 Alt: 21.46 Spd:4.85
    24/01/2016 00:08:29 - 094279: EKF/ Inertial Nav Check ERROR: Bad Variance.
    24/01/2016 00:08:31 - 094830: EKF/ Inertial Nav Check ERROR: Bad Variance cleared.
    24/01/2016 00:08:34 - 095564: EKF/ Inertial Nav Check ERROR: Bad Variance.
    24/01/2016 00:08:59 - 101127: Auto Mode is waiting for Throttle.

    Also this looks a tad worrysome

    06/01/1980 - 000579: Mode Initialised at 0 0 Alt: 0 Spd:0
    06/01/1980 - 000595: RADIO ERROR: Late Frame Detected, i.e. Longer than 2 Seconds.

    Really looks like something is causing your mag to go crazy and your VCC to drop to dangerous levels.

    • Hey Craig thanks for taking a look at that!

      I did mount the external compass a little closer to some electronics, but I calibrated it before the flight and it looked fine?  Clearly I did that with the motors and props off, anyone have a good test to try to determine if its the motors that are causing the EKF compass issues (besides flying :P)?  

      Also the VCC error seems like a fluke because I wouldn't have been able to fly at all at 0V :), and my video shows that the voltage in Mavlink is pretty decent for a 16Ah 4S battery at near full charge.  It never seems to drop to 0V.

      • Developer

        try doing a compsssmot. Even if you through away the results it will give you an idea of the amount of interference your compass is getting from the power electronics

        see http://copter.ardupilot.com/wiki/common-compass-setup-advanced/#com...

        Just moving the mag an inch further way could cure the issue. 

        • Thanks Bill.  I'm in the process of converting the "roll" bar into a more of a useful antenna mast.  Will post pics when its done, but I'm going to stick the compass up there in hopes that will cure the issue in addition to doing the compassmot.

      • Craig, how did you get the errors spelled out in a list wise fashion as above?  I can only see them in a graphical format that is rather cluttered using the "review a log" button in Mission Planner?  

        • Go here. Kev made an awesome little program for this. I tend to be a little partial to it since I had a small hand in making it :)

          http://www.rcgroups.com/forums/showthread.php?t=2151318

          • Thanks Craig!  I kinda get the feeling people are viewing the thread because the word "crash" is in it and there's a video!   

            Really appreciate the help and comments.  Hopefully I figure it out and report back.  

This reply was deleted.

Activity

DIY Robocars via Twitter
RT @gclue_akira: 柏の葉で走行させてるjetracerの中身 #instantNeRF #jetracer https://t.co/giVvuE4hP7
Jul 4
DIY Robocars via Twitter
Cool web-based self-driving simulator. Click save when the AI does the right thing https://github.com/pncsoares/self-driving-car
Jul 4
DIY Robocars via Twitter
RT @donkey_car: Human-scale Donkey Car! Hope this makes it to a @diyrobocars race https://www.youtube.com/watch?v=ZMaf031U8jg
Jun 25
DIY Robocars via Twitter
Jun 25
DIY Robocars via Twitter
Jun 16
DIY Robocars via Twitter
RT @GrantEMoe: I won my first @diyrobocars @donkey_car virtual race! Many thanks to @chr1sa @EllerbachMaxime @tawnkramer and everyone who m…
Jun 13
DIY Robocars via Twitter
RT @gclue_akira: JetRacerで自動走行したコースを、InstantNeRFで再構築。データセットは別々に収集 #jetracer #instantNeRT https://t.co/T8zjg3MFyO
Jun 13
DIY Robocars via Twitter
RT @SmallpixelCar: SPC 3.0 Now the motor also works. This car is doable. I just need to design a deck to mount my compute and sensors. http…
Jun 13
DIY Robocars via Twitter
RT @SmallpixelCar: My new car SPC 3.0. https://t.co/CKtkZOxeNQ
Jun 7
DIY Robocars via Twitter
RT @SmallpixelCar: High speed at @diyrobocars thanks @EdwardM26321707 for sharing the video https://t.co/o4317Y2U1S
Jun 7
DIY Robocars via Twitter
RT @SmallpixelCar: Today at @RAMS_RC_Club for @diyrobocars. Used @emlid RTK GPS and @adafruit @BoschGlobal IMU. Lap time 28s https://t.co/R…
May 28
DIY Robocars via Twitter
May 15
DIY Robocars via Twitter
May 14
DIY Robocars via Twitter
May 13
DIY Robocars via Twitter
RT @f1tenth: Say hi to our newest #F1TENTH creation for @ieee_ras_icra next week in Philly. It’s going to be huge! 😎 🔥 @AutowareFdn @PennEn…
May 13
DIY Robocars via Twitter
May 11
More…