LOITER and RTL crashing bug

Hi there,

I'm writing to report a very likely bug affecting at least firmwares 3.1 rc8 to 3.1.2 (), at least for hex X frames. The bug manifests when RTL is the last step of an AUTO mission and RTL return altitude is different from the current altitude. The drone crashes consistently in these conditions. Additionally, the drone crashes from LOITER after a non-deterministic time - sometimes is quick (tens of seconds), sometimes it can loiter for minutes. In both cases the symptoms are the same: the drone goes from hover holding output (motors ~ 1600) to shutting down 1-2 motors in 100-200ms, and then shutting down the others in another 100-200ms.  The way it shots down the motors is very specific: it reduces throttle to 12xy where xy is the same for all motors. In the attached logs for example, after LOITER the motor commands are as follows:

RCOU, 487396, 1695, 1645, 1672, 1667, 1443, 1874, 32767, 32767
RCOU, 487495, 1523, 1597, 1517, 1602, 1231, 1824, 32767, 32767
RCOU, 487595, 1398, 1421, 1368, 1451, 1231, 1581, 32767, 32767
RCOU, 487696, 1231, 1231, 1231, 1231, 1231, 1231, 32767, 32767
RCOU, 487796, 1231, 1231, 1231, 1231, 1231, 1231, 32767, 32767
RCOU, 487897, 1231, 1231, 1231, 1231, 1231, 1231, 32767, 32767
RCOU, 487996, 1231, 1231, 1231, 1231, 1231, 1231, 32767, 32767
RCOU, 488096, 1329, 1444, 1401, 1372, 1231, 1545, 32767, 32767
RCOU, 488195, 1392, 1380, 1545, 1231, 1299, 1474, 32767, 32767
RCOU, 488297, 1404, 1369, 1501, 1271, 1231, 1545, 32767, 32767

Obviously 1231 is not sufficient to sustain flight - the fact that all motors have the exact same value (1231) reinforces the likelihood of this being the output of a bug.

The RTL example is similar - the RTL command is executed right after the first RCOU:

RCOU, 742799, 1697, 1603, 1542, 1756, 1571, 1728, 32767, 32767
RCOU, 742898, 1241, 1241, 1241, 1241, 1241, 1241, 32767, 32767
RCOU, 742799, 1697, 1603, 1542, 1756, 1571, 1728, 32767, 32767
RCOU, 742898, 1241, 1241, 1241, 1241, 1241, 1241, 32767, 32767

This time the motors are all 1241 - interestingly different from 1231, but still very close, not nearly enough for flight and all equal in all motors.

In case you wonder if there is something with the drone, both bugs have been reproduced in two different drones several times: the loiter crash three times and the RTL crash twice.

In case it helps, in addition to the dataflash logs I have telemetry logs from mission planner as well as videos, but I assume that the dataflash logs are sufficient, as they had enabled practically everything relevant (I think) - certainly they had ATT, CTUN, NTUN, IMU, RCOU, GPS, and RCIN.

If this is a known bug, please drop a line here, if I can help in tracking it down, let me know.

Best,

Mihai

loiterCrash2.log

rtlFromAutoCrash.log

You need to be a member of diydrones to add comments!

Join diydrones

Email me when people reply –

Replies

      • Developer

        Hi Mihai,

        Yeh, I came into this thread a little late when John asked me to have a look. This particular bug is a hard to find one because it is disguised behind other problems (or simply large command inputs) that you may not ever notice otherwise.

        There are three factors that must come together to cause this bug to be seen.

        1. High throttle command

        2. High roll output to motors

        3. High pitch output to motors

        and it helps if yaw is high too.

        The problem is all these things tend to happen if you are experiencing any sort of motor problem or cg offset. The final problem looks very similar to a lost prop or motor unless you can see the motor logging and generally that is turned off.

        So thanks for being persistent with your posts, you have made a major contribution to the code with your find!!!

        • So can you give the rest of us some guidance please?  Is this a problem for quads as well as hexes? In any event, is there a published fix yet and how do we access/implement it? Thanks!!

    • Leonard, can we add another safety feature then? If it detect a faulty motor it force to land instead. What do you think?

      • @Jay - there was no motor failure: all my motors (on many drones) run fine - the posts above come from 3 different drones, all of which flew again and crashed again and flew some more. If a motor would fail, you'd see the APM try to rev it up - instead the APM commands them to stop - first a few then all at the same low value. Not idle, not stopped, but low enough to fall (e.g., 1231 or 1241).

        Best,

        M.

    • I had a crash yesterday. The copter was out about 100 yards and I hit the RTL on the transmitter right after it hit the 1st of 2 waypoints. The copter tilted to the left hard and went down. Please if someone could take a look at my attached long and see if I may have been the cause of this or is it something in the software. 

      Thanks

      2014-04-05 20-15-13_62774.log

      2014-04-05 20-15-13_62773.log

  • If your motors stop completely at 12 hundred and something, could that be the cause? That is a quite high throttle setting (each motor output throttle I mean) for just cutting.
    Regards, Soren
    • They don't stop completely - I miss-spoke, but they are nowhere close to being able to sustain flight - since they are all equal, they don't provide stability either.

      Motors are fine.

      M.

  • I don't know enough to say this is a global problem (plane / copter), but experienced plane users are experiencing LOITER related navigation issues that sound similar to the issues discussed here. The primary plane developers have either been busy or quiet, because there are very few responses to go from.

    More plane issues
    http://ardupilot.com/forum/viewtopic.php?f=96&t=6899

    http://ardupilot.com/forum/viewtopic.php?f=96&t=6850
  • RTL is not on your mode switch. You are switching to stabilize, lowering your throttle to zero, and watching the copter fall out of the sky.

    In addition, something is going on with your motors in the loiter crash log. Particularly, motor 5 is very low while motor 6 is very high. Do you have a weight on arm 6? Are any motors swapped?

    Please post future logs to the forums, not here.

    • Sorry, this is not correct. Roll diverges from desired roll and pitch diverges from desired pitch before the switch to stabilize is commanded. I see now that you meant that the copter is crashing during Auto, not during RTL.

      You are correct that the motor channels all go to approximately THR_MIN. I'll keep examining this and try to reproduce it. However, note that typically these events are caused by motor issues. I've seen many users insist that there's a bug in AC, only to eventually find that their motors are failing intermittently.

This reply was deleted.

Activity