Hi there,
I'm writing to report a very likely bug affecting at least firmwares 3.1 rc8 to 3.1.2 (), at least for hex X frames. The bug manifests when RTL is the last step of an AUTO mission and RTL return altitude is different from the current altitude. The drone crashes consistently in these conditions. Additionally, the drone crashes from LOITER after a non-deterministic time - sometimes is quick (tens of seconds), sometimes it can loiter for minutes. In both cases the symptoms are the same: the drone goes from hover holding output (motors ~ 1600) to shutting down 1-2 motors in 100-200ms, and then shutting down the others in another 100-200ms. The way it shots down the motors is very specific: it reduces throttle to 12xy where xy is the same for all motors. In the attached logs for example, after LOITER the motor commands are as follows:
RCOU, 487396, 1695, 1645, 1672, 1667, 1443, 1874, 32767, 32767
RCOU, 487495, 1523, 1597, 1517, 1602, 1231, 1824, 32767, 32767
RCOU, 487595, 1398, 1421, 1368, 1451, 1231, 1581, 32767, 32767
RCOU, 487696, 1231, 1231, 1231, 1231, 1231, 1231, 32767, 32767
RCOU, 487796, 1231, 1231, 1231, 1231, 1231, 1231, 32767, 32767
RCOU, 487897, 1231, 1231, 1231, 1231, 1231, 1231, 32767, 32767
RCOU, 487996, 1231, 1231, 1231, 1231, 1231, 1231, 32767, 32767
RCOU, 488096, 1329, 1444, 1401, 1372, 1231, 1545, 32767, 32767
RCOU, 488195, 1392, 1380, 1545, 1231, 1299, 1474, 32767, 32767
RCOU, 488297, 1404, 1369, 1501, 1271, 1231, 1545, 32767, 32767
Obviously 1231 is not sufficient to sustain flight - the fact that all motors have the exact same value (1231) reinforces the likelihood of this being the output of a bug.
The RTL example is similar - the RTL command is executed right after the first RCOU:
RCOU, 742799, 1697, 1603, 1542, 1756, 1571, 1728, 32767, 32767
RCOU, 742898, 1241, 1241, 1241, 1241, 1241, 1241, 32767, 32767
RCOU, 742799, 1697, 1603, 1542, 1756, 1571, 1728, 32767, 32767
RCOU, 742898, 1241, 1241, 1241, 1241, 1241, 1241, 32767, 32767
This time the motors are all 1241 - interestingly different from 1231, but still very close, not nearly enough for flight and all equal in all motors.
In case you wonder if there is something with the drone, both bugs have been reproduced in two different drones several times: the loiter crash three times and the RTL crash twice.
In case it helps, in addition to the dataflash logs I have telemetry logs from mission planner as well as videos, but I assume that the dataflash logs are sufficient, as they had enabled practically everything relevant (I think) - certainly they had ATT, CTUN, NTUN, IMU, RCOU, GPS, and RCIN.
If this is a known bug, please drop a line here, if I can help in tracking it down, let me know.
Best,
Mihai
Replies
No, of course it wasn't in the navigation code. A flip could never come from the navigation code, only a flyaway.
Such as this?
QGC WPL 110
0 1 0 16 0 0 0 0 -35.362881 149.165222 582.000000 1
1 0 3 22 0.000000 0.000000 0.000000 0.000000 -35.362881 149.165222 20.000000 1
2 0 3 16 0.000000 3.000000 0.000000 0.000000 -35.364652 149.163501 20.000000 1
3 0 3 16 0.000000 3.000000 0.000000 0.000000 -35.365361 149.163501 20.000000 1
4 0 3 20 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 50.000000 1
Doesn't cause a flip, but RTL doesn't respect the altitude on the RTL command and instead RTLs at RTL_ALT.
The absolute worst thing that nav code should be able to do is cause the copter to tilt to ANGLE_MAX.
Are you available for a skype conversation at some point?
I had a similar thing happen to me about two weeks ago and did not think anything of it at the time, however I now wonder if it is the same problem described here. I was testing RTL with my Hex. I am using APM 2.5 with an external compass/GPS mounted on a mast, with the latest firmware (3.1.x - flashed the APM first couple days of March). I flew out about 150ft at an altitude of about 20 meters in loiter mode, then flipped to RTL. The Hex climbed to 40 meters as preset, turned, and proceeded to return to the home point...so far so good. Upon arrival, it again turned to face the original heading at T/O, but it then suddenly banked to the rear right and then proceeded to fly off at what appeared to be a 45 degree angle back in the direction that it just came from. Luckily I had enough altitude and was able to switch back into stabilize mode, regain control, and land safely. I am unable to check my logs at the moment, so can not say for sure that this was the case. I have not flown that hex since the incident. The flight just prior to this one was an "Autotune" flight which completed successful and without incident. I will check the logs when I can and report back. I even got video footage of the incident that I could post to my youtube channel if anyone is interested in viewing it...
Hello
so some seems a little too drastic told what I read here.
The problem is not solved when the Qud on day 5 x crashes.
it would be very nice if one of the developers something to say about this problem-
ПОДТВЕРЖДАЮ КАЖДОЕ СЛОВО!
This isn't just a hex issue. It affects quads too. I thought it had something to do with the geo fence and GPS glitches, but had the quad decide when it was in loiter mode to pitch forward, bank to the right and power up the motors. I was able to switch to stabilize and the quad flew fine. I have had the same issue on my other quad too. I have no idea why this isn't getting any developer attention as it is far from an isolated issue. Many people have posted about crashes with the exact same symptoms.
jg wrote: This isn't just a hex issue. It affects quads too. I thought it had something to do with the geo fence and GPS glitches, but had the quad decide when it was in loiter mode to pitch forward, bank to the right and power up the motors. I was able to switch to stabilize and the quad flew fine. I have had the same issue on my other quad too. I have no idea why this isn't getting any developer attention as it is far from an isolated issue. Many people have posted about crashes with the exact same symptoms.
________________
That sounds different, like it could indeed be GPS multipathing or some other GPS issue. My understanding is that an enabled fence (or loiter, etc.) means the quad is actively using GPS to decide where it is and what to do. If a GPS error suddenly makes the quad think it's breached the fence it may fly in the direction it thinks home is, or where it thinks the last commanded loiter position is, for example. Which is one of a number of reasons why we keep hearing and saying that one needs to be able to take over manual control at any time.
Yes, by now I'm convinced it's a navigation code bug, so no related to the frame. I guess most people fly without navigation (RC control)? That code runs really nice - I was impressed today on how well a hex was doing in 30mph+ wind gusts. Alternatively, judging by the number of bug fixes in navigation code from 3.0.1 to 3.1.2 they know they have a problem but couldn't just put their finger on it. I looked at the code again today and it's... tough. It's not properly commented or documented (I guess with a proper documentation the comments would be sufficient).
Mihai
I used to fly a bunch of autonomous flights both short and some longer ones that took about 7 minutes and out to a mile distance using the "continue in auto" option for lost signal. I forgot to switch the fence back one day and the quad came back as it was supposed to. Then when it was close to home it did the pitch forward, bank right and full throttle deal. It was close enough that I could see it and switched modes and resumed control in stabilize. The odd part is, the APM is not losing control. It will hold that exact angle the whole time. I have noticed though that it does manifest more when below the RTL altitude. Whether is is coincidence or connected somehow I haven't a clue. Today when it happened I had a GPS HDOP of 1.17 so I know it wasn't a glitch. It did it within seconds of switching to loiter. So instead I just took off under the fpv goggles and flew in stabilize for four batteries. I agree the stabilize code works excellent. When working the other modes work great too. But the random almost freeze up is odd. There does not seem to be an indicator it is going to happen, and it only affects modes that rely on the GPS. Which probably explains why I kept getting odd bank spikes when I tried flying in Drift Mode. I am pretty sure Drift uses GPS too. So maybe it isn't Nav related and is really related to something in the GPS coding? And possibly connected to the HDOP issues some others are having where the HDOP refuses to go down, or goes down extremely slowly. I use two different GPS modules on my quads so I am pretty sure it isn't an antenna or satellite issue.
I have noticed quite a bit of unexplained LOITER and RTL issues when i was trying to solve a plane 2.78 RTL / LOITER issue.
I have no idea if they are related to the bug you describe, but if it helps the developers, here are some unexplained events tied to LOITER..
My plane issue...
http://diydrones.com/forum/topics/apm-2-5-channel-1-servo-twitching...
Another plane issue
http://diydrones.com/forum/topics/apm-2-5-locks-servos-and-spirals-...
Unanswered Copter issues (seem to be loiter related)
http://ardupilot.com/forum/viewtopic.php?f=25&t=7037
http://ardupilot.com/forum/viewtopic.php?f=25&t=6546
http://ardupilot.com/forum/viewtopic.php?f=25&t=6455
-
6
-
7
-
8
-
9
-
10
of 12 Next