PixHawk 3.1.4 - Help me analyse this crash - X8 craft - Discussions

The Copter:

Frame: Hobbyking X666 with double motor mounts

Motors: SunnySky X2212 980kv

ESC: Afro ESC 20A Simonk

Props: GemFan T-motor style 13x5.5’ Carbon

Battery: 2x Turnigy Nano-Tech 6000mAh [ 5700mAh measured from 4.2->3.6v]

FlightController: Pixhawk with Arducopter 3.1.4

Controller: 9XR

Reciever: FrSky D8R-II Plus + PPM Encoder

It was a never crashed build until today. It was moderately new, having between 4 to 5 hours of flight.

The Pilot:

I’ve started flying multis around an year ago, since Portugal has nice weather and I can fly at walking distance from home I’ve hundreds of flight hours already. I’m quiet used to build multis, I’ve built and flow quads, hexas, octos (I really need a tri!!).

Usually I fly with KK 2.0 boards (so used to Acro/Stabilized modes) or un-Autonomous boards of the sort, Pixhawk is the most advanced controller I’ve had the chance to fly. Due to my area of work/study I had no problems getting used to it and having it flying properly.

The Accident:

I flew around one of my usual spots for around 11minutes. Since I got the 30% battery warning and my voltage was already at ~10.6 I was heading home. The whole time I was flying the rig fpv. Neither the low voltage onboard alarm (set to 3.5v per cell) nor the Pixhawk voltage alarm (set to 10.6) made any sound.

What I felt was a loss of both power and attitude, I didn’t had fast enough reflexes to change from AltHold to Stab but I did give it full throttle. The craft went down either way. Luckily it felt in 1m high tall grass so not even a single prop broke, the only damage done was my home made aluminum landing gear and my GPS post, I ended up losing my FVP TX antenna top but who cares…

The video:

http://youtu.be/H_m5XpYw91I

(Jump to the last few seconds to see the fall or enjoy the whole trip)

Comments about the video:

- Loss of power (no motor sound)

- Pixhawk turn on sound

- Motor rotate for a little bit (?!?)

- Puff ground.

After Crash Photos:

Some thoughts:

Upon inspection craft looked okish. Only thing that cough my eye was that my XT60 parallel connector was loose from the ESC input, I don’t know if it either happened because of the fall or it mid flight (less likely... do XT60 come loose ???).

When does the Pixhawk reboot? I’ve it powered by both the power module and the ESC UBEC to have power redundancy. If one of this power sources fail (for example the ESC UBEC) does the board reboot (this would be really weird...) or it keeps on running from the power module power?

I couldn’t find anything wrong on the logs and that’s worrying me...

Logs here:

https://www.dropbox.com/sh/h8cwwbet29twdyk/AADf2S2yHU3EhovAa2wXVcU5a

I need help analyzing this crash since for the first time I don't know for sure what went wrong.

current consumption correctly! WARNING: You could try calibrating the module: https://www.youtube.com/watch? v=tEA0Or-1n18

"

When the voltage decreases normally during the flight the current increases for the same power output. At some point around the crash the current consumption was more than the batteries could supply (how many C's are the batteries?) and that caused a brownout on the supply lines to the pixhawk. I'll have a look at the logs, because what I sent you is the result of a very good (and improving) automated tool....

Actually, everyone calling each other stupid is just bad form overall when asking people for advise, granted neither side should be doing any name calling regardless.

This is how one of the Scorpion guys explained how a brushless ESC works. It's basically like a PWM signal and if you have a 20 A ESC, the current switches on/off with a 50% duty cycle for a 10 A draw. While the average over the period is 10 A, the ESC FETs are still hitting on/off at 20 A. Just because you limit the Pixhawk to only draw 80% of the theoretical 24 A max to get 20 A, it's pulling 24 A 80% of the cycle giving you an average draw of 20 A. You're still over the rating of the ESC.

Limiting the average current draw by using the end points on a transmitter or through the Pixhawk doesn't mean you can use ESCs that are under spec'd or are right on the edge. You're just asking for trouble with that.

The conservative rule of thumb for an ESC is that the steady state motor current draw at full throttle shouldn't exceed 80% of the ESC rating. That gives you a nice buffer so you don't overdraw the ESCs leading to possible failure, especially for those peak current cases that are transient. You didn't like the advice to over spec the ESC and said it was terrible advice for the hobby but that's your opinion based on your misunderstanding of things.

You said the motor was rated at 15 A peak. What prop is that rating for? If you're a experienced engineer, you would always look at spec sheets to make sure everything is going to work together properly in the system. Looking at the motor spec sheet I found online, that motor with an 11x4.5 prop on 3S supposedly pulls 17 A, 2 A more than the 15 A you stated for an unknown prop size. If you're sticking a 13" prop with a higher 5.5 pitch on a motor that pulls 17 A with an 11" prop, you're going to pull more than 20 A the ESC can provide. It almost seems you aren't putting your system together by looking at specs at all.

You give evidence that your measured maximum draw was 37 A, which is half since you're not measuring the entire setup, so the maximum is 74 A as you stated and that gives ~10 A per ESC. Can you say without a doubt that ALL the ESCs were only pulling < 10 A and there was no uneven draw among all the motors and that they were equally sharing the load? Unless you have a separate sensor on each ESC, you can't say definitively that each ESC was pulling less than 10 A and it is still entirely possible that one of them could have failed causing a potential brown out and reboot. Sure, redundant multirotor systems have redundancy them them in that if a prop goes bad or a motor quits working, you can still fly enough to land. If you have an electrical problem because the ESC goes out, all bets are off because you need a stable power source to the flight controller. That's where the UBEC advice is great and it seems you will do that and it should alleviate potential brownouts in the future so should something fail, the redundancy of your X8 will save the multi.

As for the motors all failing at the same time, what would you expect if the Pixhawk rebooted? Of course they should all stop at the same time unless you expect the ESCs to keep going on their own with no signal. If everything rebooted, your receiver will have gone down too and a failsafe won't help in that case since it's going through the Pixhawk.

While there's no definite conclusion on what might have happened, completely discounting the ESCs because you were theoretically pulling less than 10 A per ESC on motors that were WAY over-propped isn't very wise. You may not like to over-spec but you're better off being conservative in your designs and following spec sheets like an engineer instead of running them to the ragged edge. Good engineers don't run things on the edge unless there's no other option. They build in buffer for non-theoretical and unpredictable points of possible failure.

Match your gear properly and failures like this won't be quite so mysterious.

Replies

Oliver Volkmann May 23, 2014 at 10:46am

A client of mine had this exact same thing happen to him when switching from AUTO to Loiter also with firmware version 3.1.4 however it was with an APM 2.6 and not a Pixhawk. There was plenty of juice left in the lipos but the craft just proceeded to come down, ignoring the user input of full throttle. Luckily it came down the 40 meter pine tree, branch by branch and didn't crash into the road. Could this be a firmware issue? Attached is the Log File.

2014-05-17 10-35-24.log
- Pedro Palha > Oliver Volkmann January 12, 2015 at 5:15pm
  
  I have the same problem in my Hexa APM2.6 after finish the misson and switch to Loiter :-(

Luis Vale Gonçalves May 23, 2014 at 5:03am

I would start looking at this (power supply issues)

WARNING: If an APM Current Sensing Power Module is fitted then it is failing to measure

235.82 ~ 773.69       9

GPS-Sats

Eff(mA/m)

Luis Vale Gonçalves May 23, 2014 at 4:09am

No problem...

I'm also at OSX, but have a Virtual machine "nearby"

I suppose you're using APM Planner 2.09

Plug the USB to the PixHawk, On APM Planner go to terminal and use that connect button to connect to the PixHawk (on mine it works better on terminal if set to 57600)

you'll get a menu with one option of logs

go to logs

there just give the enable all command.

done

log analysed.pdf
Luis Vale Gonçalves May 23, 2014 at 3:55am

Olá Miguel

Hi

Just look at your logs parsed by this app APM Log flight analyzer and there seems to exist some inconsistencies on your power setup that might lead to the PixHawk reboot during flight. Also on the PixHawk console enable more logs manually.

just my 2c a partir de Cascais :)

On my Hexa, I can have 2 parallel batteries (5800 4S) or only fly with one, and only have a single power supply to the PixHawk via the 3DR Power Module.

My power lines are like: Battery ( or 2 Batteries with parallel plug) -> Power Module -> PixHawk

The High current side is:

Battery ( or 2 Batteries with parallel plug) -> Power Module -> Power Distribution Board ->ESC's

Luis
T3

healthyfatboy May 22, 2014 at 8:05pm

You mentioned losing all 8 motors at the same time and seemed to indicate that as a reason for saying that's why you know the ESCs didn't fail and could be excluded. If one failed and killed your electrical system temporarily making it reboot, it could still be a potential cause. You only said nothing was broken, not even a prop, but you didn't indicate that you spooled everything back up to see if everything else worked so we don't really know if everything is fine. If you have and it's all fine, them it may not be the problem, although one could have still overheated as you didn't say that you check all the motors and ESCs after to see if any were hot.

The way you had it set up with no backup UBEC, you can't expect it to keep flying if an ESC failure that stresses your electrical takes it down because of a reboot. Put in the proper redundancy and then you can keep flying with a bad ESC. That's a good lesson learned but you lucked out since nothing seemed to break! You were lucky.

So how exactly do you get +/-11.1 V out of a 11.1 V DC battery? Does it really go +/- VCC or is it rectified and only goes between 0 and VCC? Regardless of how the ESC works, which I'm obviously wrong about since electrical AC/DC motors and generators were my weak point in school and what I originally wrote was based off what I heard three years ago, an ESC rated at 20 A continuous is still not safe when the motor pulls more current than that. Even if the current limit is 80% of max because of what you did with the Pixhawk, you're potentially running on the ragged edge there.

Say it's a linear increase in current based on the size of the prop. A 10x4.5 pulls 13.2 A on that motor, a 11x4.5 pulls 17.2 A, if you extrapolate from there, a 12x4.5 would pull 21.2 A, a 13x4.5 would pull 25.2 A, and you're using a 13x5.5 which would increase from 25.2 A since it's a higher pitched prop. 80% of even the 13x4.5 would be 20 A and right on the edge. With the conservative system setup, you should be running at least a 30 A ESC. Sure, you might only be pulling 10 A per motor now but unless you're never going to increase the payload with anything like a gimbal and camera setup, those ESCs will run be on the edge. If you ever have to power out hard in a bad situation and you get above the 74 A peak you saw, you don't want to end up in a different bad situation throttling up and hitting the ESC limit.

Teetering on the edge of potential electrical failure when higher spec'd items are available isn't really innovation. If you have no other option because you're working on an experimental high performance, advanced research project that no one has done, sure, that's innovative. There are plenty of examples out there of people running ESCs right on the edge that failed while others lucked out. There's nothing innovative about that.
T3

healthyfatboy May 22, 2014 at 6:22pm

Actually, everyone calling each other stupid is just bad form overall when asking people for advise, granted neither side should be doing any name calling regardless.

This is how one of the Scorpion guys explained how a brushless ESC works. It's basically like a PWM signal and if you have a 20 A ESC, the current switches on/off with a 50% duty cycle for a 10 A draw. While the average over the period is 10 A, the ESC FETs are still hitting on/off at 20 A. Just because you limit the Pixhawk to only draw 80% of the theoretical 24 A max to get 20 A, it's pulling 24 A 80% of the cycle giving you an average draw of 20 A. You're still over the rating of the ESC.

Limiting the average current draw by using the end points on a transmitter or through the Pixhawk doesn't mean you can use ESCs that are under spec'd or are right on the edge. You're just asking for trouble with that.

The conservative rule of thumb for an ESC is that the steady state motor current draw at full throttle shouldn't exceed 80% of the ESC rating. That gives you a nice buffer so you don't overdraw the ESCs leading to possible failure, especially for those peak current cases that are transient. You didn't like the advice to over spec the ESC and said it was terrible advice for the hobby but that's your opinion based on your misunderstanding of things.

You said the motor was rated at 15 A peak. What prop is that rating for? If you're a experienced engineer, you would always look at spec sheets to make sure everything is going to work together properly in the system. Looking at the motor spec sheet I found online, that motor with an 11x4.5 prop on 3S supposedly pulls 17 A, 2 A more than the 15 A you stated for an unknown prop size. If you're sticking a 13" prop with a higher 5.5 pitch on a motor that pulls 17 A with an 11" prop, you're going to pull more than 20 A the ESC can provide. It almost seems you aren't putting your system together by looking at specs at all.

You give evidence that your measured maximum draw was 37 A, which is half since you're not measuring the entire setup, so the maximum is 74 A as you stated and that gives ~10 A per ESC. Can you say without a doubt that ALL the ESCs were only pulling < 10 A and there was no uneven draw among all the motors and that they were equally sharing the load? Unless you have a separate sensor on each ESC, you can't say definitively that each ESC was pulling less than 10 A and it is still entirely possible that one of them could have failed causing a potential brown out and reboot. Sure, redundant multirotor systems have redundancy them them in that if a prop goes bad or a motor quits working, you can still fly enough to land. If you have an electrical problem because the ESC goes out, all bets are off because you need a stable power source to the flight controller. That's where the UBEC advice is great and it seems you will do that and it should alleviate potential brownouts in the future so should something fail, the redundancy of your X8 will save the multi.

As for the motors all failing at the same time, what would you expect if the Pixhawk rebooted? Of course they should all stop at the same time unless you expect the ESCs to keep going on their own with no signal. If everything rebooted, your receiver will have gone down too and a failsafe won't help in that case since it's going through the Pixhawk.

While there's no definite conclusion on what might have happened, completely discounting the ESCs because you were theoretically pulling less than 10 A per ESC on motors that were WAY over-propped isn't very wise. You may not like to over-spec but you're better off being conservative in your designs and following spec sheets like an engineer instead of running them to the ragged edge. Good engineers don't run things on the edge unless there's no other option. They build in buffer for non-theoretical and unpredictable points of possible failure.

Match your gear properly and failures like this won't be quite so mysterious.
Emin Bu May 22, 2014 at 4:22pm

Anyway...enjoy your flights...I agree ESC might not be the reason but discussion just ended that way...

You should know for real answers post your problems on Forum where developers and technicians from 3DR live...

http://ardupilot.com/forum/viewforum.php?f=3

only problem is I cant find darn thing there....
Emin Bu May 22, 2014 at 4:17pm

I really do not understand how e calc can say 16% and you 50%???

btw all of my setups are coaxial and foldable and not very usual and ecalc was always very near
- Emin Bu > Emin Bu May 22, 2014 at 4:19pm

of 4

This reply was deleted.