This is a discussion re the bad Accel, Gyro and Baro values that we're seeing with ArduCopter-3.1. The increase in the SPI bus speed from 500khz to 8Mhz has exposed a hardware problem on some boards. That hardware problem is that the 3.3V regulator has been blown so all sensors are running at 5V instead of the intended 3.3V.
How have these regulators been burnt out?
- Attaching a radio receiver or MinimOSD to the APM while the APM is only powered through the USB (see video below)
- Some clone boards seem to come from the factory with blown regulators. 3DR boards might also come with blown regulators although they do a specific check of the regulator as part of the regular QA process.
- It is not (as far as we know) actually caused by the AC3.1 software itself, it just exposes the problem. You could prove this to yourself by checking the 3.3V regulator (see video above) before and after the upgrade.
How can we fix the regulator?
Option #1: If it's a new board (so that it's less likely you burned it out yourself) you could report the problem to the retailer that sold you the board and ask for an replacement. If it's 3DR it's called an "RMA".
Option #2: if you're handy with a soldering iron you can replace the regulator yourself. On the APM2.5.2 (and higher) boards it's not that difficult. On the APM2.5 it's far more difficult.
For APM2.5.2 : TPS79133DBVR
For APM 2.5: MIC5219-3.3YML TR
How can I stop it from happening again?
Do not connect any devices such as a radio receiver, MinimOSD, GPS, etc while the APM is powered especially while powered only through the USB cable.
Attaching a 100uF capacitor across any of the APM's radio input's 5V and GND pins will stop the regulator from being blown by plugging in a receiver. video here!
There are very few reports of regulators being blown twice and no reports of it ever failing in flight.
Below are some graphs of the types of values that we are seeing on these boards.
Replies
Well, good luck and bad luck. At least you got that regulator which you can handle with a regular soldering iron. The older would require more specialized tools or using a different regulator patched onto the board.
The top left pin next to the two brown capacitors in the picture is where you should have 3.3V
Use a big blob of fresh solder on the pins, heat it up good by alternating sides quick enough but not extreme long and it should slide off very easily. Do not use force and do not apply too much heat otherwise you risk ripping the pads of the board. You will see its easier than it looks.
Another method if you are handy with a raiser blade or little knife to score and cut the legs off first (careful to not slip off). Do a couple softer strokes not with force at once. When you got all 5 legs cut and the boddy is gone you can simply un-solder each. Do not use a wire cutter - you risk ripping the pads off as well that way.
Oz -
They DO have a Schottky diode mod; however, its effectiveness on this design has been disproven. Most modern linear regulators, like the LM1117 and supposedly this one as well, incorporate such protection internally. The problem appears to be a device-specific issue related to the TPS79133; it self-destructs if the voltage potential between IN & OUT turns more than a few millivolts negative. Clearly, momentary surge current loading from abruptly charging a Low-ESR cap distributed across a complex circuit can make EXACTLY this condition appear.
The ceramic cap mod appears to be an effective buffer; remember, this board design doesn't have any buffer caps at the power rails of any of the plug-in headers. This is, in my opinion, a serious oversight.
Actually - this reminds me...
DEVs - Have we performed any similar testing with the APM 2.5 boards and the MIC5219-3.3 regulator?
If I recall my device history, that part SHOULD be a descendant of the LM1117 family, only repackaged for SMD. I'm curious if we have confirmed that this part has a similar failure history in our application.
Tim -
First - just to clarify:
You WERE able to reproduce this failure with the APM plugged into a battery?
Was this via a BEC at the OUPUTS rail or with the APM POWER MODULE?
Was JP1 jumpered or open?
The essence of the mod is right at the top of every page in this thread - a 100uf CERAMIC capacitor from GND to the (+5V) rail of the ANALOG inputs header; A2 was suggested as it is rarely used and would be a good place for a plug-in solution.
My contention is that in order to be properly effective, the cap should be soldered in place; as close to the A2 end of the ANALOG inputs header appears to be a very good, low-resistance location. Also worth considering would be across the (+5V) and (GND) pin header right in front of the 3.3V regulator.
The reason this capacitor was chosen was that being ceramic, it will be very low ESR (My 60uF bank measured at less than 0.01Ω ESR) which will sharply buffer very brief transient surges in current, yet be small enough capacitance to prevent damage to supply circuitry when it initially charges. So far, it has proven to do just that.
My more aggressive approach was to put buffer caps at the ANALOG, INPUT, OUTPUT, and I2C plug-in headers, not just the one we're having a problem with today. The UART1/GPS header appears to be pretty well protected by the cap I have at the INPUT header.
mnem
Over-volted, under-protected. The story of my life.
My sneaking suspicion is that that regulator does not have the problem. We never had this problem before the 2.5.2 came out. Now, one might say "but the problem only cropped up with 3.1 code, which came after the 2.5.2 was prevalent". Yes, but remember, this failure can also cause the 3.3V supply to go to zero volts, which can cause the gyros to not work at all, and also can cause the EEPROM to not work at all.
ie: On the one 2.5.2 that I have with a fried reg, I discovered it before 3.1 came out. That is because for me, it manifests itself at 0V, causing no dataflash error.
I have never seen that on a 2.5.0.
I believe what's happening here is that, this hot-plugging thing is causing an instability in the TPS regulator. Some design quirk, just the right capacitance and impedance in just the right places, that when you give the system a good poke, it goes unstable.
I have hot plugged receivers on pre-production and older APM2.x boards more times then I care to think about during the PPM encoder development. No problem there. I am fairly sure this is unique to the new regulator.
You have some good points, so an applications engineer (or paper document) states that if Vin is a few milli volts lower than Vout we have self destruct, kinda hard to believe, since Micrel is a top leader in voltage regulators and this quote from the data sheet "Supply Input Voltage (VIN) –20V to +20V".
Quote ( I hate doing this, the forum is sub-standard to RCG);
"the cap should be soldered in place; as close to the A2 end of the ANALOG inputs header appears to be a very good, low-resistance location." Why not solder in parallel to the regulators input caps, can't get any better than that for low esr?
I have a 4400uf bulk cap soldered (not the high resistance servo plug) soldered to the Rx input rail since day one, has it helped my APM from damage, maybe, maybe not.
Oz -
First, NOT the MIC5219-3.3, that was only on the APM Rev 2.5. I was asking if we had done testing on that setup.
The APM 2.5.2 uses a (TI) TPS79133. It shows a very low -V tolerance in the datasheet; definitely not hard to overlook that spec, and truthfully, not something we SHOULD have to worry about anyways; we aren't deliberately applying a negative voltage. I believe this is just a case of "The role of failure in design". You design it, test to the best of your ability, but it isn't until it's in the hands of the public at large that you get to see where "The proof is in the pudding".
Philip was looking for a simple device that even those without advanced soldering skillz could implement to provide some measure of protection; A2 is a good choice for that. My suggested alternate location of the (+5V) and (GND) headers right in front of the reg actually share the same trace as the solder lands for the reg. ;) HOWEVER... that location will require slight modding of the APM case, or adding leads to the capacitor (which may decrease effectiveness) so it can be tucked away further inside. This also might induce a vibration issue with the gyros/accels.
Tim -
If you're powering from the Power Module, the filter caps in that should be providing some protection... hmmmm...
*Pokes around his APM board with a meter for a moment*
AH- HAH!!!
Yeah, those caps in the APM Power Module aren't going to help a lot; they're on the other side of polarity protection diode D1 right by JP1. No way that's going to pass enough current fast enough to allow them to do any appreciable amount of buffering on the other 5V rail. DEFINITELY need some localized stiffening RIGHT THERE.
Ummm... Philip, you may want to rethink your notion that the APM Power Module offers any protection against this failure; Tim's experiment gives evidence that suggests otherwise.
PK
Because Philip wanted the fix to be something that was easily workable for people with varying skillsets to be able to accomplish at home. Soldering a large through-hole package to the power rail is easy. Soldering a tiny SMD cap in parallel with another SMD cap is not so easy.
Yes this was with Lipo and USB connected.
(However I suspect that APM still draws from USB even if Lipo connected as I tried to use my USB keyboard but got the error that the hub did not have enough power to supply the device. Not 100% familiar with the in's and outs of the USB protocol so it could have simply sent a canned message that I need x.xx amps of current even if it is not drawing it.)
APM Power Module
JP1 is open (5v not connected).
Cheers
Tim
FYI I also finished upgraded my MinimOSD Extra to the latest 750 pre-release and looks like everything is back up and running.
No issues so far.
-
5
-
6
-
7
-
8
-
9
of 56 Next