Developer

This is a discussion re the bad Accel, Gyro and Baro values that we're seeing with ArduCopter-3.1.  The increase in the SPI bus speed from 500khz to 8Mhz has exposed a hardware problem on some boards.  That hardware problem is that the 3.3V regulator has been blown so all sensors are running at 5V instead of the intended 3.3V.

 

How have these regulators been burnt out?

  • Attaching a radio receiver or MinimOSD to the APM while the APM is only powered through the USB (see video below)

  • Some clone boards seem to come from the factory with blown regulators.  3DR boards might also come with blown regulators although they do a specific check of the regulator as part of the regular QA process.
  • It is not (as far as we know) actually caused by the AC3.1 software itself, it just exposes the problem.  You could prove this to yourself by checking the 3.3V regulator (see video above) before and after the upgrade.

 

How can we fix the regulator?

Option #1:  If it's a new board (so that it's less likely you burned it out yourself) you could report the problem to the retailer that sold you the board and ask for an replacement.  If it's 3DR it's called an "RMA".

Option #2: if you're handy with a soldering iron you can replace the regulator yourself.  On the APM2.5.2 (and higher) boards it's not that difficult.  On the APM2.5 it's far more difficult.

3691073788?profile=originalFor APM2.5.2 : TPS79133DBVR

For APM 2.5: MIC5219-3.3YML TR

How can I stop it from happening again?

Do not connect any devices such as a radio receiver, MinimOSD, GPS, etc while the APM is powered especially while powered only through the USB cable.

Attaching a 100uF capacitor across any of the APM's radio input's 5V and GND pins will stop the regulator from being blown by plugging in a receiver.  video here!

There are very few reports of regulators being blown twice and no reports of it ever failing in flight.

 

Below are some graphs of the types of values that we are seeing on these boards.

3691073724?profile=original

You need to be a member of diydrones to add comments!

Join diydrones

Replies are closed for this discussion.

Replies

        • Well, good luck and bad luck. At least you got that regulator which you can handle with a regular soldering iron. The older would require more specialized tools or using a different regulator patched onto the board.

          The top left pin next to the two brown capacitors in the picture is where you should have 3.3V

          Use a big blob of fresh solder on the pins, heat it up good by alternating sides quick enough but not extreme long and it should slide off very easily. Do not use force and do not apply too much heat otherwise you risk ripping the pads of the board. You will see its easier than it looks.

          Another method if you are handy with a raiser blade or little knife to score and cut the legs off first (careful to not slip off). Do a couple softer strokes not with force at once. When you got all 5 legs cut and the boddy is gone you can simply un-solder each. Do not use a wire cutter - you risk ripping the pads off as well that way.

          3702752206?profile=original

  • Oz -

    They DO have a Schottky diode mod; however, its effectiveness on this design has been disproven. Most modern linear regulators, like the LM1117 and supposedly this one as well, incorporate such protection internally. The problem appears to be a device-specific issue related to the TPS79133; it self-destructs if the voltage potential between IN & OUT turns more than a few millivolts negative. Clearly, momentary surge current loading from abruptly charging a Low-ESR cap distributed across a complex circuit can make EXACTLY this condition appear.

    The ceramic cap mod appears to be an effective buffer; remember, this board design doesn't have any buffer caps at the power rails of any of the plug-in headers. This is, in my opinion, a serious oversight.

    Actually - this reminds me...

    DEVs - Have we performed any similar testing with the APM 2.5 boards and the MIC5219-3.3 regulator?

    If I recall my device history, that part SHOULD be a descendant of the LM1117 family, only repackaged for SMD. I'm curious if we have confirmed that this part has a similar failure history in our application.

    Tim -

    First - just to clarify:

    You WERE able to reproduce this failure with the APM plugged into a battery?

    Was this via a BEC at the OUPUTS rail or with the APM POWER MODULE?

    Was JP1 jumpered or open?

    The essence of the mod is right at the top of every page in this thread - a 100uf CERAMIC capacitor from GND to the (+5V) rail of the ANALOG inputs header; A2 was suggested as it is rarely used and would be a good place for a plug-in solution.

    My contention is that in order to be properly effective, the cap should be soldered in place; as close to the A2 end of the ANALOG inputs header appears to be a very good, low-resistance location. Also worth considering would be across the (+5V) and (GND) pin header right in front of the 3.3V regulator.

    The reason this capacitor was chosen was that being ceramic, it will be very low ESR (My 60uF bank measured at less than 0.01Ω ESR) which will sharply buffer very brief transient surges in current, yet be small enough capacitance to prevent damage to supply circuitry when it initially charges. So far, it has proven to do just that.

    My more aggressive approach was to put buffer caps at the ANALOG, INPUT, OUTPUT, and I2C plug-in headers, not just the one we're having a problem with today. The UART1/GPS header appears to be pretty well protected by the cap I have at the INPUT header.

    mnem

    Over-volted, under-protected. The story of my life.

    • Quoted from Paul 

      DEVs - Have we performed any similar testing with the APM 2.5 boards and the MIC5219-3.3 regulator?

      I would like to know that as well. I was thinking I had APM 2.5 all the time. When I opened the case the other day to perform external mag. modification, I realized that I have APM 2.5.2 instead...

      • My sneaking suspicion is that that regulator does not have the problem.  We never had this problem before the 2.5.2 came out.  Now, one might say "but the problem only cropped up with 3.1 code, which came after the 2.5.2 was prevalent".  Yes, but remember, this failure can also cause the 3.3V supply to go to zero volts, which can cause the gyros to not work at all, and also can cause the EEPROM to not work at all.

        ie: On the one 2.5.2 that I have with a fried reg, I discovered it before 3.1 came out.  That is because for me, it manifests itself at 0V, causing no dataflash error.  

        I have never seen that on a 2.5.0.

        I believe what's happening here is that, this hot-plugging thing is causing an instability in the TPS regulator.  Some design quirk, just the right capacitance and impedance in just the right places, that when you give the system a good poke, it goes unstable.  

        • Developer

          I have hot plugged receivers on pre-production and older APM2.x boards more times then I care to think about during the PPM encoder development. No problem there. I am fairly sure this is unique to the new regulator.

    • You have some good points, so an applications engineer (or paper document) states that if Vin is a few milli volts lower than Vout we have self destruct, kinda hard to believe, since Micrel is a top leader in voltage regulators and this quote from the data sheet "Supply Input Voltage (VIN) –20V to +20V".

      Quote ( I hate doing this, the forum is sub-standard to RCG);

      "the cap should be soldered in place; as close to the A2 end of the ANALOG inputs header appears to be a very good, low-resistance location." Why not solder in parallel to the regulators input caps, can't get any better than that for low esr?

      I have a 4400uf bulk cap soldered (not the high resistance servo plug) soldered to the Rx input rail since day one, has it helped my APM from damage, maybe, maybe not.

      • Oz -

        First, NOT the MIC5219-3.3, that was only on the APM Rev 2.5. I was asking if we had done testing on that setup.

        The APM 2.5.2 uses a (TI) TPS79133. It shows a very low -V tolerance in the datasheet; definitely not hard to overlook that spec, and truthfully, not something we SHOULD have to worry about anyways; we aren't deliberately applying a negative voltage. I believe this is just a case of "The role of failure in design". You design it, test to the best of your ability, but it isn't until it's in the hands of the public at large that you get to see where "The proof is in the pudding".

        Philip was looking for a simple device that even those without advanced soldering skillz could implement to provide some measure of protection; A2 is a good choice for that. My suggested alternate location of the (+5V) and (GND) headers right in front of the reg actually share the same trace as the solder lands for the reg. ;) HOWEVER... that location will require slight modding of the APM case, or adding leads to the capacitor (which may decrease effectiveness) so it can be tucked away further inside. This also might induce a vibration issue with the gyros/accels.

        Tim -

        If you're powering from the Power Module, the filter caps in that should be providing some protection... hmmmm...

        *Pokes around his APM board with a meter for a moment*

        AH- HAH!!!

        Yeah, those caps in the APM Power Module aren't going to help a lot; they're on the other side of polarity protection diode D1 right by JP1. No way that's going to pass enough current fast enough to allow them to do any appreciable amount of buffering on the other 5V rail. DEFINITELY need some localized stiffening RIGHT THERE.

        Ummm... Philip, you may want to rethink your notion that the APM Power Module offers any protection against this failure; Tim's experiment gives evidence that suggests otherwise.

        PK

      • Why not solder in parallel to the regulators input caps, can't get any better than that for low esr?

        Because Philip wanted the fix to be something that was easily workable for people with varying skillsets to be able to accomplish at home.  Soldering a large through-hole package to the power rail is easy.  Soldering a tiny SMD cap in parallel with another SMD cap is not so easy.

    • Tim -

      First - just to clarify:

      You WERE able to reproduce this failure with the APM plugged into a battery?

      Was this via a BEC at the OUPUTS rail or with the APM POWER MODULE?

      Was JP1 jumpered or open?

      Yes this was with Lipo and USB connected. 

      (However I suspect that APM still draws from USB even if Lipo connected as I tried to use my USB keyboard but got the error that the hub did not have enough power to supply the device.   Not 100% familiar with the in's and outs of the USB protocol so it could have simply sent a canned message that I need x.xx amps of current even if it is not drawing it.)

      APM Power Module

      JP1 is open (5v not connected).

      Cheers

         Tim

      • FYI I also finished upgraded my MinimOSD Extra to the latest 750 pre-release and looks like everything is back up and running.

        No issues so far.

This reply was deleted.

Activity

DIY Robocars via Twitter
yesterday
DIY Drones via Twitter
RT @chr1sa: After more than a year of only virtual races, @DIYRobocars returns to the newly renovated @circuitlaunch on May 22 for the resu…
Apr 11
DIY Robocars via Twitter
RT @DAVGtech: And now available with LiDAR 🔥 https://twitter.com/Heavy02011/status/1381137016381964293
Apr 11
DIY Robocars via Twitter
RT @Heavy02011: #VirtualRaceLeague: @DIYRobocars Race #9 - #ParkingLotNerds #thunderhillracetrack, CA Join us for the next race April 24th,…
Apr 11
DIY Robocars via Twitter
RT @DWalmroth: Weather's finally cooperating, looking forward to racing 1:10 scale autonomous cars outdoors again! @diyrobocars, @NVIDIAEm…
Apr 7
DIY Robocars via Twitter
RT @AIDRI_: I finally succeeded in optimizing the trajectory and speed of a car on a #racetrack. Next step: implement a 2d controller and…
Apr 7
DIY Robocars via Twitter
@jetdillo @circuitlaunch Actually the second *in person* event in a year. We do virtual races every month
Apr 2
DIY Robocars via Twitter
Update: we're moving it back one day to Sunday (the 4th) at 11:00am instead
Apr 2
DIY Robocars via Twitter
@GrantEMoe @circuitlaunch Update: we're doing it on Sunday (4th) at 11:00am instead
Apr 2
Laurie J. Troy liked Jasper Kueppers's profile
Apr 1
DIY Robocars via Twitter
RT @chr1sa: Maybe we should have a mini @DIYRobocars race in our lower school's playground https://t.co/xLFeua1R6X
Mar 29
DIY Robocars via Twitter
If anybody wants to join us for an informal outdoors hack/race we're going to be meeting at the @CircuitLaunch park… https://twitter.com/i/web/status/1375907409223249923
Mar 27
DIY Robocars via Twitter
RT @SmallpixelCar: Ready to reopen, innovation has to continue. Inside/outside, LiDAR/GPS, race/delivery https://t.co/jpmvttoHEd
Mar 26
DIY Drones via Twitter
RT @DAVGtech: By far best race yet! Congratulations to the winner @Heavy02011 🥇🏆🍾👏👏👏 @diyrobocars @donkey_car @NVIDIAEmbedded https://t.co/…
Mar 20
DIY Robocars via Twitter
RT @Heavy02011: #VirtualRaceLeague: @DIYRobocars Race #8 - #ParkingLotNerds ⁦@DAVGtech⁩ ⁦@DWalmroth⁩ ⁦@OttawaAVGroup⁩ - join us tomorrow h…
Mar 20
DIY Drones via Twitter
RT @mrpollo: 11 years ago, the pxIMU was announced to the world on @DIYDrones, and it changed the life of many (mine included). The followi…
Mar 18
More…