Pixhawk BAD ACCEL HEALTH APM 3.2 - collecting data

A few users, myself included, appear to be having issues calibrating accels and/or arming when using APM 3.2 on a Pixhawk.  I see posts here and there, but thought it may be best if those having the issue could all 'check in' to one post so that the developers can see how widespread the issue is and gather data.

My Pixhawk will arm occasionally, and will even fly successfully.  Occasionally however it will not arm and I will receive the pre-arm error "Accels not healthy".  A few days ago I was able to arm and fly thorugh one lipo, then after landing and swapping batteries I could not arm due to this reason.  I have attached the logs from the successful flight, as well as the failed pre-arm logs, to this post.

As you can see from these images there is something amiss with IMU2.  This is a snippit from the good flight, showing values for AccX on IMU1 and IMU2:

3691166950?profile=original

and here are the same values on the second attempt, pre-arm.

3691167027?profile=original

In the 3.2 release thread a few users had mentioned the issue and Randy advised to set the log_bitmask to "131070" so that it will log everything including the pre-arm checks.  I encourage others having the issue to do the same and share the logs and experiences here so that we can find out what is going on here - is there a bad batch of Pixhawks in the wild that only now show the hardware errors due to something new in 3.2, or is there an issue in the APM software?

goodlog.bin

badlog.bin

You need to be a member of diydrones to add comments!

Join diydrones

Email me when people reply –

Replies

    • Hi,

      I uploaded this FW, and after a lot of attempts, I got it to fail again.  The first time, I wasn't connected to mission planner, so I don't know what the prearm failure was. The second failure, I got the message "altitude disparity" (which was new for me).  The artificial horizon in MP was tumbling all over the place.

      I was unable to get the failure indoors (warm, 20 deg C), but after about 30 minutes outdoors (chilly, 7 deg C), I got the failure.   Not sure if cold was the cause, since I've had failures indoors as well in the past.  Copter is warming up inside now, and seems to be OK after about 15 minutes.

      The logs of the failed arm attempts seem to show the same offset and flatlined IMU2 values as before.  I noticed that the ErrG and ErrA values are zero for all logs and both IMU's.

      Also, I noticed that for the failed attempts, GPS/RelAlt fluctuates up and down like a sine wave between -50/+50.  Don't know if it's significant, or maybe a consequence of the IMU error.. just thought I'd mention it.

      See attachements for 3 logs:

      1 - prearm OK

      2- prearm failed (I idn't catch the message)

      3- pream failed - altitude disparity

      Hope this helps -- let me know if I can do anything else.

      Cheers,

      Erik

      PS:   off-topic:   I noticed that this is AC V3.3.  Out of curiosity, where can I find a changelog for V3.3?

      prearm fail.zip

      https://storage.ning.com/topology/rest/1.0/file/get/3702539069?profile=original
      • Erik,  Thanks for doing this testing for us, it really helps.

        Unfortunately, there is not yet a changelog for V3.3.  It's way too early in the release cycle, and Randy has not prepared a summary like that yet.  What you are flying is bleeding edge code right now, and you should know that there is some risk to doing this. It's not bad, as we're pretty good about this the past 2 years or so, but just so you know.  I fly leading edge code on my test quads, but would not use it on a valuable machine. 

        You can actually see the changes here:

        https://github.com/diydrones/ardupilot/commits/master

        But it would probably be hard to follow, it's not really intended for regular users.  To try to answer your question more succinctly, I can't think of any significant changes, things that really make a difference for how it flies from a consumer perspective.  It's too early for those changes to have been merged in yet.  Most of what is going on is nuts-and-bolts type stuff like these Gyro bug fixes, etc.

        It would be great if you could keep helping us out with this, but when done, you might want to revert back to 3.2-stable.

        Rob

        • Rob,

          Thanks for the explanation.   I was just wondering what's in store for the next version.  I really like the new PosHold mode in 3.2. I had actually already installed github to download and looked through the changes out of curiosity, although it's a bit hard to follow sometimes. I'm trying to get into learning something about the code, and hopefully making some additions myself sometime.

          Tridge,

          Here's two logs for the latest build. One OK (indoors, 20 deg C), one failure (after 30 minutes outdoors @ 5 deg C). The failure was again "Altitude disparity". The artificial horizon was tumbling and the reported position on the map was going around in big circles/ovals.

          I have an RTFHawk v2.4.3 from witespy. 

          For some reason, the failed log gets a filename date in 1980 (even though the GPS was working).

          Doesn't the MPU6000 (and maybe also the other IMU) have a temperature sensor?  Would help to log temps, since there appears to be a correlation?

          hope this helps,

          Erik

          logs.zip

          • Developer

            Hi Erik,

            Thanks for those logs, they are very interesting. What the logs show is that you have an intermittent lsm303d sensor (the lsm303d provides the second accelerometer and the internal magnetometer).The magnetometer portion of the sensor seems to be OK, but the accelerometer is giving complete rubbish output.

            Assuming that this board hasn't been in a bad crash (so we couldn't explain it as impact damage) then ordinarily for this sort of failure I'd ask you to send it back to 3DR for analysis. I don't know if witespy would accept a return or not. You could try to do the analysis yourself, but it depends if you are willing to go into depth learning about how to debug sensors at the NuttX nsh level. What we need is a register dump from the lsm303d when it is failing (ie. run the commands "lsm303d regdump" and "lsm303d info" at an nsh prompt when it is failing).

            The bit that puzzles me is that the IMU2.ErrA value doesn't show a non-zero error count. That seems to imply that the lsm303d does have all the correct register settings, but is still giving rubbish data. We saw something similar in the very early days of development of the Pixhawk in 2013, but we thought we had fixed that issue.

            Cheers, Tridge

            • OK.  I got the failure again.  The first time I connected via MP's terminal, I got screenfulls of garbage characters.  Tried a few more time and got into the nsh shell. Screenshots of the command results are attached (I can't copy-paste text from MP's terminal -- is there a trick to this?)

              Only thing is: when connecting via the terminal, the pixhawk resets and the status light does not work as per usual, so I have know way of knowing for certain if the sensor failure is actually occurring by looking at the light.  (It very likely is, since the other 5 "regular" startups I did get the failure)

              3702875455?profile=original

              3702875421?profile=original

            • OK, i'm an idiot.

              I just realized I can get into the nsh shell via the "test" menu in mission planner's terminal.  However, I'm not getting the failure right now.  I may need to wait til later this afternoon when it cools down a bit.

            • Tried connecting Teraterm via Telem1 and an FTDI, but no dice.  I can connect to the COM port, but no sign of life.  The pixhawk beeps like crazy (+/- 10 rapid beeps followed by a short pause -- low battery? - it's powered via the FTDI).  

              According to https://pixhawk.org/users/serial_connection I should be greeted by an nsh> prompt in the terminal.

              Any ideas what I'm doing wrong?  I'm not sure I'm connected to the right port - the setup section on the above webpage about which port/uart I need is confusing me...

            • Tridge,

              I don't mind doing that myself, but you'll need to walk me through the steps. I'm trying to follow the steps on this page: https://pixhawk.org/users/serial_connection but i'm not getting very far.

              I installed the PX4 toolchain for windows.  When I connect to the PX4 serial port via USB (57600,8N1) in Teraterm, I get a bunch of garbage characters on the screen.  I guess it's just spitting out mavlink messages?

              Same problem if I try connecting via TELEM1 (which is UART1, I think?), although I'm doing this via a 433MHz radio link.

              I'll try via an FTDI on TELEM1, but I need to completely disassemble my copter first for this. What I don't understand is, how will the pixhawk know not to spit out Mavlink messages on these ports?

            • Developer

              Hi Erik

              Can you please take a photo of the side of the PCB that has no plugs? I want to see exactly what sensors are fitted.
              Thanks Phil

              • Philip,

                I can do this, but I need to totally tear down my quad for this.  Maybe I will do it this weekend.  What are you looking for specifically?

This reply was deleted.

Activity