After more than two hundred flights with my APM2 equipped quad, today, in the remotest of possible locations my quad failed to respond and attempted to fly away by itself! All I can say is thank goodness for Knobthorn trees.
I wanted to get some video of the area but didn't have my small cam with me so I attached my Galaxy Smartphone to the front of the quad, I have done this before and know that the video won't be that great (lots of jello) but will at least give an idea of the scenery. I waited for satellite lock and then hand launched (too much dust on the ground) just in front of where I was standing, I went slowly up to about 20m, hoverered then did a 180deg pirouette and slowly started descending back towards my position.
When the quad was about 3m from me and 2m off the ground I throttled up to stop the descent which was fine but now the quad was drifting forward slowly so I tried to tilt it backwards... suddenly I had that sickening realization that it wasn't responding, I wiggled the sticks...NOTHING!. I ran towards it but it was still drifting forward and climbing very slowly, now at about 2.5m. It continued very calmly and smoothly and I briefly considered throwing the only thing in hand (my transmitter) at it to try bring it down, but it flew on until it collided with a Knobthron tree at about 4m high.
3 motors were jammed by branches but one was still spinning away calmly. I wasn't... I was able to run to the nearby parked pickup and drive under the tree, reach up to unplug the battery and then get the quad down.
I have read the horror stories of fly-aways but one naturally never expects it to 'happen to me'.
- My radio is a tried and tested two year old Hitec Aurora 9 which has one of the most robust AFHSS 2.4GHz systems. The quad was 3m from me when it happened!
- My failsafe is setup correctly and double-tested, with the throttle channel going to 900ms on event of signal loss.
- All four my SS 30A ESC's (2A BEC) power the APM2 (each tested 0.09v of each other) effectively quadrupling the current capacity. I've never had a suggestion of a brownout.
So what happened? I don't know, I can't see anything definative in the logs (attached). If someone has any ideas I'd be very grateful!
As for now my faith in APM2 has been shattered, like an old friend you can no longer trust.
We are working on this exact thing at the moment where the APM will be looking for valid input signals and will trigger a failsafe mode if it does not receive a signal within a specific time period - probably 1 second.
It might be useful if we could come up with a rudimentary test procedure for RF interference rejection ability of the APM from the most common sources that the APM could plausibly have to endure in common use cases. Things like WiFi tranceivers, cell phones (or the DroneCell and the like?), some of the most popular telemetry transmitters people might have and so on.
Test interference from such radio sources in a homebrew anechoic setup and see if and when the chips on the APM freak out. A location in the middle of nowhere, with no trees or buildings, is plenty anechoic enough. Start varying the setup, with different wiring for example - can the wiring from the RC Rx to the APM make it worse and how to minimize it? How about RF shielding cans over the chips on the APM board? Mu-metal enclosures and shielding for the parts that aren't magnetometers or radios?
I think we have some people with experience in software defined radios in this community; it should be plausible to create something that chirps a blast of RF energy at the board. No need for excessive power levels either; if a highly directional horn antenna close by sending a few milliwatts of power doesn't make the PPM encoder go crazy like sending frozen channel values, it would be a data point that suggests the cell phone in this fly-away might not have been the root cause after all. Or the opposite could happen and we could look into how to harden the APM to make it less likely to happen in the future...
I've been brave enough to now fly untethered a few times again and have had no sign of a re-occurance of this issue. I was never able to recreate the same Rx>APM freeze in my testing that was experienced in this attempted fly away. I have put a brownout capacitor on the APM though just for a little extra peace of mind.
I've also asked Michael if it's possible for a visual and audio notification of failsafe condition here: http://diydrones.com/forum/topics/request-for-audio-and-visual-noti... but this will obviously only notify you if you have the MP connected. In my case however the APM never recognised any problem and thus never went into failsafe
" If the phone cannot hear a signal from a cell tower, it does not transmit, only listens"
It depends on the phone, but most will "ping" periodically to see if it can hear a tower.
Interesting concept - this would surely give your APM a few milliseconds of power during a brownout. what are the cap specs?
Trunk now also has a check whether the ppm encoder has died. So if the ppm encoder stops sending updates to the main CPU the APM will notice and trigger a failsafe the same as it would if you pulled the radio wires out or turned off the receiver. This and the other fixes mentioned earlier in the thread will all be in 2.9. I've got to hope that this is enough to catch your issue (and other like it) should it reoccur.
Well...at least this incident was a good opportunity to review all our failsafe behaviour.