Can someone tell me if
THIS and
THISwill work together and is this a good choice for failsafe and reliability on my first quadcopter.

Thank you,


Views: 10320

Reply to This

Replies to This Discussion

You're telling me that the fly-away of my 2-day old quad was NOT due
to the 9x I was using but rather an issue with the APM?

Not quite.  You reported total loss of control which could have been:

  • Loss of power to the Rx - a Failsafe Rx would not have helped
  • Antenna break - Failsafe Rx would have saved it
  • Tx failure/restart - Failsafe Rx would have saved it

Regardless, someone needs to look at the APM.

Kevin, gotta remember that we're running prototype open source software on prototype hardware with DIY mechanicals and DIY wiring.  There's a high possibility of failure for a wide variety of reasons.  That's why I wait a while before loading any new version of firmware.

I have replaced a large number of broken props, bent arms, broken motors/mounts ... many from my own flying stuffups, but some from software/hardware failures. What happened to you was, as they say in Air Crash Investigations, an unfortunate series of events.  For most people, the failure you encountered would have sent their quad into a tree or the ground.  You just happened to be climbing at the time ...

Ok I think I need to clear up some facts about using outputs from a microcontroller because you guys are making some pretty wild assumptions and don't have all the information you need to make a good decision.

While it is possible that the PPM encoder (it's actually a sum, not so much encode according to those in the know), could contribute by misreading the receivers output, it is not going to work if it gets hung. These are 8 bit microcontrollers and the software runs in a loop. In order for the PPM encoder to keep puting out the same input values to the mega 2560, the software loop would have to contain a statement. Otherwise, if something caused the loop to get stuck, then NO output happens. That's how it works period. It's physically and logically impossible any other way. So, even further, the 2560 which is using Fast PWM outputs must also run in a loop and thus cannot and will not output out all the PWM channels in the event of a lockup in the loop (caused by anything). It's like the distributor on a car. If it stopped spinning, it might fire on one plug but not all them, the others would have no signal. So in the case of a quad, it's IMPOSSIBLE, for the APM to have a lockup in the software and output equal PWM to all the motors. You might have a crash where one motor was spinning and the rest weren't, but that's a crash or flip, not a flyaway. In order for all the motors to get equal PWM, the APM main code in the 2560 had to be operating 100%, there is no other answer. You are implying that somehow, the output section of the code, due to some type of bus lockup gets put into a loop and just keeps repeating the same values, and what I'm telling you is that that's fundamentally not possible. SInce the PPM encoder must end a digital summed signal to the main 2560, it too MUST be working in the loop. The ONLY other option is a piece in PPM encode failsafe that would output high throttle and neutral other summed signals. Again, there is an sum that must happen so a software loop must be running in order for that to even happen. The source for the PPM encoder is out there so you can look for yourself.

 

Again, you guys have a semi valid gripe the receiver and the PPM encoder do not work well together. My point is, PPM encoder is separate from APM firmware. So no, the APM did not cause this crash. The combination of that receiver and the PPM code WE ALL USE, is the problem. You can't get much finer detail than that.

Again, no solder joint , no magnetometer, no GPS or any other sensor caused this. Pure and simple, it's known that receiver and that PPM encoder firmware don't mix.

 

You guys are reading into other posts and specualting when you do not understand how the code and system works. I am trying my best to explain the functions in simple terms so that you can understand the how and why what I've said is true. Sorry for beating this up but that last thing we need is wild speculation. Again, based on the designed function and hardware limitations I have described, there is a very narrow envelope of what could have happened to have the exact fly away event occur. I have attempted to explainthat fully. The bottom line is that it's not the ardupilot firmware, but a reaction between your reciever and the PPM encoder and thus, that's what the discussion needs to be narrowed down to.

It seems to have been buried in the thread, but somebody needs to check this:

Dave said that he has seen the APM appear to treat "No PPM input signal" as normal operation, holding the last good values from the Rx internally.
- I don't know whether that is a misunderstanding, generally true or unique to his setup. It needs to be tested ASAP.

A lack of PPM input must trigger failsafe, regardless of all other inputs. It clearly means that the pilot has no control of the aircraft.

Secondly, Vernon, please calm down and do some more research because what you posted earlier today about PWM is completely untrue.

Here is how PWM works in a microcontroller like the AVR:

In almost all microcontrollers, PWM output is run in dedicated hardware that does nothing more than count and compare every 'tick'. It does not depend on any part of the firmware still going, it merely needs the clock source to be running.
- On some MCUs, the PWM 'tick' may be an external clock running at a completely different rate to the rest of the MCU, and on many it even keeps going during 'sleep'.

Thus if the firmware did hang or crash, the PWM signals will keep outputting the 'most recent' values until the MCU is reset - because they are running on dedicated silicon.

In the case of the AVR/Arduino, "Fast PWM" is indeed a hardware PWM. (The ATmega 2560 has 15 semi-independent channels.)

For output to ESC/servos they are set to a 20ms period, and duty cycle between 1 and 2ms. The duty cycle is given new values regularly by the main loop and keeps using those values until a new one is set.

If the main loop doesn't give them new values, they'll keep using the old ones.

Try it if you like - write a program that starts a PWM ouput then goes into a "while (1) {};"

- The PWM continues, even though the AVR is sat in a busyloop doing nothing at all!

On top of that, if the main loop has hung, interrupts will still fire and be processed.
- For example, in ArduCopter firmware the PPM input itself is processed by a "capture" interrupt.

This means that if the main loop crashes, the PPM input is still being received - though turning that input into new values to output won't happen because the AHRS and PIDs are in the main loop.

A lot of stuff still happens if the MCU has crashed, you cannot say that a lack of main loop means no output - the result depends very much on how the code operates and what crashed where.

In the case of the ArduCopter firmware, if the main loop hangs at a moment of full-power-all, then it'd continue do that until the MCU is reset.

The only way to prevent the PWM outputs from continuing to send the 'last settings' for period and duty cycle if the firmware has hung is to use the watchdog timer, which will automatically force the MCU to reset if not fed correctly.

- In a quick search I wasn't able to find the place(s) where the wdt_reset() or wdt_enable() are called in the ArduCopter firmware though, which worries me somewhat. They are being used, right?

All that said, a multicopter would probably crash pretty fast if the APM was not running properly, as these need continuous active correction to fly at all.

Vernon, you're railing on way too hard.  There's plenty of code issues which could have caused the problem.  My APM2 used to lock the throttle at high for sometimes 20 sec with the last firmware.  Now it just seems to goose the throttle for 1/2 sec at TX startup.

There's defiantly a need for more robust code in many respects.  And your explanation of the processor architecture is lacking.  It is not difficult to set up a RTOS type system for pre-emptive multitasking and preventing hangs and freezes.  There's watchdog timers, regular timers, and interrupts that can be used for breaking out of frozen loops and preempting lockups.

The processor doesn't have to run straight through the code and loops.  That's just something that needs to be written in.

I'll be doing plenty of testing with the 9X and APM2, and I'm sure this issue can be figured out.  The APM is far from perfect in it's handling of PWM inputs and outputs.  You don't need to endlessly and vehemently defend it.  I'm sure the dev team would be the first to admit that there are plenty of bugs, issues, and improvements that need to be worked on. 

I have to second Richard's statement.  Almost all that Vernon wrote about PWM is incorrect.

Note that I was talking about loss of PWM inputs - the separate Throttle, Rudder, ... connections between Rx and APM.  What happens with loss of the PPM input I haven't looked at.

As Richard said, the PWM input is processed by a change-of-state interrupt routine.  It the PWM disappears, no interrupts are generated and so no new PWM values are determined - the old ones are retained.

There is a timer associated with this processing in order to determine the PWM 'on' time.  It would be a relatively simple matter to add processing to the timer overflow interrupt to check if there has been PWM activity since the last timer overflow and, if not, trigger a Failsafe.

If it's as bad as you say it is, we all should stop flying today for fear of the APM locking up and your heli flying away? I mean based on what your saying, loss of signal or the APM crashing basically results in flyway for sure? I read that as if your receiver lost power, the last inputs are "locked in" in the PPM and that's pretty much a gone aircraft from there?

Please help the developers fix these flaws in both the PPM and the main code. Obviously, you guys seem to have a solid handle on this.

That's why I'd like somebody else to test it.

I don't have APM hardware and I've only spent a short time looking at the code, nowhere near enough to really understand what it's doing.

If you've got an APM, please try the following tests with Props OFF:

First test (basic RX failsafe)

  1. Spin up to about 25% throttle
  2. Turn off the Tx
  3. What happens?

Second test (APM loss-of-signal failsafe)

  1. Spin up to about 25% throttle
  2. Unplug the Rx
  3. What happens?

Both of these should result in some kind of safe shutdown of the copter. (My copter just kills the motors.)

If either do not, then you have a problem that needs solving before you fly again, because you don't have a failsafe unless both of these shut down the aircraft.

- Note that APM does support an 'autoland' when the failsafe triggers.
Be sure that you know what the aircraft should do on loss of signal before you fly!

Ok, at the risk of upsetting others, here is my test.

APM2.0, arducopter 2.5.5, RC Timer ESCs with SimonK firmware, 880kv motors, Spektrum AR8000 receiver, DX8 TX,quad in + setting, also with sonar and 3DR 900mhz telemetry.

Just trying to give you full hardware specs.

Power on,arm, bring motors to 1/4 throttle.

Turn OFF DX8 TX, motors stop nearly immediately, the glitch here is that the TX takes a second or two to shut down (confirmed the TX caused the slight delay in the next test).

Turn on transmitter, APM still armed,so motor spin up to the throttle position.

Now,unplug the power and ground from the APM to the AR8000 receiver. Motors stop instantly!

Now final test, and I think this is where the gripes against what posted come in, is that throttle up, motors spinning, remove ONLY the throttle wire from the receiver to the APM, motors continue to spin.

Conclusion of this test, is that YES, there is a flaw in the PPM sum firmware on APM 2.0. The flaw is that if valid signal is received on some of the input channels and throttle is lost, then the PPM continues to sum into the output the last received throttle signal. This is in fact a dangerous flaw in the firmware, and sorry to those who I upset trying to defend it. This explains the test conducted by another user and this was my point, to isolate the EXACT failure so the development team can replicate and fix the problem.

Thus, is also my conclusion of the current situation. If your receiver lost RF to the TX (noise, distance, or interference) and that receiver then drops all outputs, then the APM does as expected and shuts off the motors. The AR8000 I tested does just this in default mode out of the box. If your wiring, soldering or other signal interuption occurs between the APM and the radio receiver, then the PPM (another assumption as it could be in the APM firmware) continues to send the last inputs.

To clearly recap my testing of APM 2.0 and arducopter 2.5.5:

Lost RF link with AR 8000 and DX8 = motor shutdown

Power loss to AR8000 receiver= instant motor shutdown

Loss of throttle channel connection only from RX to APM=motors continue to run 

I would call this a valid concern. Sorry to those who I upset, but again, I'm trying to make it right.

So, further assumptions from the above test.

Prevous fly away event occured using :

a stock  9X radio that has been reported in other threads to have a failsafe problem

No previous test of failsafe conditions(TX off, RX loss of power, and  single channel loss)

Last input was high throttle to climb at some unknown distance from the TX

 

Assumptions:

Crash was root caused by loss of RF link between TX and RX.

Secondary factors:

Possible known issues of receiver output being at least considered a valid PPM  on at least one channel.

Possibly newly discovered flaw in APM 2.0 arducopter 2.5.5(Assuming present in older versions) that if a single input is valid to the PPM, then it's possible the APM continues the last known good throttle inputs. (not yet validated on APM 1.0)

Other possible factors:

Bad solder joint or loose/bad wiring on throttle signal input to APM could cause the same scenario if valid input is received at the exact same time on other input channels.

Final assumptions:

Genuine Spektrum AR8000 receiver and DX8 seem to invoke proper failsafe motors off if RF link is lost.

Loss of throttle connection between RX and APM2.0 could cause throttle to remain at last position IF other channels from RX to APM are valid. Possible root causes include RF link loss combined with invalid failsafe in RX, bad solder joints or wiring between the RX and APM 2.0 on the throttle channel only (thus allowing valid signal on other channels but loss of signal on throttle only).

There may be other factors but at least this can be reproduced. I strongly encourage others with different hardware (especially different RX) to follow the tests described (TX off, RX loss of power, single channel loss, mutliple channel loss, and also APM 1.0 VS APM 2.0 to cross check). Please use common sense prop off testing (sorry but had to put in a disclaimer). 

Sorry to all and again, the goal here is to give back by showing a test that may help the Dev team solve this.

 

I need to add so as not to scare the whole community, my testing of a Spektrum DX8 with AR8000 showed the APM 2.0 performed as expected under all but one condition, where throttle channel was lost but other channels continued to be valid inputs to the APM. This seems to be a specific and rare/unlikely scenario with Spektrum hardware, but in theory, could happen with any brand of RC equipment.

Also again, sorry to anyone who I argued otherwise.

 

 

 

In one RC forum a poster stated that the 9X failsafe on signal loss is to hold the last output on all channels EXCEPT the throttle.  The throttle supposedly goes to zero, or maybe it turns off alltogether.

Don't have my gear handy at the moment and I'm leaving on a trip shortly.  But this could explain the fly away issue.  It seems to fit exactly with what you're saying.

This discussion is relavent to the recommendation of the 9X radio in this thread.

Jake,

Sorry for all the back and forth,but yes, if that is true where the receiver maintains the last signal on rudder, elevator, and aileron, but drops the signal all together on the throttle channel, this seems to be the magic combination that affects the APM 2.0. That would also seem a logical failsafe, say in an electric plane where the ESC directly connected to the receiver would stop the motors and in theory the plane would glide straight and level. In addition, the ESC would likely beep with loss of signal making it an effective plane finder once the thing landed/crashed. Not an entirely undesireable combination in an electric plane. This is bad in other combinations as I believe a servo in a gas plane or heli would maintain the last position (not forcefully,motor not active) but also not close the throttle unless a mechanical return spring was in place on the throttle linkage.That seems to be a warning for some usage scenario of that radio, that while the failsafe seems logical in and electric plane to act as a beep device, it also doesn't in theory work right in gas vehicle with a throttle servo and signal is lost.

Also, just a theory, since I think I have isolated this, is that if the PPM or APM 2.0 firmware (update also effects  APM 1.0) were fixed to remove this bug of maintaining the last known stick position if throttle PPM input is lost but other channels remain valid, then the 9X variants of receiver would in effect become "compatible", as well as increased safety for users against in flight wiring sitatuations that could happen with any radio system.

Also, Update, this also happens on APM 1.0 arducopter 2.5.5

Tested with "Orange Box DSM2" receiver

LP5DSM TX

Test is to arm the system and bring up the throttle.

Turn off TX=motors shut down (also tested the RX with a servo and it sends a low signal on all channels when RF is lost)

Remove power from RX =motors shut down (effectively no input on all channels to APM 1.0)

Remove only the throttle input signal=motors continue to run

Anyway, this is definitely a bug that should be fixed for safety reasons.

Reply to Discussion

RSS

Groups

Season Two of the Trust Time Trial (T3) Contest 
A list of all T3 contests is here. The current round, the Vertical Horizontal one, is here

© 2019   Created by Chris Anderson.   Powered by

Badges  |  Report an Issue  |  Terms of Service