Fly aways the failsafe and the 9x

 I recently steeped down as moderator so I can address this issue and the BS about it. Over the next few day's I be going over the code and addressing the real issues and possible solutions or not possible solutions. Ones that work or don't work.

 First we need to establish exactly what the APM should do in failsafe and what the safety issues really are. So lets hear the issues cause and effect and we'll get to the bottom of this one way or another. I hear talk about fixes that I don't consider fixes just work arounds. I know that one of them works for the 9x but it doesn't fully address the issue and nothing I have seen so far does.

 I'll be digging out the code and finding the truth myself so lets have at it.

You need to be a member of diydrones to add comments!

Join diydrones

Email me when people reply –

Replies

  • Not sure how this will go as I never really tested it in flight and now I run FrSky I dont need it but...

    I added some user code to watch a "Heart Beat" from a 9X running er9x. It is simple if the heartbeat stops for 10 secs (count 100 on 10hz loop) send a RTL and I also set the LED B to come on to indicate it was triggered (for testing etc).

    uservariables.h

    int heart_beat,count_lost,mode;
    boolean lost;

    usercode

    void userhook_MediumLoop()
    {
    if (heart_beat == g.rc_8.radio_in){
    count_lost++;
    }else
    {
    count_lost=0;
    if (lost){
    lost = false;
    digitalWrite(B_LED_PIN, LED_OFF);
    set_mode(STABILIZE); // control_mode = STABILIZE;
    }
    }
    heart_beat = g.rc_8.radio_in;
    if (count_lost>100){
    lost = true;
    digitalWrite(B_LED_PIN, LED_ON);
    set_mode(RTL); // control_mode = RTL;
    count_lost=100;
    }

    }

    Then on the er9x set up a heart beat on CH8

    as per this link http://postimage.org/image/w0fumpa9n (creds to buzz from http://9xforums.com/forum/viewtopic.php?f=35&t=1054)

    I got to the point of testing the LED comes on meaning the code activates for a failsafe but never tested in the air.

    Hope that helps someone and love to here is someone is brave enough to try

  • Ok, so I took a look at the Watchdog Timer and Brown-Out detection implementation on a 3DR APM 2.5 board.

    3DR APM 2.5:

    • Bootloader correctly handles WDT interrupt handler
    • Bootloader correctly handles WDT reset
    • Bootloader clears MCU Status Register

    Since the bootloader clears the MCUSR that means we have no idea what type of reset just happened. It is possible the MCUSR value is getting cached somewhere before it is cleared... does anyone know which bootloader is used? I really think logging these two data points would help with crash analysis.

    It is worth noting that I also tested a 2.5 clone board and it did not have a patched bootloader, so while the ISR worked, on WDT reset the bootloader continuously reboots until power is cycled.

  • Developer

    I posted about my latest work on "single channel fail-safe detection" in the "Kevin Finisterre.." thread to try and give the thread some sanity, but I guess this i a better place for it.

    So here is a repost:

    I had some free time and had a look at the ArduPPM and single channel fail-safe detection. I am not promising anything right now since I haven't finished testing jitter. But I think I found an acceptable compromise that will allow single channel detection on all channels and still maintain good general performance with regard to jitter and stick resolution. Trick was to migrate most of the fail-safe detection to the more timing deterministic PPM output generator, instead of doing it all in the PPM input interrupt.

  • Really great input in this discussion. I'm new to this forum and relatively new to multicopters but really interested in both the apm and maximising safety.

    A thought that keeps crossing my mind regarding the failsafe when just a few feet from the RTH location is, why not use a ground switch (GS).
    All military aircraft employ many safety interlocks one of which is a ground switch or Weight-on-Wheels (WOW) even the ones with skids.

    I'm sure there will be some testing involved to find the optimum setup, perhaps as many as 4 tiny microswitches on an interlock loop to ensure correct ground detection. The APM could verify that all is well with the (GS) at take off.


    I would also love to see the watchdog problem solved.

    Initially I thought just feed the motor outputs of the APM into another cheap controller (say a KK board, with inertia) and then modify the KK firmware to primarily be a watchdog for the APM but secondary role programmed to perform an emergency landing if the APM stops stroking the dog.
    Sounds good, but of course this simply moves the problem from the APM to the KK board. If the KK goes down all is lost, unless it has a very fast reboot (hmmm, need to explore this further).

    I think the best plan for now is as previously mentioned, to use a relay system with a simple watchdog which can switch between two controllers.
    This would be easy to rig up and test. initially switching between two controllers using a spare rx channel.

    This is my first post at this forum that interests me, so if I'm way off the mark please be kind, don't launch your heavily armoured multicopter to shot me down.
  • Why can't we do a 2 or 3 tier failsafe system?  first tier enabling RTL if the throttle failsafe is enacted AND the copter is above 1M (maybe 2? or user defined), the 2nd tier to enable RTL on geofence, and the 3rd enable a radius as new code to set RTL, or auto-land (in the even of low batteries) at a user defined radius from launch, if you've got a good radio that could be 500m, enabling you to fly anywhere at a moments notice without sacrificing safety. 

    beyond these suggestions I think we would be dipping into the realm of "user error recovery" which is an entirely different issue...

  • I've had a great deal of thought about this as well. And in my opinion failsafe is in essence a three step process.
    1. Detecting a fail safe situation (Loss of RC link/ failure to complete programmed mission)
    2. Choosing an appropriate fail safe action
    3. Performing the action

    The function of failsafe(in my opinion) is to reduce the consequences of a failure as much as possible, if one occurs. With this thought we end up with a tiered list of possible outcomes, from worst to best.
    - Chance of injury (Aircraft behaves dangerously, throttling up the motors when operator does not expect this when the aircraft is landed)
    - Fly away (Aircraft maintains controlled flight, leaves and is unrecoverable)
    - Crash (Aircraft leaves controlled flight and crashes nearby)
    - Landing (Aircraft lands without significant damage, autonimously or not)

    A failsafe action is a good one if the chance of moving down this list is higher than the chance of moving up this list.

    I would attempt to solve the problem as following:
    - Define what triggers a failsafe.
    RC Link loss is an obvious one, but what about the model banking above acceptable tolerance, going overspeed etc. All of these problems should cause the autopilot to abandon it's mission and go into a failsafe state.
    - Define what failsafe actions the autopilot should perform at what times.
    I'm thinking a single failsafe program for all different types of failures is acceptable. It doesn't matter what exactly failed, wether it be an RC failure or a gust of wind making the mission plan impossible. Predictability is important here, the operator needs to know what the aircraft will in any failsafe scenario.
    The first step of the failsafe program should be regaining control of the situation. In case of multirotors it should stop and hover. In case of planes it should orbit a fixed point. Regaining control of the situation does not only consist of the model regaining stable flight. It should also keep fighting being blown downwind for instance.
    If the failsafe is not cancelled by the operator taking manual control or the failsafe problem disappearing in a reasonable amount of time, the autopilot should perform actions to increase the chance of a better outcome. This boils down to(in my opinion):
    - Abandoning the flightplan. We are never going to attempt the mission anymore. Safe recovery is now top priority.
    - Gaining altitude to reduce the chance of colission with ground objects
    - Return to the operator. This both increases the chance of the operator regaining control and reduces the chance of the aircraft being unrecoverable. In my opinion flying back to the operator directly is better than retracing the aircrafts flightpath because it is so much clearer to the operator what is going on. The operator might be unaware that the failsafe has been activated at all.
    - Stay near the operator. The aircraft should not attempt atonomous landings because this is potentially dangerous/damaging.
    - Once a landing is inevitable (Low battery). The aircraft should try to touch down near the operator(a random location is fine, it beats an uncontrolled crash) with at least speed as possible. Throttle should probably be completely disabled for autonimous failsafe landing. It will be a bumpy landing. But a broken aircraft is preferable over potential injury/external damage.

    This would be a universal failsafe model that should work for most failsafe situations. If additional info is available, like the operator having supplied safe landing instructions, the autopilot might be configured to take different action. But what I've listed above should be in my opinion the default failsafe for any aircraft. These defaults can only be manually overwritten so the operator is allways aware of what the failsafe program will do to the aircraft.

    Cancelling the failsafe should be treated carefully as well. The RC link reestablishing should not be immediate cause for the failsafe to disengage. An example in which this would go wrong:
    1. There is a high wind blowing the aircraft away from the operator.
    2. The aircraft leaves RC range and enters failsafe mode. It tries to fight the wind to stay stationary.
    3. As the aircraft is at the edge of the maximum operating range, sometimes a short burst of RC activity reaches the aircraft.
    4. The aircraft reads an RC link and disables failsafe, it gets blown further downwind.
    5. The cycle repeats.
    In this case the model gets nudged away from the operator bit by bit untill the operator disables the transmitter(which is something you might think to do in a loss of control situation). A better solution would be to disable failsafe once a reliable RC link has been detected. (like: at least 2 seconds of RC link in the last 4 seconds).

    I've skipped one situation so far and that is the situation in which the failsafe program itself is impossible. The reason the aircraft goes into failsafe could be loss of knowledge of it's position(like a GPS lock loss). In this case the autopilot should try to run as much of the failsafe program as possible but skip the steps that have a high chance of making the situation worse. It should never attempt return to the operator without reliable navigation. But it should still attempt to maintain it's current position.

    I hope my thoughts on the matter help you in increasing the quality of APM's failsafe. I am not entirely happy with the way it is at the moment either. I do not get the feeling that the APM tries very hard to make a bad situaion better when something goes wrong, and sometimes it actually makes the situation worse.
  • I don`t know if it fits here. For me a RTL after failsafe is sometimes not the best solution (flying missions in an area with special topologic). In this case the copter should keep or return along the mission-path. Additionally some waypoints should be defined as "clear" waypoints, where the copter could land, if the battery is too low ...

  • I mean you can't really plan for every possible configuration and diferent type of radio that's out there.

     

    There's a standard output from most RX's when the signal is lost.

     

    Radio systems that have their own built in fail safe systems should have limited support or not supported at all. If supported, these should have configuration options of their own... say you can configure the APM to go into fail safe when the aileron is set to 1000ms / full roll for 3 seconds or minimum throttle / 900ms for 3 seconds or channel 7 is 100ms etc.

    Then tune the radio to output full aileron as 900ms or minimum throttle as 1000ms.

     

    For me, I have the t9x radio but am using a frsky setup that has its own user configurable fail safe settings.

  • My Naza will unarm if sits on the floor for a few seconds. If you like to fly again you have to rearm.

    Unless you want to fly missions were you land between waypoins, looks like a nice feature to have.

    We dont do so many touch&go with copters, and I prefer to rearm insted of having my copter flying away.

    Pablo

  • I've tried to figure out LEDs on several occasions so I have to say "No" it's not properly documented. It seems there a five different versions, all saying things like add code, and compile in modules or something. I actually gave up thinking it never got added to the APM release. Of course, maybe in the mean time someone has come along and cleaned up the docs and compiled it all in. I dunno. I'd like to add a strobe or two. I have lights in all four corners, but an "Armed" strobe would rock my world. 

    Any hoo..... About the 9x. I have a stock FS-Th9x with the stock 8/9 channel receiver, and so far as I can tell, fail safe doesn't work. First off, "setting" fail safe in the radio, you use the stick. Which you can only set down to 0% throttle. But the APM would see that as my minimum throttle of 1015 or something, which is still in the normal range and won't trigger the APM failsafe properly.

    BUT..... this is all rendered moot as when I turn off my transmitter..... nothing happens. All the channels freeze where they were. So much for the 9x failsafe. I suspect the receiver does not support failsafe, or I'm doing something wrong. 

    So I could set this up then go out of radio range and the copter would just keep on truckin' even though I "setup" failsafe. 

This reply was deleted.

Activity

DIY Robocars via Twitter
RT @SmallpixelCar: Test my algorithm on real car https://t.co/idWRxcrmuK
15 hours ago
DIY Robocars via Twitter
RT @breadcentric: Evo is coming to town. If you have an #AWS #DeepRacer, you can purchase an extension pack! https://t.co/w60JwI98Hp #Machi…
21 hours ago
DIY Robocars via Twitter
https://t.co/bzpbxytpsv
yesterday
DIY Drones via Twitter
RT @chr1sa: My talk on PX4 and FAA certification is coming up at 1:45 PST today on the PX4 Dev Summit livestream. Includes some cool new st…
Jul 7
DIY Drones via Twitter
RT @seesharp: I'm tuned into the PX4 / Dronecode free live conference. Great stuff. Microsoft AirSim talk in 10 minutes. https://t.co/0zbZ2…
Jul 6
DIY Robocars via Twitter
RT @masato_ka: 距離センサを3つとESP32を付けたラジコンカーをDonkeyCarライクにNNで自動走行。3層FC極小モデルをTensorFlow Lite for microcontrollerで動かしてる。機体は借り物でRumiCarって言います。Tenso…
Jul 5
DIY Robocars via Twitter
RT @SmallpixelCar: My car was able to go all the way autonomously until the crosswalk. It was only 100 yards from the target. What should b…
Jul 4
Liam left a comment on Agricultural UAVs
"Hi
I'm Liam from T-MOTOR. I would like to reach out to see if there is any possibility for us to work together.
We are a propulsion system manufacturer who offers motors, propellers and ESCs for all kinds of drone applications which vary from secur…"
Jun 30
DIY Robocars via Twitter
RT @SmallpixelCar: Smart move. The car used the shadow to guide it through the bridge. This was never in the training samples. But it learn…
Jun 30
DIY Robocars via Twitter
RT @SmallpixelCar: Getting closer to the target. Single camera. Untrained road. https://t.co/Wsr7RwDamj
Jun 29
Richard Cox left a comment on Australia
"Anyone in the DIYDRONES Australian subgroup based in Alice Springs, NT?
I am experimenting with Ardupilot (standard Arduplane), Pixhawk 4 FC in a 4-ch
RC "AXN Floater Jet" foamy plane..."
Jun 29
Omar Sykes left a comment on Australia
"Hi everyone, I am looking for someone who is good at drone building, repair and software in Adelaide. Please give me a call on 0477 319 219."
Jun 29
DIY Robocars via Twitter
RT @RoboticMasters: #donkeycar https://t.co/czuLoVRcA4
Jun 29
DIY Robocars via Twitter
https://t.co/DKQbNTbz0g
Jun 29
DIY Robocars via Twitter
RT @RoboticMasters: Donkey car, car car car car car car; Donkey car, car car car car car car; Donkey Car. Anyone like our tiny tiny donkey…
Jun 29
DIY Robocars via Twitter
RT @SmallpixelCar: After improving DBSCAN speed, I can get 11 frame per second on @NVIDIAEmbedded Jerson Xavier MAXN mode and the autonomou…
Jun 26
More…