Precision of Vertical Maneuvers – It's All About Safety

Hi,

Not long ago we did tests with multiple different drones to find out how precisely can they follow a standard photogrammetry route - https://www.ugcs.com/en/page/comparing-precision-of-autopilots-for-....

The quality of obtained data will mostly be affected by precision on a horizontal plane – whether the shots were made in desired positions or not.

Let’s imagine a case where the survey area has power lines crossing it. It’s a common scenario. To survey the area you would either have to divide the field in two different segments – one on each side of the power lines or you would have to make the drone fly above the power lines like so:

Provided you have a 3D model of the power lines (or any other obstacle), UgCS can automatically generate a flight path which will look safe. But in reality there is no information on how precisely will the drone follow the route with aggressive vertical manoeuvres, so we decided to do some tests.

Test 1. "Jump" over an obstacle

Here we will test how will the drone move over an imaginary obstacle such as a power line or a building.

The drone will start flying from the leftmost waypoint, accelerate up to 10 m/s (in reality no drone managed to accelerate to this speed in a 60 m segment, maximum speed was around 7 m/s) and then “jump” over the obstacle.

We tested 3 drones:

- old workhorse - DJI Phantom 2 - as representative of first generation of DJI autopilots

- quad with fresh ArduCopter 3.5 firmware

- DJI Phantom 4 as representative of the latest generation of DJI drones.

For reference we decided to use a telemetry track recorded using the emulator drone that is built-in in UgCS. It flew very well :)

Let’s start with Phantom 2

Turn type set to Stop & Turn. Not ideal, but nothing dangerous here. Maximum error was around 2 meters. Funny that 4 years ago Phantom 2 was one of the best consumer level drones in terms of flight precision…

Turn type - Adaptive Bank Turn. Ooops…

Turn type set to Bank Turn. Almost the same outcome as with Adaptive Bank:

Phantom 2 conclusion: Never use any other turn type except Stop & Turn if you need to make vertical maneuvers near obstacles.

The same should apply for all first-generation DJI autopilots including DJI Naza-M V2, Wookong-M, A2, Ace. They all have similar behaviour in horizontal flight and it seems that they share common flight algorithms.

ArduCopter quad

Turn type - Straight. The result is not very good at the part where the drone has to descend. If you are trying to fly over a building with a flat roof or close to the roof and walls, you should set the route with a safety margin of roughly 10 to 15 meters.

So with the turn type set to Spline it drew this strange picture:

ArduCopter conclusion – When flying around obstacles only use Straight turn type and allow a safety margin of 10 – 15 meters between the set route and the obstacle. In UgCS there is a special parameter just for this – “Safe distance from Obstacle” that can be adjusted in the vehicle profile.

Phantom 4

Turn type - Stop & Turn. The result is almost perfect. We saw the same in horizontal precision tests – on-par with PX4.

Adaptive Bank Turn. The results are much better than we expected. Maximum deviation was less than 1 meter.

Phantom 4 conclusion – one of the safest drones to use in this scenario. However, keep in mind that DJI changes the firmware quite often so if you need to fly routes like this often, we recommend to test the behaviour of the drone after each firmware update.

Test 2. Loop

This is somewhat of a rare scenario, mostly we see the demand for such trajectories only for the purpose of drone shows, but nevertheless it can demonstrate autopilot potential very well.

We decided not to spend time testing this with Phantom 2 since it isn’t meant for manoeuvres like that.

Here again we used UgCS emulator for reference:

ArduCopter 3.5

Turn type - Straight. It seems that ArduCopter is much more fond of aerobatic manoeuvres like this than boring “jumps” over power lines. :)

Turn type - Spline:

Phantom 4

Turn type - Stop & Turn.

Turn type - Adaptive Bank Turn.

That’s all, guys. We hope this article was informative for you. If you feel like making more tests like this, head on over to http://www.ugcs.com, get UgCS and start experimenting yourself. We are curious to see how other autopilots perform in these scenarios.

Safe flights,

UgCS Team

 

 

 

 

Views: 1128

Comment by Pablo Legarreta on September 21, 2017 at 8:56am

Very very interesting!! Thank you for sharing.

The big problem of DJI nowadays is the constants firmwares updates with no fly zones update requirements that you are not sure if you would fly that day or not, specially in places with no internet connection. The drones that makes are very very good but with that BIG problem.

Comment by Chris on September 21, 2017 at 8:19pm

I suspect what's happening is that Arducopter is saying it's hitting the waypoint early as it goes up, thats why your getting a lift before and a drop before the actual waypoint.  Some tweaking of the parameters should yield far better results if tuned for it and it that is the use case that a commercial operator needs then they should have a set of tuned parameters for it.

Thats the benefit if the open source stuff, we can change those sorts of parameters to meet the various non standard use cases.

Since the test is using out of the box parameters it is what it is.

I am impressed that the DJI P4 did so well in the testing.


Developer
Comment by Marco Robustini on September 22, 2017 at 12:06am

About "Ardupilot quad", to have that bad result it means that something has been wrong in the parameterization of the drone.
Personally I get better results, even Phantom 4, so please check your quad parameters and try again... :P

Comment by Alexey Dobrovolskiy on September 22, 2017 at 3:50am

Hi Chris, Marco - we used almost default parameters. Purpose of this article was not to make competition between drones or autopilots. We wanted to point to important issue for flights safety.

BTW, how many ardupilot users will spend lot of time to fine-tune parameters, especially to achieve better vertical precision? I think not many. It's much more easy to have 10m of additional safety margin) 

Comment by Alexey Dobrovolskiy on September 22, 2017 at 4:03am

Pablo, you are right! Very big advantage of ArduPilot/PX4 that you can decide when and where to fly - according local rules, acquired permissions etc.

We are receiving lot of urgent requests from professional operators - we are on the field, my DJI XYZ is not willing to fly, I have permission from authority, what I can do??? Ask DJI... I thing that they should implement some kind of perpetual unlocking for professional operators.


Developer
Comment by Marco Robustini on September 22, 2017 at 5:24am

Hi Alexey, making a perfect tuning with Ardupilot is imperative, otherwise it's better to buy other drones almost tuned from the factory.
But what i mean is: this kind of comparison would be made with an Ardupilot quad perfectly tuned, or the result is invalid, you can't do these tests with the default.
I've performed similar tests and my error doesn't exceed 100cm from the scheduled path.
The Phantom 4 use "uBlox M8N" gps, in the Iris i think is an M6.
Before publish these comparisons do more about it, thank you! :P

Comment by Alexey Dobrovolskiy on September 22, 2017 at 5:50am

Marco, if you have fine-tuned ArduCopter vehicle why not to repeat our test? It will take just 5 minutes... I can send you test routes (in UgCS or KML format). We will add your results to article. Also it will be interesting to see difference between default and tuned ArduCopter.  

Comment by Rob_Lefebvre on September 22, 2017 at 8:16am

Hi Alexey,


You can get much better performance from Arducopter with a very simple change.  Just set a 1 second pause/hold time at every waypoint.  Now the copter will hit each waypoint exactly, hold for one second, then move on.

What you are seeing in your tests, is that Arducopter is designed to quickly and smoothly move through waypoints if it has not been instructed to stop at them.

Simple as that.

Comment by Alexey Dobrovolskiy on September 22, 2017 at 3:30pm

Hi Rob,

of course we can get almost ideal route following using this trick, but it will be far from real life. And not interesting )

Comment by Auturgy on September 23, 2017 at 7:46pm
Alexey, I'm not sure your point to Rob is valid: if "stop and turn" is used for phantom, it isn't a useful comparison unless you do the same for Ardupilot. Using artificial assumptions and poorly tuned gains when comparing any open source flight controller to a pre-tuned rtf product doesn't yield results that are worth anything.
If you tune the aircraft properly, and construct the mission properly, how does the performance of Phantom and Ardupilot compare? I have no idea- and from the description above, neither do you.

Comment

You need to be a member of DIY Drones to add comments!

Join DIY Drones

© 2017   Created by Chris Anderson.   Powered by

Badges  |  Report an Issue  |  Terms of Service