This post is to share a bit of the final experimental work of my PhD, since my doctoral defense is next week...
Multirotor Unmanned Aerial Vehicles (MRUAV) have become an increasingly interesting area of study in the past decade, becoming tools that allow for positive changes in today’s world. Not having an on-board pilot means that the MRUAV must contain advanced on- board autonomous capabilities and operate with varying degrees of autonomy. One of the most common applications for this type of aircraft is the transport of goods. Such applications require low-altitude flights with hovering and vertical take-off and landing (VTOL) capabilities.
Similar as before in this project we use the AltaX Flight Stack which is compromised by a Raspberry Pi 3 as companion computer and a naze32 as flight controller.
The slung load controller and the machine learning estimator is running on the RPI3, although of course the training of the recurrent neural network was done offline in a big desktop computer. The RPI calculates the next vehicle position based on the estimation of the position of the slung load, everything is running using our framework DronePilot and guess what? its open source ;). Keep reading for more details.
If the transported load is outside the MRUAV fuselage, it is usually carried beneath the vehicle attached with cables or ropes, this is commonly referred to as an under-slung load. Flying with a suspended load can be a very challenging and sometimes hazardous task because the suspended load significantly alters the flight characteristics of the MRUAV. This prominent pendulous oscillatory movement affects the response in the frequency range of the attitude control of the vehicle. Therefore, a fundamental understanding of the dynamics of slung loads as they relate to the vehicle handling is essential to develop safer automatic pilots to ensure the use of MRUAV in transporting load is feasible. The dynamics of the slung load coupled to a MRUAV are investigated applying Machine Learning techniques.
The learning algorithm selected in this work is the Artificial Neural Network (ANN), a ML algorithm that is inspired by the structure and functional aspects of biological neural networks. Recurrent Neural Network (RNN) is a class of ANN that represents a very powerful system identification generic tool, integrating both large dynamic memory and highly adaptable computational capabilities.
In this post the problem of a MRUAV flying with a slung load (SL) is addressed. Real flight data from the MRUAV/SL system is used as the experience that will allow a computer software to understand the dynamics of the slung in order to propose a swing-free controller that will dampen the oscillations of the slung load when the MRUAV is following a desired flight trajectory.
This is achieved through a two-step approach: First a slung load estimator capable of estimating the relative position of the suspension system. This system was designed using a machine learning recurrent neural network approach. The final step is the development of a feedback cascade control system that can be put on an existing unmanned autonomous multirotor and makes it capable of performing manoeuvres with a slung load without inducing residual oscillations.
The machine learning estimator was designed using a recurrent neural network structure which was then trained in a supervised learning approach using real flight data of the MRUAV/SL system. This data was gathered using a motion capture facility and a software framework (DronePilot) which was created during the development of this work.
After the slung load estimator was trained, it was verified in subsequent flights to ensure its adequate performance. The machine learning slung load position estimator shows good performance and robustness when non-linearity is significant and varying tasks are given in the flight regime.
Consequently, a control system was created and tested with the objective to remove the oscillations (swing-free) generated by the slung load during or at the end of transport. The control technique was verified and tested experimentally.
The overall control concept is a classical tri-cascaded scheme where the slung load controller generates a position reference based on the current vehicle position and the estimated slung load position. The outer loop controller generates references (attitude pseudo- commands) to the inner loop controller (the flight controller).
The performance of the control scheme was evaluated through flight testing and it was found that the control scheme is capable of yielding a significant reduction in slung load swing over the equivalent flight without the controller scheme.
The next figures show the performance when the vehicle is tracking a figure-of-eight trajectory without control and with control.
The control scheme is able to reduce the control effort of the position control due to efficient damping of the slung load. Hence, less energy is consumed and the available flight time increases.
Regarding power management, flying a MRUAV with a load will reduce the flight times because of two main factors. The first one relates to adding extra weight to the vehicle, consequently the rotors must generate more thrust to keep the desired height of the trajectory controller, hence reducing the flight time. The second factor relates to aggressive oscillations of the load for this reason. The position controller demands faster adjustment to the attitude controller which increases accordingly the trust generated by the rotors. The proposed swing-free controller increases the time of flight of the MRUAV when carrying a load by 38% in comparison with the same flight without swing-free control. This is done by reducing the aggressive oscillations created by the load.
Check the nice quality gif here.
The proposed approach is an important step towards developing the next generation of unmanned autonomous multirotor vehicles. The methods presented in this post enables a quadrotor to perform flight manoeuvres while performing swing-free trajectory tracking.
Look at this beautiful light painting photo comparison:
Don’t forget to watch the video, it is super fun. Original post in here.
Comments
Oh yes, I have seen that video before, great work! you were using the AscTec Hummingbird right? Beautiful bird indeed!!! that inner loop running at 1khz is super super nice! I always wanted one of those, but my Uni does not have enough budget for it... impressive research tool!
So cool! I did this (in a simpler setup) as an undergrad with a simple modification of the position control loop to take into account the angle and angular acceleration of a slung payload. Also did everything in a vicon room with no onboard processing (back in the day when autopilots were much more expensive!) Your work looks great!
" I guess it would benefit to have the payload position and resulting force to beed back into the vehicle positions/attitude controller?" -> exactly, that is precisely what I'm doing ;) the estimator estimates the states of the payload and its then feedback to the swing free controller than then computes a new position for the vehicle and this position controller sends attitude commands to the flight controller, a tri-cascade controller.
having a active slung load controller will means that your flying times will be increased (as I proved in my experiments) because the vehicle is not fighting against the payload, this was one of the results in my tests... vehicle with payload and swing-free control will fly 38% longer than the same vehicle with the same payload but not swing-free control.
I did two different types of estimators, one using machine learning (presented here) and another one using computer vision (image in the previous comments) but worked great and is very workable these days indeed. In your case a downwards camera with a rpi3 will be more than enough, or even a pixycam and a rpi/edison. The pixy connected to the CC and the CC to the APM, the swing-free is computed in the CC (using desired trajectory, vehicle pose and SL relative position from the pixy) and velocity commands are sent to the APM, it could be beautiful ;)
Ok, that does look quite a bit better. I guess it would benefit to have the payload position and resulting force to beed back into the vehicle positions/attitude controller?
I have actually been thinking about having active slung load control like you are doing here. But I was initially thinking it would require the slung load to be attacked to, basically an upside-down joystick with X and Y axes to give the position of the load.
But I supposed that simply having a downward facing camera with MV looking at the payload would be quite workable these days, wouldn't it?
I forgot to ask you, when you say "navigation/position control system seems to be significantly under-performant compared to Ardupilot" to what plots are you referring to? how are you making the comparison? if its to the plots of the top views, that performance is due to the slung load affecting the vehicle actually... this a plot of my position controller against a circular trajectory:
Hi Rob!
Oh yes, I have seen your videos before, pretty intense stuff you're doing, really amazing!. In your application and because your slung is very long, the oscillations of the load are going to be minimal (not so aggressive), that is why is more than enough to just use the normal position controllers from APM.
In my case of a much shorter tether and indoors work, the oscillations get pretty aggressive pretty fast. Actually the minimal length that my system can stabilise is 1.6 meters, something shorter than that and my position controller is not fast enough to remove the oscillations. And even in my case, my normal position controller will stop the oscillation after some seconds... The main point is to remove entirely the oscillations.
Actually I have another method, which I did not showed in this blog post, where I put a gimballed camera on a vehicle pointing downwards and run in the companion computer a CV algorithm to locate the colour of the load, and then use the same swing-free approach to reduce the oscillations... and precisely in one I'm using pixhawk. You can see the diagram here:
The main problem with this setup is that it requiere a bit larger vehicle, and having a slightly larger vehicle than a 330mm inside my lab it gets very problematic, there is barely space for a step response test. And I need my lab in order to obtain the "real" slung load position from the motion capture so I can assess and compare the slung load estimators.
I used both flight controllers in my experiments, but for indoors and mocap I use the "racing" flight controllers in my lab just because several reasons:
- Price of the flight controller (£20 for naze32 against £150 for pixhawk) (I usually had several crashes where the flight controller gets damaged and its easier to replace a naze32 than a pixhawk)
- It is easier to "inject" pilot commands with a companion computer to the naze32 than with the pixhawk. I created a library called pymultiwii that sends pilot commands to the board in a breeze. I did also the rc_override with dronekit but it did not work exactly in the way I wanted and it caused a couple of crashes inside the lab, there was always always a lag. At the moment is much better, but at that stage I had to settle with what I had.
- size comparison of the platforms:
At the moment I prefer the px4+ros approach, and I did made it work inside the lab, using vrpn from optitrack mocap directly to the px4 ekf, it was just wonderful, super nice!
I you have big problems with the slung load in your helicopter perhaps we can chat more and I can suggest or design a solution for you, if we combine our skills, nothing can stop us ;)
Your heli is beautiful, good work!
Thanks a lot!
I'm very interested in this topic, as I'm currently developing a system to do just this thing for magnetometer surveys. The payload must be slung in order that the sensor is not affected by the magnetic interference from the UAV. So far I've been successful doing it "open-loop". Just using Ardupilot's high performance control algorithms and soft navigational acceleration rates. It helps a lot that a helicopter has a much greater attitude control power than a multirotor.
I notice from the graphs you posted, that the navigation/position control system seems to be significantly under-performant compared to Ardupilot. What was behind your choice to base your work on top of a hobby grade racing flight controller? Your slung-load estimation system might perform better if it was driving an Ardupilot-powered flight controller.