We took our Telemaster up again today, this time with two APMs on board. There were two aims with this flight. The first was to get an idea of how much two APMs would vary in their attitude and altitude solution, and the second was to test a new experimental GPS that one of the CanberraUAV team members received as a birthday present from his brother (his brother builds GPS modules for a living).
For the outback challenge competition we plan on having two APMs in our plane. The idea is that the mission control computer on the plane (a pandaboard) will constantly monitor both APMs and will switch over to the backup APM if the first one starts misbehaving. To do that we need to know how much variation between the two APMs we normally should expect, so we know when we should be thinking about switching between them. The logs from todays flight gives us some idea of what variance we should expect.
The setup we used today was that both APMs were connected via USB to the pandaboard, plus one of them was connected to the normal servos, receiver etc, and was used when we were in stabilise or loiter mode. Both APMs logged detailed telemetry to the SSD on the pandaboard via mavproxy.
Apart from being in slightly different positions in the plane, the only other differences between the two APMs were:
- one used a 5883L magnetometer and the other used a 5843.
- one was connected to the planes pitot tube airspeed sensor and the other wasn't
To analyse the results, I wrote a little graphing utility (mavgraph.py) for MAVLink logs, which I have put in the examples directory for pymavlink.The raw logs are here and here.
While looking at the results I found that the magnetometer on the 2nd APM hadn't initialised, which means its yaw and heading values are rubbish. We'll need to do more flights in the future after ensuring that both magnetometers initialise correctly. I'll also need to add a warning message to mavproxy for when the magnetometer doesn't produce good data.
Results
The first thing I wanted to look at was the roll/pitch solutions. Here is a typical graph for a short section of the flight.
The tracking between the two APMs was reasonable, although not as good as I had hoped for. The two IMUs will of course have had slightly different calibration, so we expect a constant offset, but the graph shows that the offset isn't constant. Sometimes the 2nd APM shows a larger roll, and sometimes the first one does.
The altitude solutions show a similar variance:
The VFR_HUD altitudes are from the two barometers, the GPS_RAW values are from the GPSes, both are in meters above sea level. The two height estimates tracked each other quite nicely, with the variance small enough not to bother a plane. A quadcopter hovering might have some problems at low altitude, but at the altitude we were flying at today (around 100m or so) it wasn't an issue for the Telemaster.
The airspeed/groundspeed values were about what I expected:
the pitot tube produces a much noisier signal than the GPS does, and the two GPSes tracked each other quite nicely. The difference between the pitot and GPS readings could well have just been the wind, which was several meters/second.
I was curious to see if the raw gyro readings tracked each other. The MAVLink RAW_IMU packet allows us to see the raw sensor values before that are processed by the APM, and I wanted to see how much noise showed up:
They tracked better than I expected them to! I have the hardware sensor filters enabled on both IMUs, and I suspect this shows that those sensors are doing their job quite well.
We'll be looking at these log files a lot more closely in the coming weeks, but this quick look has already told us a lot of what we need to know for designing our OBC avionics, and working out how to do failover between the two APMs.
Comments
The pandaboard runs mavproxy, which means it has a MAVLink feed from the APMs, and it proxies that feed to the ground station. I'm in the process of writing mavproxy modules for the OBC search and rescue challenge.
The image processing code is available in our git tree. It is currently not integrated with mavproxy, but that will change over the next few weeks as we start linking all the components together.
Cheers, Tridge
Thanks
@Omar, both APMs are connected via USB serial to the pandaboard mission control computer. For the test flight I described in this post one of the APMs was connected to the RX and servos, the other was just gathering data. In the future we will have both APMs connected to the RX and servos via a multiplexor, and we will be able to switch control between them.
Hi, Please, are there some communication between the two APMs? Serial? Other?
Thanks for the feedback. I'm sure that many people here will be following your efforts with great interest.
Hi Bill,
We're still working out the exact conditions when we will switch between APMs. One fairly clear situation is when one of them experiences sensor failure, such as GPS failure, or if one of the APMs just locks up and stops triggering a heartbeat line. We also will have the ability to switch APMs using manual commands from the GCS, which will allow us to fly experimental code on long missions with some confidence that if the experimental code malfunctions we can switch the plane over to the alternate APM and get it to fly home. We'll know it is malfunctioning both because we'll have the telemetry streams from both APMs on the ground station, but also because we plan on having digitial video streamed from the plane to the ground, and it should be pretty obvious to the operator when the plane is behaving strangely.
I suspect however that we will be able to tell which APM is wrong when the attitude solutions diverge. There is a fair bit of redundancy in the various sensors from each APM. For example, we can cross-check the attitude solution against the 2 magnetometers to see if both solutions are consistent with the mag values. We may also run some sensor fusion on the pandaboard from the RAW_IMU stream from the two APMs.
The other type of thing we will want to do is compare two different algorithms for the same task. With one APM you need to bring the plane home, then re-flash it and fly it again, hoping that the weather conditions haven't changed enough to invalidate the test. We will be able to flick back and forth between two algorithms and immediately see the effect. We will even be able to re-flash an APM while in flight (I already do all my compiling for the Telemaster APMs on the pandaboard in the plane, over an ssh link from the GCS).
Cheers, Tridge
I do like the sound of having a failover system, but in reality how many events will allow you to identify that your primary APM is misbehaving? There is obviously the situation of complete loss of data from the primary APM to the monitoring pandaboard and loss of GPS lock, but diverging AHRS figures would ring alarm bells but can you tell if it is the primary or secondary APM which is at fault?
Really interesting to study this! Thanks!
Good stuff, thanks.