The development team is always trying to find ways to improve our testing and the quality of our code. We've recently add a new automatic testing system that should help a lot with finding bugs and ensuring that the huge range of features that we have added (and plan to add!) to the code keep working.
If you have a look at http://autotest.diydrones.com you'll see the results from 'Tools/autotest', a test system that runs on every change to the ArduPilot and ArduCopter code. The test script currently checks the following:
The flight itself is controlled using a python scripting system that flies ArduCopter using MAVLInk commands. Currently that script does the following:
Running these test steps every time we make a change to the code is already ensuring that a lot of our code works, but we have plans to greatly expand this test suite. What we'd like to do is have every ArduPlane and ArduCopter feature tested, with code that validates that it actually does the right thing. At the moment the test suite only has very rudimentary testing of whether the flight was actually correct, whereas we should be checking lots of parameters (roll, pitch, altitude, speed etc) throughout the flight.
The other obvious missing piece is ArduPlane testing. The test suite checks that ArduPlane builds, but doesn't yet check that it flies correctly. I hope to fix that this weekend, with ArduPlane flying some simple missions plus some 'manual' (ie. python scripted) flying. I'll probably do this by interfacing with FlightGear, but jsbsim linkage of even a new physics model is a possibility.
Under the covers
This new test system involves quite a few new bits of code, some of which we hope will be used for things other than automatic testing. The quadcopter simulator itself is a simple physics simulator written in python, which Michael has since translated into C# for use in the planner (thats how you can fly quadcopter HIL in the planner now). The communication between the various pieces uses pymavlink, a MAVLink implementation in python. That uses a bunch of helper scripts that make it fairly easy to script a ArduPilot vehicle. I'm hoping this system will be used for scripting real copters and planes in the future, along the lines of the acrobatic scripting system we discussed earlier in the year.
The other big part of this autotest system is the 'desktop' (also known as SIL for "software in the loop") build of ArduPlane and ArduCopter. This builds the code using a normal C++ compiler, and creates executables which run directly on your desktop computer, rather than on an APM board. The advantage of this is that we can use all the debug tools (such as gdb and valgrind) which are normally used for C++ development, but also it runs much faster and doesn't rely on hardware being plugged in. We've already taken advantage of the ability to run ArduCopter under valgrind to find some bugs, and it really increases our confidence in the code to be able to run that sort of sophisticated memory checking on every commit.
If you want to have a look at autotest, then start by grabbing the latest APM code and having a look in Tools/autotest. You might like to start by looking at arducopter.py and seeing how it controls the quad. We're hoping that we will get contributions of new tests. We also plan on putting up a wiki page describing how you can run the autotest system on your own computers (right now it only runs on Linux and MacOS, although Micheal has had some limited success on Windows using cygwin).
How you can help
The autotest system is already quite useful and it has already helped us find some subtle bugs in the QuadCopter code, but we'd love to expand it to cover more aircraft and features. Here are a few things that it would be great to get contributions on if someone has the time:
Hopefully this will be the beginning of a much more rigorous testing effort for ArduPilot. We'll post more information as the system develops!