If you are one of the 21 people that read my discussion post earlier this week, then you will know I'm an ArduCopter novice making beginners mistakes, and using pymavlink tools to interface my system.
Actually, I think the vast majority of those 21 people were me checking back to see if I had any replies, but I digress. Onwards and upwards as they say, without letting you know that sometimes it's followed by rapid tumbling downwards and CRUNCH, like oh so much angry robot confetti. Embarrassing? yes a little, but since these tools seem to be widely underutilised I figured maybe I should write about how I used them to analyse today's incident.
First, an overview for the unfamiliar. MAVLink is a binary messaging protocol specifically designed for communicating with micro unmanned vehicles (MAVs). There are various autopilots (including ArduPilot) and ground segment programs (including both QGroundControl and the ArduPilot Mission Planner) that use MAVLink to communicate. Go here http://www.qgroundcontrol.org/mavlink/start if you want to know more about it.
Andrew Tridgel wrote a software library for communicating in the MAVLink protocol from the python programming language, called pymavlink (http://www.qgroundcontrol.org/mavlink/pymavlink). When you download the source code, it includes a folder called "examples" containing a bunch of scripts that demonstrate how you might use this library in your own python programs. And by the way, the examples are also such a useful collection of tools that you probably won't ever need to write your own python programs.
The three "example scripts" I used today are mavproxy.py, mavlogdump.py, and mavgraph.py. I put example scripts in quotes because these are my complete ground segment software, I have never even used the Mission Planner and don't intend to start. I'm not just being deliberately perverse, there are also sound reasons related to the architecture of our Out Back Challenge system (http://www.canberrauav.com/). In fact the whole reason I'm tinkering with an ArduCopter in the first place is because I wanted more diverse experience with pymavlink. Well, that and it's kinda fun :)
The main program in the suite is probably mavproxy.py, it's a command line GCS that does a lot more than it's name suggests. In simplest terms, it pulls MAVLink data from your modem and sinks it into a number of different places. One of those places is a log file, which we will see more of in a moment. It can also send it to other GCS programs via serial links or a UDP network socket. Or 50 different UDP sockets and serial links, whatever, it's a thing for doing the "plumbing" to send MAVLink data wherever you need it.
mavproxy's development is being driven by our OBC campaign where we are using two copies, one on the ground segment and one on a linux computer in the plane. We have multiple components in the airframe that produce and consume MAVLink messages (including redundant autopilots in a failover configuration), as well as multiple data links to the groundstation and onboard ethernet. Mavproxy is our pony express that makes sure the messages always get through.
It has one more trick up it's sleeve; a command line interface. Actually, an interactive command shell is probably a better name for it, because you can interact with mavproxy while it's running to do basically anything that could be done by clicking buttons on a more conventional GCS. This interactive shell lends itself to another layer of scripting, which is how our artificially intelligent mind reading software is going to guide the vehicle to where the judges think Joe is hiding. Or something like that, computer vision blah blah blah.
Today, I was just using mavproxy to record a log file of my fateful quadcopter flight while verbalising interesting facts through the laptop speakers. I started it something like this:
mavproxy.py --speech --aircraft=hopter --quadcopter --baudrate=57600 --master=/dev/ttyUSB0
The --aircraft option tells it to store the log file (and a copy of the configuration parameters used) in a folder called "./hopter/logs/$(todays-date)/flight$(number)/" which is very handy because otherwise I'd probably loose it. The --quadcopter option instructs mavproxy to use an interactive shell with the appropriate dialect of domain specific language (otherwise it uses the fixed wing dialect by default) for command and control. Typing "help" at the prompt is the best way find out about that, basically equivalent to browsing the buttons and menus in a GUI GCS.
There are plenty of other options...
So I have a log of my flight and crash. It's a big lump of MAVLink goo, what am I supposed to do with it? This brings me to the next tool in the suite, mavlogdump.py. This converts binary MAVLog data into text. It's got options and stuff but I haven't looked at them yet, I just type this
and pipe it into a text processing chain of unix tools like grep, sort, head, less etc. Then I eyeball it and guess what the data means, from the names and values of the text-rendering of MAVLink messages. You see stuff like "ATTITUDE.roll" and wonder, frob frob frob.
So, about my crash. This is what happened, as best as I can recall your honor: I started mavproxy, switched on my transmitter, powered up the quadcopter, and stuffed around for a while while it sat on a level surface. Then I carefully carried it over to the flight line, put it on the edge of the runway and walked back to the pilot box (forgetting to groundstart again, oops). Then I disarmed, gave the motors a little spin, and realised I had forgotten to plug in the low voltage alarm. So I re-armed it, walked back and plugged that in and returned. This time I disarmed again, spun up and climbed to knee height and tried to keep it in the same spot for a moment before climbing up to a more conventional stooging-about altitude. It got rather wobbly, so I climbed more to give myself recovery room, and started flying big left/right tick-tacks parallel with the runway. Suddenly it stopped flying and tumbled to the ground. I recoverd the pieces then killed the mavproxy.py process.
So, all that's left now is to figure out why. I'll cut to the chase:
mavgraph.py SERVO_OUTPUT_RAW.servo1_raw SERVO_OUTPUT_RAW.servo2_raw SERVO_OUTPUT_RAW.servo3_raw SERVO_OUTPUT_RAW.servo4_raw ATTITUDE.pitch:2 ATTITUDE.roll:2 mav.log
That's the whole log, showing some sitting still, then some getting carried, then a quick spin up followed by some more sitting. Then there is a squigly bit, a squiglier bit, and finally a very squigly bit. The mavgraph window has pan/zoom buttons, so I used my mouse to zoom into the squigly bit and some of the squiglier bit, thus:
Here you can see the motors pulsing and the pitch/roll oscillating, probably due to my quad having shorter arms than the default ArduCopter frame (I didn't tune the PID coeficients yet, a bit less Proportional weight might sort this out). If only that was the worst of my problems...
Here I zoomed right into the last few seconds. Look, when pitch and roll suddenly show tumbling, SERVO_OUTPUT_RAW.servo1_raw jumped to MAX. This is not power output from the motor pluged into the rightmost plug on the APM, it is the autopilot shouting "pull, dammit" (PWM 2100) to motor number 1. Because it had suddenly stopped pulling.
Bullet plugs! I had been warned, and had started replacing them with directly soldered connections, but only done motors 2 and 4 because I was being impatient. Lesson learned.