It took me a while to notice this, in fact I did a test flight before without noticing the problem, so I'll post a video.
The compass in my APM2 swings back to between 20 and 50 degrees as soon as the unit sits still. It swings "correctly" while the unit is turning, but as soon as the unit sits still (no change in yaw) the compass starts swinging back to NW and settles there in around 5 to 10 seconds.
Here is a video that shows the problem. The video was filmed in late afternoon so you can tell west by the suns location. The ocean is to the east. The video starts with the plane facing roughly south. As you can see, the compass reading change while the plane is turning, but when ever I fly straight for a while the compass starts returning to NE.
I thought there was something in the plane interfering with it, but I have removed the APM2 from the plane and sat it alone on a wooden bench with only the USB cable attached. The behavior is still the same.
Any thoughts anyone?
I disabled the compass, figuring I could use GPS heading until the problem was resolved.
I tried to confirm that the heading did not change when I rotated the APM2 after the compass was disabled, I reset the unit after disabling the compass.
The heading still changes (although it shouldn't have) only now it works correctly and when the unit is still the heading stays fixed at the new value.
However with the compass disabled in software I can't add my declination, so the heading is still incorrect.
Now I'm quite confused.
After looking around a bit I found references to the manual calibration option in the compass hardware settings. I had entered a declination, but I hadn't calibrated the compass. This is a must do!
After calibration the compass works great (funny that). Doing manual calibrations I found that rotating it slowly around each axis works better than spinning it quickly (slowly being 3-5 seconds per rotation, quickly being around 1 second).
I hope this helps someone. :)