You can do it from CLI at least but you should be able to do it from APM Mission Planner too.
On CLI go to setup and type "frame"
Also if you compile/upload manually your software with Arduino IDE, you can define this on compile time.
With values of in APM_Config.h file:
#define FRAME_CONFIG HEXA_FRAME
#define FRAME_ORIENTATION X_FRAME
Thanks! So physically installing the APM. I already have the board mounted pointing toward a boom arm on my hexacopter for + mode. If I change it to X mode do I have to re mount the ardupilot so its pointing in between booms?
Yes, Depending on what type of frame you are using. You usually can just rotate APM carrier boards 90 degrees and then it will automatically point between arms.
And you need to rotate APM 90degrees Counter clock wise eg to left. That way you don't need to change any OUTput cables on APM. Just define frame X and new motor order should be ok.
Here is a picture from our wiki pages showing same thing:
On + mode, APM points towards motor 1
On x mode, APM points between motors 3/7
If you move it any other direction, you will have problems on setting up motor orders etc.
Just wanted to add my 2 cents. The above configuration works for a Pixhawk Hexacopter, but you have to plug motor 7 into Pixhawk output 5 and Motor 8 into Pixhawk output 6. I searched around and did not find this advice. Actually, I found very little help on how to configure a hexacopter in apm forums or diydrones, unfortunately.