Is it possible to bypass a transmitter altogether and control the aircraft using a computer and joystick via the radio modem? Of course, I'd have the transmitter on and hanging from my neck for legal reasons. (wink wink)
Long story short, it's possible, but expect latency issues with the XBee (comm drop-outs lasting several hundred miliseconds or more, depending on the model and the band it's operating on).
As far as I know there are no regulatory requirements explicitly stating that you should use an RC controller to control your aircraft. It's OK as long as you're in charge, no matter what means you're using (RC, radio modem, leash, whatever).
I'm not aware of any similar modules with low latency. You could turn off all the error checking and re-transmit functions of the xbee and handle the error detection and correction yourself, that might help in your specific application.
Or you could get a pair of digital radios and do all the networking yourself (something like these), including all the error checking. It might not help at all though. YMMV :)