Hi. I was wondering how the whole MAVLink, Mission Planner, and PX4 Pixhawk all come together in terms of sending and receiving information. From what I've read so far, MAVLink is a protocol, Mission planner interprets incoming and outgoing messages, and the Pixhawk reads the messages that are sent from the Mission Planner from the radio transmitter. However, what do all these things really mean?
My goal is to receive live video feed from my quadcopter using a gopro, use image processing (like OpenCV) to identify a target (like a red circle), and send movement commands to my quadcopter to follow this target when it moves.
My main problem is being able to send the commands to my quadcopter. I know you're supposed to use something like rc override to send the command, but I am not sure where to start. Also, could anyone suggest whether this task is better done in C++, Python, or Java?
Replies
I am also working on a similar project and interested to find out more.
So far, I found a related blog post on controlling drones using Python through the Dronekit libraries.
https://www.ddmckinnon.com/2015/12/30/idiots-guide-to-dronekit-pyth...
Hi Bill
since we face a similar problem, did you finally get a solution to your request? Can you please share it with us?
MISSION PLANNER COPTER SIMULATION
I'm trying to simulate a copter mission in MP. The python script that I'm writing must check for the existence of a file in some specific directory. This file is a QGC formated (.waypoints) file including the waypoints to follow and some other actions. In case of such file existence, the MP must make its pre-arming checks, then to arm the vehicle, to take off in "GUIDED" mode and afterthat to follow the trajectory described in QGC format in the "trajectory.waypoints" file. The final command inluded in the above file will be an RTL command.
Can someone please indicate to me where to look for the MP commands for the above actions or alternatively related examples?
HI Bill,
Were you able to find a solution for your project, we are interested to know how it works.
I am actually working on an identical project (autonomous tracking using visual targets). I am trying to have a fixed wing aircraft search for the known target and then loiter around it while maintaining visual. The wall I am hitting right now is how to send gimbal commands to keep the target in the FOV. I know the syntax for sending RC commands to my gimbal through a specific RC channel, but I would prefer to send a Region of Interest (ROI), Point of Interest (POI), or point camera here command. Please let me know. Have you found syntax for sending MAVLink commands? that might be the answer for me.
As for your problem, I have done similar work in the past where I put a vehicle in guided mode and set guided waypoints using python scripting through mission planner. For your application (assuming you are doing the processing on the ground station), I would set up a script with OpenCV to determine the target location relative to the vehicle, then set a new guided waypoint for the quad to fly to, to adjust the targets position in the image. To do this, one solution is to pass in the heading, lat, lon, and alt of the quad and pass out the new waypoint. see code below for more detail. Feel free to ask for clairification and good luck.
lat = float(cs.lat) #gets lat of quad
lon=float(cs.lng) #gets lon of quad
alt=float(cs.alt) #gets lon of quad
heading=float(cs.groundcourse) #gets gc of quad
item=MissionPlanner.Utilities.Locationwp() #Creates a waypoint
MissionPlanner.Utilities.Locationwp.lat.SetValue(item,lat) #sets lat of wp
MissionPlanner.Utilities.Locationwp.lon.SetValue(item,lon) #sets lon of wp
MissionPlanner.Utilities.Locationwp.alt.SetValue(item,alt) #sets altof wp
MAV.setGuidedModeWP(item) #Sets waypoint
I haven't dealt with MAVLink, Mission Planner, and Pixhawk as individual components, but I have done similar types of tasks with various other programs and protocols. From the way you laid out their components I can venture a guess on how it "comes together." MAVLink is the way things communicate (much like information over the internet which typically uses TCP/IP. The mission planner is the core of all the functions, so it speaks using the MAVLink (the TCP/IP). The Pixhawk is similar to the MAVLink, but for very specific messages (such as from the radio transmitter since it can't spare a lot of energy telling you a lot of things.
As for your goal using Open CV and cameras, there are a lot of ways you can do this. The best way (based on my experience) is to try and design your control (based on vision), and have it operate on the vehicle itself. I don't know of the setup you have, but video transmission is a bandwidth hog, but the other issue is the amount of computational power you have to sacrifice (you have to decide the lesser of the two evils). That said, I would suggest not using your computer in the loop for an autonmous control, and use whatever microcontroller you have (or find one that is able to do it).
I guess it depends how you plan to navigate...
Are you thinking of setting up just a directional input while flying in say loiter mode? You could do this with python and the joystick inputs reading from the OpenCV output...
Mav link data from what I have experience is just a serial connection... Like a old BBS AD AT modem. So as long as you have a system to connect with and shake and encode decode you are good to go...
I am am programmer but have yet to really dive into much of the pixhawk code, so I am sure their is a better way to integrate with the auto mode ...
I am sure someone will reply, if not I will get in there in the next week as I have several projects to integrate for our uses also
As a note, I have already browsed through these forums for several days, and I have already read the MAVLink for dummies article somewhere in this forum.
Hi Bill,
It's great to know that I'm not the only one at lost here. This is exactly my problem too. I'm new to pixhawk programming, but I've done for example communication via RS 232. Any help?