Hi. I was wondering how the whole MAVLink, Mission Planner, and PX4 Pixhawk all come together in terms of sending and receiving information. From what I've read so far, MAVLink is a protocol, Mission planner interprets incoming and outgoing messages, and the Pixhawk reads the messages that are sent from the Mission Planner from the radio transmitter. However, what do all these things really mean? 

My goal is to receive live video feed from my quadcopter using a gopro, use image processing (like OpenCV) to identify a target (like a red circle), and send movement commands to my quadcopter to follow this target when it moves. 

My main problem is being able to send the commands to my quadcopter. I know you're supposed to use something like rc override to send the command, but I am not sure where to start. Also, could anyone suggest whether this task is better done in C++, Python, or Java?

You need to be a member of diydrones to add comments!

Join diydrones

Email me when people reply –

Replies

  • I am also working on a similar project and interested to find out more.

    So far, I found a related blog post on controlling drones using Python through the Dronekit libraries.

    https://www.ddmckinnon.com/2015/12/30/idiots-guide-to-dronekit-pyth... 

    Idiot’s Guide to DroneKit-Python: A Journey to WHOZ CHILLIN
    DroneKit Python enables hackers to run code onboard the 3D Robotics Solo drone to drive interesting behaviors and applications. Learn DK Python here.
  • Hi Bill

    since we face a similar problem, did you finally get a solution to your request? Can you please share it with us?

  • MISSION PLANNER COPTER SIMULATION

    I'm trying to simulate a copter mission in MP. The python script that I'm writing must check for the existence of a file in some specific directory. This file is a QGC formated (.waypoints) file including the waypoints to follow and some other actions. In case of such file existence, the MP must make its pre-arming checks, then to arm the vehicle, to take off in "GUIDED" mode and afterthat to follow the trajectory described in QGC format in the  "trajectory.waypoints" file. The final command inluded in the above file will be an RTL command.

    Can someone please indicate to me where to look for the MP commands for the above actions or alternatively related examples?

  • HI Bill,

    Were you able to find a solution for your project, we are interested to know how it works.

  • I am actually working on an identical project (autonomous tracking using visual targets). I am trying to have a fixed wing aircraft search for the known target and then loiter around it while maintaining visual. The wall I am hitting right now is how to send gimbal commands to keep the target in the FOV. I know the syntax for sending RC commands to my gimbal through a specific RC channel, but I would prefer to send a Region of Interest (ROI), Point of Interest (POI), or point camera here command. Please let me know. Have you found syntax for sending MAVLink commands? that might be the answer for me.

    As for your problem, I have done similar work in the past where I put a vehicle in guided mode and set guided waypoints using python scripting through mission planner. For your application (assuming you are doing the processing on the ground station), I would set up a script with OpenCV to determine the target location relative to the vehicle, then set a new guided waypoint for the quad to fly to, to adjust the targets position in the image. To do this, one solution is to pass in the heading, lat, lon, and alt of the quad and pass out the new waypoint. see code below for more detail. Feel free to ask for clairification and good luck.

    lat = float(cs.lat)   #gets lat of quad

    lon=float(cs.lng)   #gets lon of quad

    alt=float(cs.alt)   #gets lon of quad

    heading=float(cs.groundcourse)   #gets gc of quad

    item=MissionPlanner.Utilities.Locationwp()  #Creates a waypoint

    MissionPlanner.Utilities.Locationwp.lat.SetValue(item,lat)  #sets lat of wp

    MissionPlanner.Utilities.Locationwp.lon.SetValue(item,lon)  #sets lon of wp

    MissionPlanner.Utilities.Locationwp.alt.SetValue(item,alt)  #sets altof wp

    MAV.setGuidedModeWP(item)   #Sets waypoint

  • I haven't dealt with MAVLink, Mission Planner, and Pixhawk as individual components, but I have done similar types of tasks with various other programs and protocols.  From the way you laid out their components I can venture a guess on how it "comes together."  MAVLink is the way things communicate (much like information over the internet which typically uses TCP/IP.  The mission planner is the core of all the functions, so it speaks using the MAVLink (the TCP/IP). The Pixhawk is similar to the MAVLink, but for very specific messages (such as from the radio transmitter since it can't spare a lot of energy telling you a lot of things.  

    As for your goal using Open CV and cameras, there are a lot of ways you can do this.  The best way (based on my experience) is to try and design your control (based on vision), and have it operate on the vehicle itself.  I don't know of the setup you have, but video transmission is a bandwidth hog, but the other issue is the amount of computational power you have to sacrifice (you have to decide the lesser of the two evils).  That said, I would suggest not using your computer in the loop for an autonmous control, and use whatever microcontroller you have (or find one that is able to do it).  

  • I guess it depends how you plan to navigate...

    Are you thinking of setting up just a directional input while flying in say loiter mode? You could do this with python and the joystick inputs reading from the OpenCV output...

    Mav link data from what I have experience is just a serial connection... Like a old BBS AD AT modem. So as long as you have a system to connect with and shake and encode decode you are good to go...

    I am am programmer but have yet to really dive into much of the pixhawk code, so I am sure their is a better way to integrate with the auto mode ... 

    I am sure someone will reply, if not I will get in there in the next week as I have several projects to integrate for our uses also

  • As a note, I have already browsed through these forums for several days, and I have already read the MAVLink for dummies article somewhere in this forum.

    • Hi Bill,

      It's great to know that I'm not the only one at lost here. This is exactly my problem too. I'm new to pixhawk programming, but I've done for example communication via RS 232. Any help?

       

This reply was deleted.

Activity

DIY Robocars via Twitter
RT @a1k0n: @SmallpixelCar @diyrobocars It's just something that's easy to track with chroma keying. I ended up using different colors on th…
10 hours ago
DIY Robocars via Twitter
10 hours ago
DIY Robocars via Twitter
RT @TinkerGen_: "The Tinkergen MARK ($199) is my new favorite starter robocar. It’s got everything — computer vision, deep learning, sensor…
Nov 23
DIY Robocars via Twitter
Nov 23
DIY Robocars via Twitter
RT @roboton_io: Join our FREE Sumo Competition 🤖🏆 👉 https://roboton.io/ranking/vsc2020 #sumo #robot #edtech #competition #games4ed https://t.co/WOx…
Nov 16
DIY Drones via Twitter
First impressions of Tinkergen MARK robocar https://ift.tt/36IeZHc
Nov 16
DIY Robocars via Twitter
Our review of the @TinkerGen_ MARK robocar, which is the best on the market right now https://diyrobocars.com/2020/11/15/first-impressions-of-tinkergen-mark-robocar/ https://t.co/ENIlU5SfZ2
Nov 15
DIY Robocars via Twitter
RT @Ingmar_Stapel: I have now explained the OpenBot project in great detail on my blog with 12 articles step by step. I hope you enjoy read…
Nov 15
DIY Robocars via Twitter
RT @DAVGtech: This is a must attend. Click the link, follow link to read the story, sign up. #chaos2020 #digitalconnection #digitalworld ht…
Nov 15
DIY Robocars via Twitter
RT @a1k0n: Got a new chassis for outdoor races (hobbyking Quantum Vandal) but I totally didn't expect that it might cause problems for my g…
Nov 11
DIY Drones via Twitter
First impressions of the Intel OpenBot https://ift.tt/36qkVV4
Nov 10
DIY Robocars via Twitter
Nov 9
DIY Robocars via Twitter
Excellent use of cardboard instead of 3D printing! https://twitter.com/Ingmar_Stapel/status/1324960595318333441
Nov 7
DIY Robocars via Twitter
RT @chr1sa: We've got a record 50 teams competing in this month's @DIYRobocars @donkey_car virtual AI car race. Starting today at 10:00am…
Nov 7
DIY Robocars via Twitter
Nov 6
DIY Robocars via Twitter
RT @a1k0n: Car's view, using a fisheye camera. The ceiling light tracking algorithm gave me some ideas to improve ConeSLAM, and having grou…
Nov 5
More…