Why are we starting to use companion computers?

Posted by Nicholas Witham on February 13, 2017 at 5:05am

So I've been looking into companion computers now for a while, and haven't really been able to find what all the hype is.

I just watched a video where you need an intel edison to read code to start, take off, fly somewhere, land then stop the aircraft. Excuse me if im wrong, but can't a pixhawk do all that on it's own anyway?

I know there are other bits of code that can let you find a red baloon, but what's that for?

Im not trying to discount the work anybody is putting into these applications, but I am just lost as to how I can use them.

I've built a number of quads, small and large, so I want to try something new and I know this is where it's all going, Im just trying to figure out why?

Here's a link to a previous post I made looking for guidance on hardware and firmware.

You need to be a member of diydrones to add comments!

Join diydrones

Email me when people reply –
Follow

Replies

  • Duffy February 20, 2017 at 11:32pm

    I suspect Nicholas doesn't need to be told the benefits of companion computers, in fact he has already chosen one...

  • Gavin Greenwood February 20, 2017 at 1:49pm

    Have anything that will simply count livestock in a open field? Also get with me on a better wifi system I specilize in super long range wireless internet! Cause I own an internet company!!!

    dhiraj said:

    IMHO there's two layers where you can tinker with drones, one is the core flight (autopilot layer) and the other one is applications which use drones for some task. Once your drone is ready to listen to your commands, what would you like it to do for you?
    Companion computer stuff is useful if you are working on the second layer. All the ecosystems that you mentioned provide collection of tools, APIs for autopilot, etc. so that you can get started and do some basic things such as video streaming.

    Now at this point you can enjoy creating your own companion computer image from scratch, building every module yourself. But thats not going to keep you occupied forever. Instead you may find it better to just get the ready to use image, setup your companion computer with autopilot and start developing / trying out applications.

    I'm a developer from FlytBase team and we have already worked on some visual following algorithms. We have used color, shape based detection and tracking algorithm as well as OpenTLD algorithm which can learn target to be tracked on the go. The visual detection and tracking part runs on companion computer. You get video stream on mobile phone app. User can select object to follow on mobile phone and drone will start following. FlytOS gives you access to these open APIs and drone navigation commands. 

    Here's a video for visual object tracking,

    There are many other examples which we are building on FlytOS such as visual object classification using machine learning, smart shots like solo, visual servoing for onboard camera, gps follow me, obstacle avoidance, etc. And these are not just built in applications, the source code for them is open so you can go ahead and shape them into whatever you like.

    It is possible to implement similar application with dronekit as well. Solo drone uses dronekit python to implement the smart shots.

    Connectivity for companion computers is certainly a key issue. Most of the modules running on them require transports which can speak IP (internet protocol). Also the rich data that companion computer need to share with ground stations requires more bandwidth than what conventional serial telemetry devices can provide. WiFi is certainly the best option. There's a discussion by Bill in companion computer group over wifi modules / routers. Using Rocket / TP link point to point outdoor routers is certainly a good long range solution. As you dive in you will be able to find out working solution for your particular scenario. E.g. We have been using an OpenWRT powered router on FlytPOD (its an autopilot + odroid xu4 + wifi router) since last 1 year, and we never faced any connectivity issue flying within 200 meters. And it extends upto 500 meters with any decent ground router. 

    In a nutshell companion computer give you tools to develop features provided in pretty much any commercial drone. So jump in, develop something cool and stick it to your phantom/mavic buddies.


    Nicholas Witham said:

    Don't get me wrong, I really enjoy building and figuring these things out, I actually prefer designing and building over flying.

    I get my satisfaction when I have a completely flying and tuned aircraft that does what I set out for it to do. For that reason I'd never buy a DJI. I like to tinker.

    I really just want ideas on what I can do with the companion computer, and how to do it. It seems there's lots of information out there, and different eco systems (Drone-Kit, Emlid, FlytOS, APSync, Dronesmith), It just makes it hard to know where to start and what to choose. I like the look of the Dronesmith board with it's slim form factor and onboard Edison, but (AFIK) edison can't do computer vision stuff.

    Really for now, I'd like to be able to build my aircraft, install the flight firmware, install the platform on the companion computer and just load the pre-made apps for that particular firmware. Maybe that's where we are headed? Will each of these eco-systems have an "app store" so you can do that pre-programmed "function".

    I also wonder about the connectivity of all this. Some of the eco systems are pushing 4G connection, some are pushing WIFI, and some are saying use a telemetry module. I mainly use RFD900s on my aircraft for 2-way telemetry (on a windows tablet) in tandem with an RC transmitter for manual flight. Do I need to change my gear to get these companion computers to work?

    I've seen some layouts where the telemetry unit is plugged into the pixhawk which is then plugged into the RPi, with no direct link between ground and the RPi. If I understand it correctly, this kind of set up would be for something like obstacle avoidance?

    What kind of equipment layout would I need if I wanted to do something like track "me" (or some other arbitrary thing) that I click on from the live video feed from my aircraft shown on my windows tablet? I know what the mavic can do, which is track the person/vehicle, plus avoid obstacles at the same time. Is that kind of thing achievable with an Edison or Odroid XU4, or something like that? My assumption is I'd have to ditch the RFD900, and use a WIFI link to send and receive data+video. Whats doing the computation then? Is it the tablet or the companion computer? What range are we getting with a WIFI module? I find it hard to find them any more than 100mW, so we ain't going too far on that, considering all the other 2.4GHz stuff around.

    I can already do follow me mode cos I have GPS in my windows tablet, but being able to do it by using computer vision would be cool. Having obstacle avoidance would be even cooler. Then I can stick it to my buddies that own phantoms and mavics and think they are awesome cos they can press buttons.

    Having all the other features like orbit and cable cam I guess would be trivial in comparison, and be done all on the same hardware layout?

  • Gavin Greenwood February 20, 2017 at 1:48pm

    I have a way that I don't use any RC signal at all only laptop and occasionally usb joystick over wifi 2GHz

    Richard Joy said:

    Glad to see that I am not the only one trying to work out what I can do with a companion computer. Experimented a bit with Dronekit and more recently installed Flytos on my RP3. My concern too is using wifi as the transport. Since this typically operates at 2.4G won't that interfere with the RC transmission ?. I guess I could use 5.8G wifi or 4G/LTE if I need the range. How does DJI use the same transport for everything and get such a long range ?.

  • Gavin Greenwood February 20, 2017 at 1:46pm

    Here's an example of a practical companion computer that takes any pixhawk or apm and makes it wifi capable in station or AP modes. 

    http://www.4stateuav.com/product/3dr-802-11n-wifi-telemetry-dongle/

  • Gavin Greenwood February 20, 2017 at 1:44pm

    Companion computers are super important in the furtherance of the tech that's onboard UAVs. From Wifi to LTE and even computer vision tasks are now possible due to companion computing. Awesome stuff if you ask me! 

  • Jon Croucher February 20, 2017 at 5:13am

    Python, OpenCV, and DroneKit are an incredible set of tools to expand the capabilities of the wonderful ArduPilot Flight Controller. They even give the old APM a new lease of life. A Raspberry Pi 3 with a camera is a great and economical way to start. You can even simulate the whole shebang on your home computer at no cost.

    The other big advantage is that you don't have to hack the magnificent  ArduPilot code to get some outstanding results.

    So to  wet your feet, download/install Python, DroneKit, DroneKit SITL, MAVProxy and be prepared to be hooked!

  • Nicholas Witham February 15, 2017 at 4:38pm
    I'm already fairly experienced at building a pixhawk powered quad, even built a heavy lift X8. I can tune them all fairly well, and have them doing planned missions within a couple of weeks, with constraints due to family commitments.
    I'm looking into these companion computers because I know that's where the community is headed, and I need to learn what all the rage is.
    I've found that FLYTOS is probably going to be my go to OS for now, but will probably try Drone Kit later.
    I will however get this new quad flying and tuned properly first, then add the CC, I'm just trying to find out all the information I need, just in case there's something different I should be doing from the very start of the build. As I mentioned previously I use RFD900s for my telemetry/communication in tandem with a 433MHz RC, but if I should be switching to another form of communication now, I need to know.
  • Developer
    John Arne Birkeland February 15, 2017 at 3:34pm

    Don't try to do everything at once.

    Get the drone up and flying on a regular Pixhawk. This should keep you busy for many months, getting to know the system doing missions etc.

    Then and only then should you start looking at taking the next step adding a companion computer, and getting into bleeding edge stuff like computer vision, SLAM, machine learning etc.

    An example of companion computer/higher computing that is actually useful and used today is the DJI Phantom/Mavic precision landing, where the drone takes a picture of the ground during takeoff and uses this to land at the same spot again. And of course obstacle avoidance which is the holly grail for automated flight.

  • Richard Joy February 14, 2017 at 6:42am

    Glad to see that I am not the only one trying to work out what I can do with a companion computer. Experimented a bit with Dronekit and more recently installed Flytos on my RP3. My concern too is using wifi as the transport. Since this typically operates at 2.4G won't that interfere with the RC transmission ?. I guess I could use 5.8G wifi or 4G/LTE if I need the range. How does DJI use the same transport for everything and get such a long range ?.

  • dhiraj February 14, 2017 at 4:36am

    IMHO there's two layers where you can tinker with drones, one is the core flight (autopilot layer) and the other one is applications which use drones for some task. Once your drone is ready to listen to your commands, what would you like it to do for you?
    Companion computer stuff is useful if you are working on the second layer. All the ecosystems that you mentioned provide collection of tools, APIs for autopilot, etc. so that you can get started and do some basic things such as video streaming.

    Now at this point you can enjoy creating your own companion computer image from scratch, building every module yourself. But thats not going to keep you occupied forever. Instead you may find it better to just get the ready to use image, setup your companion computer with autopilot and start developing / trying out applications.

    I'm a developer from FlytBase team and we have already worked on some visual following algorithms. We have used color, shape based detection and tracking algorithm as well as OpenTLD algorithm which can learn target to be tracked on the go. The visual detection and tracking part runs on companion computer. You get video stream on mobile phone app. User can select object to follow on mobile phone and drone will start following. FlytOS gives you access to these open APIs and drone navigation commands. 

    Here's a video for visual object tracking,

    There are many other examples which we are building on FlytOS such as visual object classification using machine learning, smart shots like solo, visual servoing for onboard camera, gps follow me, obstacle avoidance, etc. And these are not just built in applications, the source code for them is open so you can go ahead and shape them into whatever you like.

    It is possible to implement similar application with dronekit as well. Solo drone uses dronekit python to implement the smart shots.

    Connectivity for companion computers is certainly a key issue. Most of the modules running on them require transports which can speak IP (internet protocol). Also the rich data that companion computer need to share with ground stations requires more bandwidth than what conventional serial telemetry devices can provide. WiFi is certainly the best option. There's a discussion by Bill in companion computer group over wifi modules / routers. Using Rocket / TP link point to point outdoor routers is certainly a good long range solution. As you dive in you will be able to find out working solution for your particular scenario. E.g. We have been using an OpenWRT powered router on FlytPOD (its an autopilot + odroid xu4 + wifi router) since last 1 year, and we never faced any connectivity issue flying within 200 meters. And it extends upto 500 meters with any decent ground router. 

    In a nutshell companion computer give you tools to develop features provided in pretty much any commercial drone. So jump in, develop something cool and stick it to your phantom/mavic buddies.


    Nicholas Witham said:

    Don't get me wrong, I really enjoy building and figuring these things out, I actually prefer designing and building over flying.

    I get my satisfaction when I have a completely flying and tuned aircraft that does what I set out for it to do. For that reason I'd never buy a DJI. I like to tinker.

    I really just want ideas on what I can do with the companion computer, and how to do it. It seems there's lots of information out there, and different eco systems (Drone-Kit, Emlid, FlytOS, APSync, Dronesmith), It just makes it hard to know where to start and what to choose. I like the look of the Dronesmith board with it's slim form factor and onboard Edison, but (AFIK) edison can't do computer vision stuff.

    Really for now, I'd like to be able to build my aircraft, install the flight firmware, install the platform on the companion computer and just load the pre-made apps for that particular firmware. Maybe that's where we are headed? Will each of these eco-systems have an "app store" so you can do that pre-programmed "function".

    I also wonder about the connectivity of all this. Some of the eco systems are pushing 4G connection, some are pushing WIFI, and some are saying use a telemetry module. I mainly use RFD900s on my aircraft for 2-way telemetry (on a windows tablet) in tandem with an RC transmitter for manual flight. Do I need to change my gear to get these companion computers to work?

    I've seen some layouts where the telemetry unit is plugged into the pixhawk which is then plugged into the RPi, with no direct link between ground and the RPi. If I understand it correctly, this kind of set up would be for something like obstacle avoidance?

    What kind of equipment layout would I need if I wanted to do something like track "me" (or some other arbitrary thing) that I click on from the live video feed from my aircraft shown on my windows tablet? I know what the mavic can do, which is track the person/vehicle, plus avoid obstacles at the same time. Is that kind of thing achievable with an Edison or Odroid XU4, or something like that? My assumption is I'd have to ditch the RFD900, and use a WIFI link to send and receive data+video. Whats doing the computation then? Is it the tablet or the companion computer? What range are we getting with a WIFI module? I find it hard to find them any more than 100mW, so we ain't going too far on that, considering all the other 2.4GHz stuff around.

    I can already do follow me mode cos I have GPS in my windows tablet, but being able to do it by using computer vision would be cool. Having obstacle avoidance would be even cooler. Then I can stick it to my buddies that own phantoms and mavics and think they are awesome cos they can press buttons.

    Having all the other features like orbit and cable cam I guess would be trivial in comparison, and be done all on the same hardware layout?

This reply was deleted.