3D Robotics

Let Skynet Become Self-Aware!

3689666587?profile=original

Here's a crosspost of my article this morning in Hackaday. We'll see how many people get the Terminator joke. 

Not so long ago, it was hard to fly. Forget actual manned aircraft and pilots licenses; even flying model aircraft required hours of practice, often under the tutelage of a master at a flying field. But along with that training came an education in the rules of safe flight, including flying at a designated airfield and watching out for obstacles.

We accidentally messed that up. We in the drone industry made aircraft super easy to fly — perhaps too easy to fly. Thanks to smart autopilots and GPS, you can open a box, download an app and press “take off”. The copter will dutifully rise into the air and wait there for further instructions — no skill required. And it will do this even if you happen to be in an NFL football stadium in the middle of a big game. Or near an airport. Or in the midst of a forest fire.

The problem is that along with taking training out of the process of flying a drone, we inadvertently also took out the education process of learning about safe and responsible flight. Sure, we drone manufacturers include all sorts of warning and advisories in our instructions manual (which people don’t read) and our apps (which they swipe past), and companies such as DJI and my own 3DR include basic “geofencing” restrictions to try to keep operators below 400 feet and within “visual line of sight”. But it’s not enough.

Every day there are more reports of drone operators getting past these restrictions and flying near jetliners, crashing into stadiums, and interfering with first responders. So far it hasn’t ended in tragedy, but the way things are going it eventually will. And in the meantime, it’s making drones increasingly controversial and even feared. I call this epidemic of (mostly inadvertent) bad behavior “mass jackassery”. As drones go mass market, the odds of people doing dumb things with them reach the singularity of certainty.

We’ve got to do something about this before governments do it for us, with restrictions that catch the many good uses of drones in the crossfire. The reality is that most drone operators who get in trouble aren’t malicious and may not even know that what they’re doing is irresponsible or even illegal. Who can blame them? It’s devilishly hard to understand the patchwork quilt of federal, state and local regulations and guidelines, which change by the day and even the hour based on “airspace deconfliction” rules and FAA alerts written for licensed pilots and air traffic control. Many drone owners don’t even know that such rules exist.

Drones Themselves Should Know Rules of Each Area

Fortunately, they don’t have to. Our drones can be even smarter — smart enough to know where they should and shouldn’t fly. Because modern drones are connected to phones, they’re also connected to the cloud. Every time you open their app, that app can check online to find appropriate rules for flight where you are, right then and there.

Here’s how it works. The app sends four data fields to a cloud service: Who (operator identifier), What (aircraft identifier), Where (GPS and altitude position) and When (either right now or a scheduled time in the case of autonomous missions). The cloud service then returns a “red light” (flight not allowed), a “green light” (flight allowed, with basic restrictions such as a 400 feet altitude ceiling), or “yellow light” (additional restrictions or warnings, which can be explained to the operator in context and at the point of use).

image01

Right now industry groups such as the Dronecode Foundation, the Small UAV Coalition (I help lead both of them, but this essay just reflects my own personal views) and individual manufacturers such as 3DR and DJI are working on these “safe flight” standards and APIs. Meanwhile, a number of companies such as Airmap and Skyward are building the cloud services to provide the up-to-date third-party data layer that any manufacturer can use. It will start with static no-fly zone data such as proximity to airports, US national parks and other banned airspace such as Washington DC. But it will quickly add dynamic data, too, such as forest fires, public events, and proximity to other aircraft.

(For more on this, you can read a white paper from one of the Dronecode working groups here and higher level description here.)

There’s Always a Catch

Of course, this system isn’t perfect. It’s only as good as the data it uses, which is still pretty patchy worldwide, and the ways that the manufacturers implement those restrictions. Some drone makers may choose to treat any area five miles from an airport as a hard ban and prohibit all flight in that zone, even at the cost of furious customers who had no idea they were five miles from an airport when they bought that toy at Wal-mart (nor do they think it should matter, since it’s just a “toy”). Other manufacturers may choose to make a more graduated restriction for the sake of user friendliness, adding a level of nuance that is not in the FAA regulation. They might ban, say, flight one mile from an airport, but only limit flight beyond that to something like 150ft of altitude (essentially backyard-level flying).

That’s a reasonable first step. But the ultimate safe flight system would go a lot further. It would essentially extend the international air traffic control system to millions of aircraft (there are already a million consumer drones in the air) flown by everything from children to Amazon. The only way to do that is to let the drones regulate themselves (yes, let Skynet become self-aware).

Peer-to-peer Air Traffic Control

There’s a precedent for such peer-to-peer air traffic control: WiFI. Back in the 1980s, the FCC released spectrum in the 2.4 Ghz band for unlicensed use. A decade later, the first 802.11 standards for Wifi were released, which was based on some principles that have application to drones, too.

  1. The airspace used is not otherwise occupied by commercial operators
  2. The potential for harm is low (in the case of WiFi, low transmission power. In the case of drones, low kinetic energy due to the weight restrictions of the “micro” category)
  3. The technology has the capability to self-”deconflict” the airspace by observing what else is using it and picking a channel/path that avoids collisions.

That “open spectrum” sandbox that the FCC created also created a massive new industry around WiFi. It put wireless in the hands of everyone and routed around the previous monopoly owners of the spectrum, cellphone carriers and media companies. The rest was history.

We can do the same thing with drones. Let’s create an innovation “sandbox” with de minimus regulatory barriers for small UAVs flying within very constrained environments. The parameters of the sandbox could be almost anything, as long as they’re clear, but it should be kinetic energy and range based (a limit of 2kg and 20m/s at 100m altitude and 1,000m range within visual line of sight would be a good starting point).

As in the case of open spectrum, in relatively low risk applications, such as micro-drones, technology can be allowed to “self-deconflict the airspace” without the need for monopoly exclusions such as exclusive licences or regulatory permits. How? By letting the drones report their position using the same cellphone networks they used to get permission to fly in the first place. The FAA already has a standard for this, called ADS-B, which is based on transponders in each aircraft reporting their position. But those transponders are expensive and unnecessary for small drones, which already know their position and are connected to the cloud. Instead, they can use “virtual ADS-B” to report their position via their cell network connections, and that data can be injected into the same cloud data services they used to check if their flight was safe in the first place.

Once this works, we’ll have a revolution. What WiFi did the telecoms industry, autonomous, cloud-connected drones can do to the aerospace industry. We can occupy the skies, and do it safely. Technology can solve the problems it creates.


E-mail me when people leave their comments –

You need to be a member of diydrones to add comments!

Join diydrones

Comments

  • Yes, speed reduced to around 200mph by 5000 feet.

    I also agree, there is no chance to prevent intentional misuse.

    However, that is not the goal here.  The goal is to prevent innocent mistakes, and intentional mass jackassery.  Many users are not aware of how close they are to an airport, helipad, etc.  5 miles as the crow flies is much further than road distances which is what many people base their spatial perception on.  Then there are the people who intentionally do stupid things, but are also not capable of finding their way around the system being proposed.

    I agree, that some of this needs to be handled locally on the device, and not rely on a network connection.

  • Ok Rob.

    On your decent did you happen to notice your speed at around 3-5000ft? I'd say there would be a massive decrease in speed once they deploy flaps and gear whilst they descend into typical UAV altitudes.

    I completely agree with your point that there is a risk of some users intentionally operating at unsafe altitudes. But my point was that it will be impossible to stop anyone from intentionally doing so, they could just resort to another product or ditch the autopilot completely to do so. 

    As such I cannot see the here proposed "Skynet" system in any way reduce intentional misuse. It will be restricted only in managing unintentional misuse (from an inexperienced user etc). Further to this I fail to see any tangible benefit to the complicated and extensive Skynet system, beyond what can already can be achieved using exclusion zones and altitude limitations on the existing Pixhawk by way of a fairly moderate Mission Planner software and PIxhawk firmeare upgrade. The micro SD card has enough capacity to accommodate this. No 4G/LTE or Skynet required. 

    I just can't see the benefits atm, but am happy to be proven otherwise.

    Regards JB

  • JB, we're definitely talking past eachother.  First, earthpatrol maligned the idea of "skynet", by suggesting technology isn't required to solve the problem because birds manage to not crash into eachother using natural means.  To which I pointed out, birds do not fly at 500 mph in clouds, which is why technology is needed.  You then tried to make the case that we needn't consider collision avoidance at 500mph, because those speeds only happen at very high altitudes that drones can't reach.  I posted a video of an FPV flight at 17,000 feet.  It could just as easily have been a drone.

    BTW, yesterday I snapped a picture of our inflight data display showing us descending into Chicago doing 460mph at just 10,000 feet. 

  • One more thing! Once this is implemented I see no reason for commercial restrictions. Fight the FAA on that issue once we've proven we can be safe.

  • Operations on the ground are 2D and have no analog to air traffic which is an order of magnitude more complicated.

    I see the solution as simple built in no fly zones I agree that 5 miles is too much for most airports and a cone of altitude is a good solution.

    Laser range finder altitude limits

    Thinking about it LOS can be handled by GPS geofencing

    None of these have to be perfect they just need to work good enough to prove we care about safety.

    These can be implemented right now! Without any skynet.

    Add ADS-B as a future avoidance feature.

    If the first 3 item are implemented right now, we can avoid MOST of the issues.

    No Fly zones don't need a constant internet connection to work.

    This fixes everything except first responders and ADS-B will eventually solve that.

    If you want more privileges get a license something equivalent to a US sport pilot license which is easy to get anyway and prove you can be responsible.  

  • Hey Rob buddy

    Nice video! I think we might be talking past each other a little bit as to who exactly will be limited by the proposed "Skynet" safety features. As your video proves there's nothing stopping any "enthusiast hobbyist" to do a FPV and RC flight. It's easy to get in "trouble" even without GPS or autopilot, just with long range FPV and RC control. So if there is NO autopilot there would be NO "Skynet" functionality at all. The video proves that without a doubt! ;-)

    The only persons "Skynet" will affect is those who "unintentionally" break the rules and that fly an autopilot based aircraft with 4G/telmetry. We will never be able to regulate RC out of existence, so intentional breaking of the rules will always be around, until we start shooting the nasties out of the air! ;-)

    I'm aware of the real situation of what "hobbyist" can do. BLOS is common operations for our team, but we at least post spotters/pilots along the way and stay within altitude restrictions and far away from other airspace users and airports. From my perspective, the only thing the proposed "Skynet" can do is try to regulate some of the not so safety conscience RTF users from breaking the rules and in doing so endanger others and themselves.

    So given the confines of what users "Skynet" will impact, (namely RTF users) I'm wondering what other useful and worthwhile functions should it have considering my reasoning in other posts on this thread like:

    1. Exclusion zones around sensitive areas like airports. -  Pre programmed on PXH SD card and updated with MP etc. an auto geofence could also be included, but it all will only work with AP and GPS lock. Typically the user will want web access to update maps for mission planning, or pre-load them on MP for remote use, so data should be available min 95% of the time (For horizontal separation to other airspace users)
    2. Altitude restrictions - which can be done from home alt should no terrain data be available via MP terrain update for mission (for Vertical separation)
    3. Reduce the RC range and kinetic energy of UAV's for RTF users - licence and train the rest
    4. The limited amount of manned aircraft operating within proximity to the max UAV altitude and outside exclusion areas defined in the PXH in 1 & 2 above
    5. The limited velocity these manned aircraft fly at these altitudes (which makes a short range RF solution more feasible than a Cell/4G based one like 3km RF range + 30sec response and works everywhere without web) 
    6. That a web based 4G networked Skynet is prone to latency, accuracy, connectivity issues and is massively complex to orchestrate, maintain, fund, energize and will RF/data pollute web and airwaves.

    Given that both 1 and 2 above can be implemented on the PXH right now with a bit (!) of coding but no extra hardware requirements, and offers by far the most substantial benefits for safety, and given that a "Self-aware Skynet" flight terminator system is unlikely to yield any better results, apart from enslaving the population to web overlord control, then what exactly are the benefits of such a system, apart from being maybe a pre-mature nerdgasm? ;-) (No offence meant to anyone BTW! Just UAV fanatics talking here!)

    Sure it's technically possible, but hey even time travel is possible scientifically, so long it's forwards....but does that mean it solves the safety concerns for airspace users and ground dwellers? Regardless of how we propose to fix the RTF user problem, using a derivative of Skynet it will always remain reactive NOT proactive.

    As pointed out in the original post where it says:

    "We accidentally messed that up. We in the drone industry made aircraft super easy to fly — perhaps too easy to fly. Thanks to smart autopilots and GPS, you can open a box, download an app and press “take off”."

    And that I'm afraid is the only real solution to this problem; be proactive and teach people to become responsible users, especially for small affordable UAV's.

    It worked for cars and it works for bikes without licencing as well. Even though with remote control, for most users the physical disconnect to the craft makes them feel invincible, we need to persuade them otherwise. Ask any military drone operator who's shot live rounds how that feels. Failing that I fear that we will become the users needing to wear full nappies on our heads, should UAV legislation devolve into a nanny state that enables any monkey to press a button. In that regard I think CASA here in Australia got the balance right, licence and test the "car" operators, so we just need to teach everyone to ride the UAV "bikes" safely. 

    Best Regards

    JB

  • Moderator

    Tricky video that Rob, lots to like and lots to dislike. This is the sort of thing where if the five FAA test sites were free and simple to use folks could legally test this sort of thing.

  • JB, I am of course referring to manned aircraft flying around at 500 mph.  And I am talking about the very real situation we have, today, with hobbyists flying at very high altitudes.  You're not really going to make me post another video are you?

  • @ Earthpatrol

    With your movie references are you hinting at humans inability to identify excessive mis-guided progress into stupidity? I'd have to agree. Sometimes the solutions involve less, not more complexity, by simply enabling humanity to act responsibly ;-)

    --

    @ Rob

    The statement that we need to have a system that can control a 500MPH aircraft horizontally through cloud is a bit of a red herring. The question is not if there are such birds, rather if there will be such UAV's! Passenger aircraft already have sufficient (but still improvable) flight management and controls. A UAV with such speed performance would quickly be BLOS (say 5-6seconds!), which for all intents and purposes shouldn't be considered for "hobby" use at all anyway. If you're worried about an aircraft approaching your UAV at cruise, then I'd say what on earth are you doing up at cruise altitude 30,000ft with a hobby, out of the box, RTF UAV?? ;-)

    --

    @all

    I think that there needs to be consideration given to how aircraft, in particular passenger aircraft operate, and tier each level of aircraft back from there to establish which aircraft will in fact need to interact with UAV's, and what their individual flight envelopes are.

    Aircraft landing or taking off are at much more reasonable speeds of around 160MPH, at which velocity even short range 3km RF transponder would give either pilot around 30 seconds to detect and avoid. If we exclude UAV's with a 5mile or similar buffer the glide slope of approaching aircraft will maintain significant vertical (beyond UAV climb rate performance) and horizontal clearance to UAV operations. If we limit UAV altitudes to under 400ft, then we typically have both vertical and horizontal separation to cruise altitudes of some 5-6miles at all times, essentially putting passenger aircraft that are in normal operation out of harms way.

    Then of course there are GA (General Aviation) users to consider, at which point the situation becomes more complex as they could well be sharing airspace with UAV's. However, there are (nearly) no GA aircraft that go 500MPH (220m/s)  let alone would be able to do so at low altitude under 400ft (apart from business jets that typically have flight plan and high cruise alt). I'd say on average GA aircraft at close to that altitude would be either on approach or climb around an airport, or possibly landing strip out in the back country, in either situation they'd be operating at even slower velocities of around 70MPH which allows for some 60second response times on 3km RF range. If airport 5 mile exclusion applies, or if back country the risk reduces, (with limited GA and simultaneous UAV traffic in operation and also maybe no 4G anyway), then neither would significantly benefit from a such flight control systems as proposed.

    What I'm alluding to, is that we, in fact do not want to interfere with passenger traffic at all, nor be giving UAV ops the opportunity to do so. The exclusion and altitude controls are the best method for doing so and removing those to enable closer proximity UAV flights would be entirely negligent IMHO. So if those rules are in place we only need to deal with GA. There will inevitability be some limited interaction, in particular with joy flights and heli operations but these aircraft all don't have high velocities to contend with, which means, as above, even fairly short range 2-3km RF could suffice to maintain in-air separation. So if a UAV has a local UAV/GA onboard exclusion areas and altitude database, and RF based detection system, possibly adopting the ADSB standards, then even limited range RF transponders could be enough to add another level of protection for low altitude users.

    Overall, I'd would prefer to have a distributed system rather than feeding everything through a web based centralised system, that will suffer significantly from latency, accuracy and connectivity issues, let alone be massively complex to orchestrate, maintain and fund. lso do we all want to suffer being grounded because of no cell reception or because the network is down? There are also bandwidth management and even energy (in RF pollution, internet transfer, servers etc etc) to consider, that would all be avoided if one would adopt a de-centralised and localised system using fairly robust and simple RF detection systems. There's simply no need and no benefit for the Skynet approach just for safety reasons. "Skynet" only makes sense if you also perpetuate a UAV wireless mesh network to give both air and ground users interconnectivity...essentially building out your own wireless network as you go with ground users also being a part of that network, through cars, trucks buses, homes, schools etc. But that is only really possible using SDR technology. 

    That way the UAV/internet guardians don't guards... ;-)

This reply was deleted.