Developing a Web Application for Commercial Drones – big challenges, big prospects.

Hello everyone. 
  
Watching the sUAS Expo in San Francisco really got me thinking... we are at the beginning of a HUGE trend.    
  
The consensus is that the number one application for Drone technology over the next few years will be Agriculture. More specifically: aerial mapping, NDVI vegetation analysis and multi spectral imaging.  
  
This will most likely be the sector that will break social stigmas currently attached to 'drones', turning them from deadly killing machines into friendly farming tools comfortingly hovering over wheat fields. 
 
  
Over the past couple of years I have seen incredible advances in 1). Affordable Open Source Drone Technology (pixhawk) 2). Affordable Sensor Technology (you can get light high resolution cameras off the shelf) and 3). Image Processing Software.  
  
The problem I see is that for your average user, making sense of these individual products is daunting, even for those with more experience, integrating everything for efficient work flow is a major challenge.  
  
As I see it there are 3 key components that need to be integrated:  
  
 

  •     A GPS Enabled Drone  
  •     Camera Sensor (e.g. the Sony s110)  
  •     Image Processing Software  

How it should work:  
  
The end goal is to produce valuable data in the form of images. These images help Farmer Joe make his next decision. The key is to get the hardware to talk to each other through seamless integration.  
  
This is how I see the work-flow going...  
  
1 – A survey grid is drawn using the flight planning software overlayed on Google maps.  
 
 
2 – The drone takes off and moves to it's first waypoint in the survey grid via GPS signal.  
 
 
3 – The done reaches it's waypoint and triggers the camera to take a picture of the area underneath.  
 
 
4 – The onboard encrypted telemetry sends the image down to a laptop.  
 
 
5 – The image processing software on the laptop starts stitching the images into a mosaic.  
 
 
6 – This process repeats until the entire survey grid is completed, the drone lands and the stitched mosaic is automatically uploaded to the web-based application where the customer can analyze or share the image data in a 'Google earth' type interface.  


  
What we have:  
  
1 - We have a fantastic open source GPS enabled flight controller that is very powerful and fully programmable, thanks to 3DRobotics. The hardware only costs $279 with a GPS module and can already trigger cameras to take images and pre-defined waypoints.  
  
This flight controller works with the powerful open source 'mission planner' which can do the first three points above. You can program missions using GPS waypoints and even generate survey grids.  
 
2 - We also have low cost but high resolution cameras such as Sony cameras that offer remote API functionality: https://developer.sony.com/2013/11/29/how-to-develop-an-app-using-t...  
  
3 - Finally we have access to free image stitching software or Image Composite Editors (ICE): http://research.microsoft.com/en-us/um/redmond/groups/ivm/ICE/  
  
What needs to be done:  
  
All the components are there, we just need to get them talking to each other. This means designing a web based program that translates their languages into one user friendly application.  
  
The key being an emphasis on clean powerful yet simple work flow.  
  
I can imagine a user having a log-in account where he can click on one of his fields and this brings up a Google earth type  interface with the multi-spectral or NDVI  images overlayed on top.  
  
Take a look here to see what I mean: http://demo.terravion.com/#blocks 
  
That particular company uses full scale aircraft however and are orders of magnitude more expensive.  
  
What I really like is that you can select what overlay you want on the field be it infrared, NDVI or visual...  
 
The closest I have seen to this idea is San Francisco based drone-deploy: http://dronedeploy.com/ 
 
I am sure many of us are also looking forward to 3DR robotics releasing "drone-share" although I could not find much info on that... 
 
Ideally this should be integrated directly into the mission planner software. If we can get the telemetry to transmit live photos to the groundstation which can in turn perform real time image stitching and upload it to your account in the cloud for customers, then that would be a really powerful tool. 
  
So I am looking for any input you have to offer, any current projects or programs that you think I should take a look at. What is the best way to start the development process? 
  
This is a huge project with many challenges but even more potential. I am eager to hear what you think.
  
Best regards,  
  
Jethro Hazelhurst.

Views: 2355

Replies to This Discussion

Conservation Drones
Flight Riot - are both good 'how to' guides

Dronemapper - does what it says.

Excellent, thank you Michael.

Jethro, 

That's exactly right. We'll be releasing DroneShare in about two weeks. Then, in early Aug, we'll releasing the full mapping drones (copters and fixed wing) that have all the rest of what you've described. The only bit that they they don't do out of the box is the real-time wireless uploading of images, but that's what the DroneDeploy box adds. 

In short: you'll be happy in just a matter of weeks!

Thank you for the information Chris. Very exciting times.

Hi Jethro,

it is great that you approach the technical side of things in such a methodical way. However, I think we should also look at the whole process: once your farmer has acquired his NDVI map of his crop, how should he interpret it? Is wheat different then maize in reflection, and what do the different values (shades) mean? Again, on this all the technique and knowledge is there, but I am assuming that, like I do not know about farming, farmers do not know about remote sensing.

That is a very good point. 
 
I guess one of the services a company could provide would be educating farmers on how to read the data. 
 
Perhaps even better would be a software solution that allows the farmer to log into his account and view his field as mentioned, only this time he he can click a button and annotation appears that helps him understand what he is seeing. 
 
The annotation could be written there by a professional surveyor as part of the service... one of the benefits of being on the cloud. 
 
But you are right, it is a science on it's own.
Integrating the data automatically with farming other equipment is definitely the next step after. This could be the start of a new farming revolution, that is how powerful this stuff is...

My business has received a contract from the European Space Agency to work with a Remote Sensing company that will process imagery from Satellites and UAVs, then provide tiled RS imagery into a web based Precision Agriculture Mapping System that already provides Variable Rate Application maps using hisotrical data, such as Yield Maps and Soil Sampling maps. There are a few similar systems working in the UK that agronomists are using to create VRA maps for farmers to use, the key challenges are regulations / data formats & crop models.

The KORE project stands for Knowledge Observation Response Evaluation, the cycle will repeat.  During this Autumn & Winter, twenty agronomists and farmers will be trained and certified under the BNUC-S scheme, which is recognised by the UK's CAA and allows a person to operate a micro UAV and receive their Permission For Aerial Work, then in March 2015, ten sets of UAS will be provided to these trained operaters and the system will go commercial across Europe in 2016.

Looking forward to seeing what 3DR will bring out in the next few weeks and if we're allowed to buy it, given the export restrictions that seem to have been put in place?

I'm already working in this area and here are some tough issues:

  • Generating orthomaps is not as easy as stitching in ICE. You can only decently stitch orthorectified photos, and generating those requires some hefty programming power and more items in the toolchain. In my opinion, well tuned 3-axis brushless gimbals will go a long way to almost negating perspective variation in nadir photos, especially from slower flying multicopters. But you'll still need to orthorectify to get usable photomaps, it just means your success rate will be much higher.
  • NDVI photomaps take the problem a step further. There are various issues, but one of them is the issue of inter-period consistency and comparability. Our drones fly low enough to pick up lots of shadow, which especially a problem with crops like vineyards. You need to overmap an area many times to be able to build a composite image from which to calculate NDVI. Oh and this only solves the reflectance problem (mostly). How do you adjust for differential light intensity on different days? Farmers want to monitor changes so they can reverse-diagnose interventions and fine-tune them. Again, you need to do further post-processing to calibrate images. Some even feel that you need to calibrate each time using a hand-held chlorophyl meter.

So all in all, I like your approach and your infographic is very helpful, but the discussion only starts here, there are a plethora of issues to deal with.

The only thing holds me back about agriculture is that new satellites. I have made many meetings with many people and agency in my country(Turkiye/Turkey). They all thinking to send a satellite to space just for agriculture. This satellites will have required hardware for precision agriculture. So if i were you i would think twice to say " the number one application for Drone technology over the next few years will be Agriculture"

Well articulated John.

Several companies already provide services delivering weekly agricultural crop and crop condition information based on optical satllite data, the most widely developed in Europe include Farmstar, GeoSys, Talking Fields and Fieldlook.

Farmstar is owned and backed by Astrium now part of EADS, they started in France and now operate in Ukraine, Brazil and the USA. They tried to enter the UK but were defeated by the weather (cloud cover). As you can see our weather today and for the forseable future is cloudy with rain and a bit of sunshine.

EO satellites only pass every few days or even weeks, the new Sentienl 2 satellites will pass every 5 days, so the chances of it being cloud free when the satellite over passes at this time of year are very slim.

RSS

© 2019   Created by Chris Anderson.   Powered by

Badges  |  Report an Issue  |  Terms of Service