This video is a showcase in which I provide an overview of the capabilities and usage of the 14 additional autoflight modes which will come with the upcoming release of FlightZoomer (version 2.0).
In the second half of the video I show the shortest possible flight during which any of the 14 modes is used.
In case you want to learn more about these autopilot modes I have provided another new video, which actually is a training video. It takes more than 40 minutes and explains the new autpilot modes one after another in great detail:
Here is the handout to the training video, which provides additional information in written form:
FlightZoomer is much more than an autoflight control system; but in this post I specifically address the autopilot capabilities that are as-yet untapped by current Ground Control Systems. For example:
- Works as an overlay onto the Ardupilot software using the GUIDED mode (copter only so far, but Plane would be doable quite easy as well)
- The FlightZoomer autoflight features are built as a composite solution which integrates the user layer (the groundstation), the communication layer (the cellular network) and the actual flight controlling unit (an onboard smartphone functioning as a companion computer) to get a transparent and robust flight control capability.
- Each command is stored and actually executed autonomously by the onboard companion computer, mitigating risks associated with reduced cellular network connectivity and subsequent impacts to the flight
- Failsafe integrated into the companion computer executable code
- The MCP (Mode Control Panel) not only is a comprehensive tool to control the flight trajectory via a handy touch screen, but also to shows the current state of the autopilot (Pixhawk)
- As there always may be short packet delays in cellular IP networks, care is taken to ensure each and every command is technically acknowledged and the acknowledgment state is also displayed on the MCP
- The autopilot control screens and switches are patterned after the autoflight systems of the Boeing 787 Dreamlike- considered the current state of the art
- As shown on the following diagram there are 8 basic autoflight modes (trajectory control), 3 radio navigation modes (using simulated VORs and ILSs) and 3 flight plan modes (similar to the existing APM AUTO mode):
More information can also be found in some of my earlier about FlightZoomer posts here:
http://diydrones.com/profiles/blogs/swissair-124-you-are-cleared-for-ils-runway-15-flightzoomer-now
http://diydrones.com/profiles/blogs/flightzoomer-is-now-generally-available-see-how-the-look-feel
A word about the usage of Windows Phone-8® OS
I am fully aware that I have built FlightZoomer on the least prevailing phone OS. The reason is that I can achieve the highest productivity using the Windows programming stack and I can leverage a lot between relay server and the phone apps. I would simply not have reached this level of capability and application maturity this quickly.
For the same reason (time to market) I am reluctant to begin porting the apps e.g. to Android. There would be considerable time “lost” without gaining new features (I will port the apps however as Windows Universal apps: these support Windows 8 and 10 across all device types from phones to tablets to laptops to desktops).
But, on the other hand - let me argue this way: we fly around with copters that cost thousands of dollars mostly consisting of just proprietary hardware. We buy proprietary hardware for RC systems, ESCs, FPV stuff, cameras, gimbals, OSDs or even whole RTF copters. So for non-Windows Phone users using FlightZoomer would mean just buying another two proprietary hardware devices. These cost less than almost any of the aforementioned hardware.
Or – to say it differently – imagine I would build and offer fully proprietary hardware for FlightZoomer, like an onboard companion computer with touchscreen & sensors & cellular G3, G4 connectivity & Wi-Fi & Bluetooth & camera & autonomous power supply for 50 bucks (that´s how much I paid for my "companion computer"). Everybody would cry hurray! And, with FlightZoomer you don’t need to build complicated Python scripts and debug errors….
Likewise, what if I would offer a proprietary groundstation with 6, 7, 8 or 10 inch touchscreen, which supports all the FlightZoomer features for 100 to 150 bucks? I see people investing much more in groundstations that are not as capable.
Comments
Hi Philip
Here my replies:
1. Yes your assumptions are correct. The FlightZoomer autopilot modes basically require a particular flight vector at any time, so the appropriate MAVLINK commands are sent with a fixed frequency. The tricky part is e.g. when in the middle of an altitude capture transition a new target altitude is received, which requires to reverse the ongoing transition. The same applies when the ILS needs to be captured e.g. while a turn is flown.
FlightZoomer does not generate waypoints, but directly sets either target positions or velocities (depending on the required flight trajectory).
2. While the FlightZoomer autopilot is active, the FC is put in GUIDED mode, which means that the FC operates under the control of the smartphone. The FC will simply execute whatever it receives. FlightZoomer is master, the FC slave. You can see it as nested controller. The inner (and admittedly highly critical) loop is done by the FC, while the outer control loop, which follows the target various values, is done in FlightZoomer (at a much slower pace).
3. All apps of FlightZoomer are written in C#. That way I was able to leverage my know how the best.
4. The project is closed source and I am the only developer. Dave Folts is helping me with the documentation.
About the goals of FlightZoomer:
My personal goal was really something very simple: let RC aircraft fly realistic. Have not only "Scale" looking aircraft but also fly them "Scale". I have not seen a single instance of RC flight operations, which even remotely imitate how real aircraft are flown. FlightZoomer is a huge step forward in that regard.
It can also be seen as a blend between a PC flight simulator and a RC aircraft. In that light also the ATC feature has to be seen. In FSX there is ATC who directs you more or less realistic.
Of course the present set of features can be the foundation for many many further developments. Here I am somewhat limited with my resources. I need to stay focused extremely and have to define realistic development steps.
Next focus will be to finally release version 2 (apps and docu).
After that I believe I will port the apps to Windows Universal and, as Xamarin is free now, I think I will have a good look whether and how an Android version would be doable.
My personal background:
I graduated from a Swiss UAS in 1996 as Electric Engineer. Since then I have worked in the area of professional software development (employed by a Swiss Bank currently).
My skill profile (electronics, software development, physics, mathematics, design and artwork) is shaped nicely to run a project like FlightZoomer. Any of the mentioned areas proved as essential.
Hi Martin,
Thank you for the great answers.
And thanks for clarifying about the SOP for some full scale ILS landings. Yes, more accurate altitude sensors on the drone would be needed to know exactly when to flare.
Yes, I was watching an earlier FlightZoomer video from 1 year ago that showed both a map view and a camera view during an ILS approach. Thanks for clarifying that it was showing a manual approach. It would be fun to see that same combination video with your newer system during an automated landing approach.
A few more random questions, please.
1) I'm wondering how much vector calculation work is being done on the companion computer. Is it correct to assume that it is monitoring mavlink position messages from the FC so that it can calculate things like: the vehicle's track, when waypoints are reached, when ILS entry points are reached, etc? These calculations are then used to determine what commands to send next to the FC, correct? For example, FlightZoomer has nice features to control the turn rate and the climb rate. Does FlightZoomer generate a set of waypoints along the desired trajectory that are sent to the FC in order to achieve those turn and climb rates?
2) Since the companion computer is acting as an autopilot, and the APM is an autopilot, are there any concerns about control conflicts?
3) You mentioned C++ above. Are all 3 software components written in C++ or are you using other languages?
4) Is the project open or closed source and are you the sole developer?
5) Approximately how much data on a phone's data plan might be used for a 15 minute flight?
Once government agencies open the skies to long range UAV operations, I can see a project like FlightZoomer playing a big role. The fact that the interface is designed around a certified aircraft system, as well as other aircraft industry standards for ILS, guidance, control, and even procedures, is a big advantage, imo. Darius Jack also mentioned its possible benefits for things like training. It seems that your ATC features are at least partly aimed at training, no? It's terrific stuff!
I'm curious... Can you enlighten us a bit more about the project's long term goals and your professional background?
Congrats again on a great project!
Hi Philip
Thx for the feedback!
Here the answers:
- The navigation database is not physically located on the GCS but on the relay server. It is replicated to the GCS each time the app starts. The relay server has GUI facilities to maintain the navigation database (see the version 1.5 documentation on flightzoomer.com).
- Yes, the complete flightplan with all data is sent to the companion computer when a route is activated (by pressing EXEC on the FMS page).
- Yes, the autopilot user-layer-commands are sent to the companion computer. From there various MAVLINK commands (SET_POS_TARGET_GLOBAL_INT, SET_POS_TARGET_LOCAL_NED, ...) are sent to the FC to emulate the active autopilot mode.
- No terrain altitude following feature is available at the moment.
- The implemented flying procedures simulate the workload of a real pilot. As in reality most action happens during the climb and during the approach. For the normal route flying, FlightZoomer supports the LNAV and VNAV modes, which basically fly the aircraft along the route laterally and vertically fully autonomously (=hands off).
- It would flare, however I doubt that system calculates/measures the current altitude accurately enough to properly flare at the runway altitude. I would define the ILS with a rather shallow descend gradient (maybe 5°) and just follow the glideslope into the ground. This is b.t.w. SOP (standard operation) in full scale aviation under some conditions. That way fully automatic landings should work (I have not tested fixed wing ops though so far).
- You mean the ILS approach video from a year ago? That was version 1.5 of FlightZoomer, where I had the instrument feedback but had to fly manually! Only one of maybe 7 approaches have turned out so nicely as shown in the video. Using the autopilot with version 2 I now get consistent and accurate approaches (see the second ILS approach video from 3 months ago).
@Martin I really like what you've created and have a few questions, if you don't mind.
- Waypoints are configured and stored in a database on the GCS (Windows phone), correct?
- Are the set of waypoints that define a mission sent as a complete set to the companion computer before the mission?
- When flying a waypoint mission or an ILS approach, are the position/altitude updates needed to achieve the desired track sent to the autopilot from the companion computer in real time (i.e., using mavlink GLOBAL_POSITION_INT or some other mavlink message type)? I'm wondering where the mission data is stored and which FlightZoomer component first initiates the command sent to the autopilot.
- Does FlightZoomer have the ability to maintain a minimum height over the ground, based on a terrain database and the mission plan?
- The realtime pilot workload seems to be relatively high (i.e., to change altitudes, headings, modes [VOR, ILS, APP, etc], etc. Is this something that can be fully pre-defined and loaded before a flight? The goal would be to setup the full mission, verify it via a simulation, and then fly the mission (mostly) hands-off.
5) Your demo video mentioned that altitude transition points are automatically smoothed out. For a fixed wing landing, would the also aircraft flare out during landing/touchdown?
6) One of the demo videos showed an ILS landing that ended up a bit to the right of the runway. Was this caused by aircraft GPS position errors, runway location errors, or a combination of both?
Congrats again on creating a beautiful and powerful system!
@Chris: First the shorter reply ;-)
Qt could be an option (C++ was my first OO language). As would Xamarin.
But as my solution is only half about GUI's, and the other half is dealing with OS dependent stuff like UDP communication, bluetooth, WiFi, cameras, sensors and so on, I therefore still fear a significant effort and also still diverging code bases for different OS's.
@Darius:
FlightZoomer is not intended for operations which have been banned. Of course anybody at any time must respect local regulatory rules. Just because FlightZoomer offers better ergonomics and more flexible flight path control than other systems, it does not mean, that it should be used for different purposes. When I mentioned "IFR" it only means that FlightZoomer offers IFR-like operations supplementary over existing setups, which should be fully compliant otherwise.
I clearly say, that FlightZoomer might be an additional, redundant safety layer but it is not intended to be used as the sole safety and control layer. More direct - don't use FlightZoomer without traditional RC in place as the primary control channel. And also don't use FlightZoomer without a flight controller that offers fail safe modes like RTL.
Having said that, things like Minimum Altitude Charts are not so critical anymore. Because if you operate FlightZoomer as allowed in most of the countries, you have to stay within line-of-sight. So keeping enough clearance over ground should be no big deal. The solution I could envisage would be the APM terrain follow database displayed as this:
About fly-aways:
The minimal solution which I meant would be just tracking the flown away copter. Just get feedback where it goes and where it comes down ultimately, but no attempt to improve the outcome of the fly-away.
Being able to do something more constructive with a flight controller which is going crazy might be very tricky. As there are many potential reasons for a fly-away the right response would require many different solutions. The brute force approach which you describe is probably just marginally better than doing just nothing. In other words, it does not protect the aircraft from total destruction and IMO does also not decrease the chance of significant damage on the ground.
Based on FlightZoomer an optimized system could look as follows:
- Automatic fly-away detection using the phones's sensors & GPS.
- FlightZoomer would go into fly-away resolution mode. This mode could also be triggered manually at any time.
- In fly-away resolution mode, a number of strategies could be implemented (from less to more effective but also more complex):
a. set RTL mode -> does not resolve issues with the GPS
b. set STABILIZE mode and slow descent -> might not resolve issues with the accelerometers
c. active attitude control based on the phones sensors -> fully redundant control but also highly
complex phones app
Maybe port to QT and get all OSs for free?
@Martin Rüedi,
thank you for your kind reply.
You really represent and promote professional drone flight standards, banned by
FAA (IFR vs. VFR or FPV )
FPV First Person View is called sometimes Fly Per Video.
Your FlightZoomer looks to be a highly professional drone flight learning tool, flight simulator for Drone Pilot Schools.
How a Drone Pilot School could install and use your excellent application ?
What type of licence is offered for commercial use (educational sector) ?
"
The standard approach is: know the tallest point in your flying area and add 25 to 50m to get the altitude for your flight. "
But this is exactly the opposite, small drone is flown, since Minimum Altitude Charts are available to pilots of ultralights and heavier aircraft and not via Google Maps or Google Earth service.
I would like to implement TAWS into application like yours since Minimum Altitude Charts live few levels up over the TAWS 3D navigation instrument.
I would appreciate more your comments to Drone Fly-away Syndrome
since I am not aware of twin-autopilot, twin-sonsor solutions.
What is offered in case of commercial drone is Flight Termination System to cut off
battery supply via 2nd radio link to make drone to ground immediately ( applied at FIS Ski Championships in the Switzerland this month).
Ok, you can modify your Flight Termination System to cut off power gradually, limiting voltage but you require redundable autopilot, sensors "to track the flight path precisely
over any thinkable range"
If I am wrong, pls correct me.
BTW
You are welcome to join
Peer to Drone Crash Investigators
@Kevin, the Windows Universal app will be my next move (after releasing version 2.0).
I agree also about your other comments. Of course "full featured" can have a broad meaning. Probably quite some of the conventional mission planner features are missing in FlightZoomer. On the other hand FlightZoomer offers several concepts and features which are unique and cover completely different requirements than those of another me-too groundstation.
@Darius, FlightZoomer is not suited so well for tactical low level flying. The standard approach is: know the tallest point in your flying area and add 25 to 50m to get the altitude for your flight. Full scale operations use Minimum Altitude Charts like this to have always enough air under the wings:
Generally FlightZoomer works best if there is plenty of space and you can fly longer straight legs and wide turns. That way I can trully fly IFR-like.
In a fly-away scenario at least I have a completely redundant and independent communication path between air and ground. Even if the Pixhawk would not respond to control attempts anymore, I would still be able to track the flight path precisely over any thinkable range...
@Martin,
your FlightZoomer is really excellent and features highly sophisticated GUI resembling one in Dreamliner.
The problem is flying a drone you don't pilot Dreamliner.
You fly at low altitude and don't land in airport, so you should be careful about 3D physical obstacles.
With reduced view of map, your chances to avoid power lines, poles, mobile telephony transmitters are not great.
You are much to busy with controls, controllers and map view is much reduced.
Let me know your solution to Fly-away Syndrome ?