Synthetic camera offers FPV purely as a software solution

This is my second post in the series called "FlightZoomer 3 Innovation Previews". The first was this one: Maximized terrain awareness for drone pilots

The innovation highlight presented in the video above is the synthetic camera, that offers an FPV experience without many of the disadvantages of a classic FPV approach. In the video 10 advantages of the synthetic camera are listed and demonstrated:

No video data transmission needed -> less hardware, less complexity
Truly worldwide range if there is nothing more than 2.5G cellular connectivity
Highly improved reliability
Cut the Gordian knot of zero cost Full HD FPV
Principally no yello, no shaking
No fog, no clouds, no twilight, never night
Adjust the viewing angle during the flight
Adjust the camera tilt and yaw angle during the flight
Manual snap back view direction control (MSBVDC :-))
Integrated optional zero lag gimbal

Of course there are limitations too:

Close to the ground the synthetic view lacks accuracy.
3D cities are limited, 100% world wide coverage is only offered as aerial image laid upon an 3D elevation model (which offers still remarkably stunning views, especially when cruising a bit higher)
The scenery stays static. You cant observe non-static objects, like traffic, cars,...

The solution is implemented using the MapControl of the UWP programming stack. It offers a comprehensive API to place a camera over the 3D landscape at any place and specify all the parameter, that we need (including pitch and bank angle). B.t.w. this is also the reason, why this solution is not easily portable from Windows to Android or IOS (in fact not doable at all): the Android or IOS map API do not offer the needed capabilities for this feature. Google does not support to place a camera at all at a particular position and altitude and Apple does not support the combination of setting pitch & bank while using 3D maps with an aerial image.

An important detail is the possibility, to additionally feed the video from a real FPV camera into the app. The real camera view can be placed on top of everything else as a small, moveable and resizable overlay. This feature perfectly complements the overall package, FlightZoomer offers.

Hi Gustavo

The phone is upgradeable to Windows Phone 8.1 so everything should work perfectly. I fact I never had an onboard device for more than about 40$. Any second hand WP8.1 device with Bluetooth should be fine.

Be careful about the hardware you need! You don't need a pair of phones! The groundstation device was only a phone up to FlightZoomer version 2.1. The version 3, which has the synthetic outside view and is announced above, is designed for the tablet form factor (or notebook, in fact any Windows 10 device, except phone). So what you need is one phone as onboard device, a second phone as tethering hub (can be just any smartphone out there, including Android, IOS devices) that is used by the Windows 10 groundstation tablet or notebook device to provide internet connection.

The whole communication chain is this:

Pixhawk <-> BT HC-06 <-> Onboard WP81 smartphone <-internet-> Relay Server (Windows PC at home) <- internet -> Any smartphone as tethering hub <-WiFi-> Windows 10 Tablet or PC as groundstation.

You might want to dig into the installation documentation on my homepage http://flightzoomer.com/

You need to be a member of diydrones to add comments!

Join diydrones

Comments

r3d_f1v3 March 31, 2017 at 4:53am

Great! Thanks for your reply and all the work your are doing on this system!

Let's see what you can do with all this ideas, I can't wait to try a plane version :)
Martin Rüedi March 31, 2017 at 4:02am

Fully agreed. The setup with a SiK radio would avoid the relay server entirely (so some minor features like logging & route database would still need to added to the cockpit app). It would remove complexity quite a bit. Only one app would have to be operated. While offline storage of map data is supported by the OS, the 3D view (I assume for the elevation data) is not. So very likely the requirement to have Internet on the groundstation device wont go away...
r3d_f1v3 March 31, 2017 at 1:44am

Supporting the Sik Radio, would be great for "short range" LOS operations, avoiding the need for a phone in the plane, saving weight and cost of the data usage.

Also, if it was posible to have the server app and the groundstation on the same windows laptop on the field, and the map for the area to fly could be stored locally, then the Internet connection and the online server could be optional...
Martin Rüedi March 31, 2017 at 12:44am

Thanks for the feedback and the suggestions!

Arduplane support, SiK Radio telemetry connectivity (though the groundstation app still requires Internet for the map data) and some sort of headtracker solutions are indeed very important additions, that are already on the shortlist of future extensions. I have to go step by step..
r3d_f1v3 March 30, 2017 at 11:31am

Also, it would be very nice if the synthetic image view could be controlled using the headtraker in the FPV goggles, so the real and virtual views were in sync as the head was moving... Maybe you could extract the headtracker possition info from 2 pan and tilt RC channels on the APM, and use it instead of a mouse input...
r3d_f1v3 March 30, 2017 at 11:19am

Hi Martin. I think your software would be a very good backup and security system for hobby FPV flights, in case of fog, darkness, or video loss. The thing is, I don't fly multirrotors, so It would be very interesting if there was a version compatible with APM:Plane. I know a lot of other FPV plane-flying friends out there would also be interested in buying a plane version, so I think you have a huge market there...

One question: Could this system work without a mobile phone, using only a 3DR compatible SiK Radio link?

Thanks!
gustavo March 26, 2017 at 6:00am

very clear thanks
Martin Rüedi March 25, 2017 at 7:09am

Hi Gustavo

The phone is upgradeable to Windows Phone 8.1 so everything should work perfectly. I fact I never had an onboard device for more than about 40$. Any second hand WP8.1 device with Bluetooth should be fine.

Be careful about the hardware you need! You don't need a pair of phones! The groundstation device was only a phone up to FlightZoomer version 2.1. The version 3, which has the synthetic outside view and is announced above, is designed for the tablet form factor (or notebook, in fact any Windows 10 device, except phone). So what you need is one phone as onboard device, a second phone as tethering hub (can be just any smartphone out there, including Android, IOS devices) that is used by the Windows 10 groundstation tablet or notebook device to provide internet connection.

The whole communication chain is this:

Pixhawk <-> BT HC-06 <-> Onboard WP81 smartphone <-internet-> Relay Server (Windows PC at home) <- internet -> Any smartphone as tethering hub <-WiFi-> Windows 10 Tablet or PC as groundstation.

You might want to dig into the installation documentation on my homepage http://flightzoomer.com/

FlightZoomer - Main Page
gustavo March 25, 2017 at 5:22am

could you please tell me if this phone would be ok?

http://www.ebay.com/itm/Unlocked-Nokia-Lumia-521-GSM-Windows-Phone-...
gustavo March 25, 2017 at 5:11am

thank you Martin, i would have to find a pair of W8 phones first , will make contact as soon as have it covered ,.

of 2

This reply was deleted.