Project FlightZoomer: low tech, COTS, smartphone based telemetry (+more) solution

Posted by Martin Rüedi on October 15, 2015 at 4:52am

Hi there

I'd like to present my project FlightZoomer, which offers telemetry over cellular IP at small cost:

  • Onboard device: a smartphone (cost: less than for a dronecell unit)
  • Ground device: a second smartphone
  • Crack the NAT/firewall barrier: simply by using a relay server (which adds features b.t.w.)
  • Reach the relay server via http://freedns.afraid.org/
  • UDP based
  • UDP hole punching trials did not bear fruit
  • The communication is error tolerant in any aspects
  • There is throuput throttleing as well as recovery from any failure. If e.g. the communication fails for more than 5 seconds, all the software communication layers are completely reinitialized from scratch
  • While there is a mode which guarantees message delivery, most traffic works with a second fire&forget mode
  • Communication using a proprietory, tailor made protocol
  • The onboard smartphone talks with an APM/PixHawk using a bluetooth connection via a HC-06 device (+MAVLink)

What shall I say?

It works, latency is less than 100ms, the focus already has moved to additional features, which easily can be added on a smartphone based platform (use the rest of the available sensors/periphery like GPS, cameras, WiFi..., synthetic voice output, touchscreen)...

Impressions&look&feel of the system:

More details:

flightzoomer.com

You need to be a member of diydrones to add comments!

Join diydrones

Tags: flightzoomer
Email me when people reply –
Follow

Replies

  • Daniel Fidalgo November 23, 2015 at 12:39pm

    Hi Martin, any news about the Windows 8 universal app??  BTW, great project!

  • DavidFolts October 17, 2015 at 6:25am

    Martin-

    I've been through the documentation and looked at the video. 

    The sensorics app is very slick... nice features and seems robust.

    For those of us who are NOT 787 pilots, this GCS app is way to dense on a small screen for flying a quad or fixed wing sUAS outside.  Now if you could get the 787 UI onto a large screen, say the 10.1" of the Nokia Lumina 2520 running Win 8.1 RT, then it'd be easier to not fumble whilst flying.

    Have you considered a straight mavlink connection from the sensorics app to Mission Planner or Tower?  As much as I think the 787 GCS UI is interesting, it is simply too busy on a 6 inch screen. Or, how about a simpler vehicle monitoring and control UI?

    I commend you for developing this system...I am considering buying a Lumina 1020 phone with 41 MP camera to test Flightzoomer use on my Iris+. Now, if I could CONTROL the camera (snapping shots, turning on video... zooming etc from the GCS.... those features would be VERY USEFUL to a LOT of us...

    I'd be willing to pay $100  for the following:

    1) Sensorics app that also interfaces with phone camera features fully (sensor control)

    2) GCS app that serves as a simple cockpit and also can transfer MAVLINK connection to/from Tower or Mission Planner...

    3) Control of sUAS via Mission Planner over the Flightzoomer system, OR ,

    4) Control of the sUAS using a more robust Windows tablet rather than a WP as the Flightzoomer GCS host...ie joystick control & the aforementioned camera (payload) control features.

    How can I help you in this endeavor?  My wife is a native German and we could assist with your user manuals & documentation, as well as real world test cases and flight experiments.

    Interested?

    Dave

    • Martin Rüedi > DavidFolts October 17, 2015 at 1:49pm

      Hi Dave

      Thanks for the kind words.

      I appreciate your feedback and am also eager to extend the functional scope of FlightZoomer. I think your ideas do really have merits and I can tell you that I am already working on new features in this direction.

      E.g. I decided to port the app (the groundstation first) from Windows Phone 8 Silverlight to a Windows 8 universal app. From that point on your Lumia 2520 would be supported perfectly and any other Windows 8 or 10 PC, phone or tablet as well  (though I never had problems with the screensize using the GCS on the Lumia 1320 or even the Lumia 925 before).

      Another thing which I already have done (not yet published/documented) is serving the Sony Remote API. After connecting the Sensorics device with the WiFi of a Sony camera the Sony camera will automatically go into movie recording mode while being in Locked mode. There is not more control at the moment, but supporting more camera features would be something natural and easy to do next. The feature could control either the internal camera or a Sony camera which can be accessed using the camera remote API.

      Some ideas also go in the direction for smart ROI control (hint: a third app, called "Flightzoomer ROI target" could be used to follow a moving spot visually, the "actor" would just have that app running in his pocket).

      As my communication is proprietory and not MAVLink based, coupling the system with the Mission Planer would require a bit more effort (of course MAVLink packages could quite easy be wrapped into my messaging system).

      For sure I will soon also start controlling the flight controller using the GUIDED mode. This will allow me to replicate all of the 787 autopilot modes (heading hold, track hold, altitude hold, vertical speed hold, arm and capture altitude, speed hold, arm and capture localizer, capture and hold ILS, altitude change, LNAV + VNAV = following flightplans).

      Thanks also for offering help e.g. with documentation. I would probably not be much quicker if I could write in German first, but it would definitively be very helpful if somebody could help me with my English documentation (review, correct or also extend it from user perspective).

      Also your offering to participate as beta tester is welcome. I will consider that once I have a bundle of new features ready. Participating as tester and maybe also providing some of the documentation would be really a great help.

      Rgs

      Martin

      definitively  adv.

    • DavidFolts > Martin Rüedi October 21, 2015 at 8:14pm

      Whatever help you need Martin, I'm available...test, document editor/creator, sales guy even. You have a unique and remarkable system here. I'd like to be involved.

      Dave

      ps: you can directly email me off my profile

  • DavidFolts October 15, 2015 at 9:02am

    I just took a quick peek at the GS application on  the WinPhone store. Slick!

     I have a Lumia 1520 (6" screen) running "Denim" and I cant wait to conclude my project in San Diego to get back to my "hangar" and experiment with your solution.

    So my "spare" phone is an old iPhone 4 I use with my AR Drone...looks like I'll need a cheap WinPhone for the Sensorics app...hmmm. I see a few of those on the Banggood.com store.

    Could I run the GS app on a Lumia 2520 tablet running Win 8.1 RT?

                 That unit I have also has a cellular 4G/LTE modem built-in... and it has a 9inch screen.

    • Martin Rüedi > DavidFolts October 15, 2015 at 11:42am

      Thanks David for your response...

      My groundstation runs on the Lumia 1320 (also 6") from my wife... ;-) The 1320 is probably the cheapest 6" handset out there. Very nice as a groundstation.

      About the supported versions: Sadly I cant offer more than WP support at the moment. I have not yet decided to port the apps to other OSs. Android would be a candidate or Windows 10 on the other hand, because with Windows 10 I would support any thinkable size of devices with one version.

    • DavidFolts > Martin Rüedi October 15, 2015 at 12:04pm

      Martin-

      I'm not at all perturbed that you aren't looking to port to iOS...my iPhone 4 is darn slow anyway.  If you had a Windows tablet available, would you port to 8.1RT or WIn 10?

      My wife doesn't much use her Win 8.1RT tablet (Nokia 2520), wonder how much porting would be required to run the existing WP GS app on that tablet...?

  • Darius Jack October 15, 2015 at 6:35am

    My congratulations,

    my suggestion is to remove dynamic DNS server,

    remove DNS relay server, since you can establish peer-2-peer voice@data communication

    over 3G-enabled smartphones, called 3G Video.

    3G Video stays for voice call + live video broadcasted.

    Live data channel can be used to transmit data two-way.

    DNS can be replaced by MSISDN ( subscriber number)

    Dynamic DNS assigned by mobile telephony network operator can be transmitted

    from No.1 smartphone to No.2 smartphone as data via 3G video/data link

    and the opposite.

    If you don't have 3G coverage you can still use SMS messages to broadcast dynamic IP assigned to secondary smartphone and the opposite.

    SMS message can be easily received, processed to extract assigned IP number to be used

    by web server and web requests.

    Technology is 20 years old

    (broadcasting dynamic IP assigned via SMS or via 3G data link)

    Any 3G modem offers this functionality ( software coming with 3G modem).

    You can install Open-Wrt or another web server package to read SMS to extract dynamic IP address sent and to establish IP-2-IP peer-2-peer data communication link.

    darius

    manta103g@gmail.com

    • Martin Rüedi > Darius Jack October 15, 2015 at 8:13am

      Hi Darius

      Thx for your response...

      3G video has a number of disadvantages (IMO):

      - Seriously restricted regarding supporting hardware

      - Not supported everywhere

      - Expensive usage (a permanent call would cost much more than the few dozens of KBs I send back and forth during a flight).

      My approach works always, is cheap, low tech, minimized dependencies and allows interesting multi user scenarios, because I have a machine in the middle, where a large numbers of users could be connected to. Latency is acceptable, any further improvement probably hardly noticeable.

      In addition it works well from EDGE to 4G while I doubt, that any SMS based approach would offer a satisfying latency/refresh rate.

This reply was deleted.