Hey guys,
a long time passed since the last update from OpenFPV.
Thank's to the community for the great feedback and special thanks to all people around the world who wrote me feedback mails, joined the project or donated money. I really appreciate your interest about the project.
It was very silent the last month about the project, the reason is very simple:
I do not want to ship crappy unusable software.
My vision for OpenFPV:
OpenFPV should be a simple intuitive application for all desktop platforms. Easily extendable, stable, well designed. And easily extendable was the biggest challenge. Do you want beat around with C++? Do you want build your GUI with Qt?
Imagine you can create a dedicated data channel on TX side with python and receive the data on RX side with simple JavaScript. Layout your UI with HTML5 Elements. If you want to go deeper, why not edit the GStreamer pipe by your own. Without recompiling the whole application. Create your telemetry module in no time.
The trouble:
It was insanely difficult to get the video played back inside the webkit browser engine without adding delay to it. It is not possible to do it with HTML Video, Flash, WebRTC. The biggest challenges are buffer settings. Event Flash cannot display the video fast enough (and this is the most stable technology for live streams inside a browser) for serious FPV flying. My OpenFPV workspace directory is full with prototypes, ideas, and more prototypes. After month of fighting with myself and my vision I came to a deadly simple idea. Convert the H264 stream to a format like MPEG-1 and decode it with JS.
Maybe you think: "What the ... ?"
Don't worry, that was my first though too. I started to write a MPEG-1 decoder with JS until I stumbled upon a unpopular project on github which is doing exactly this job. After integrating all the GStreamer pipelines, transcoding, sockets and web sockets and this library, I displayed the fist output inside the browser. Without high CPU load and a good quality. And the best thing: Without a noteworthy additional lag. Sometimes the simple Ideas are more difficult then the complex ones.
Demo:
I created from this Iron Bird the first prototype. And I decided to give you guys a little update about the project. I am excited about your thoughts and ideas.
- Tilman
Comments
Hey guys,
thank you a lot for the feedback!
@Guy McCaldin Thank you very much for the donation, every penny helps me a lot to invest in new Hardware. As you can imagine,
this project takes a lot of money to develop and is very time consuming. The latency on my test setup without the pipline into the browser is around 116 - 140ms. It increases over time, but I am sure that we can fix that. With the [h264 dec -> mpeg-1 enc > mpeg-1 dec] setup it is arround 200ms. But I am not done here. My goal is 116 - 120ms inside the wrapper. The release for FPV flying will be 2015 at the moment. The reason for that is, that we have to do a lot of work further optimize the trasnimission. But for observing telementry and/or the video feed, it should be possible this year.
@Jerry Hatrick
Haha no, but you make me laugh :) Sorry for my voice in this video, I't was early in the morning.
@Marius van Rijnsoever
Thank you! Nice Idea to control over 433Mhz. I would leave the telemetry on the 2.4Ghz channel to get a tight and a easy to integrate setup.
@Stephen Gloor
The problem with channel 13 is that my high power alfa 2.4 Ghz hardware dont support that channel. Maybe I have to change. What hardware do you use?
@Tearig
I will contact you for further discussion on this topic. Thank you for your interrest.
Another thing: I will remove the mpeg1 enc / dec part if possible to get the latency down to 116ms. The problem here is that we need to decode the h264 stream on gstreamer side (with HW accel) and pipe the decoded data into a JS module. This module will the draw the frames onto a canvas context. I'm be honest, I am not the H264 uber expert and not a crazy low level dev. Are you have ideas how to do this or anybody else here?
@johnkowalsky
I think it should be possible to transmit the telemetry data over 3G. Maybe a 320x240/15 stream for observing. The bitrates for telemetry will depend of how much data you want. But < 10 Kb/s. Only a very rough prediction.
best, Tilman
If the telemetry signal was embedded, and the position data was then sent, the environment could populate the base-station with data from either local drive or internet connection, (Google Earth). To illustrate this I put together a quick video of a flight made today with it synchronized, (I just did it on screen, pause start) to illustrate what I was thinking.
so if the data signal drops below a certain level the fall back could be a simulated environment. It seems that using this to the advantage on the base station could enhance remote flight capability by using the resources available on the ground.
Ah, there's this group who had a kickstarter to use 3G to both control and see video and further integration with oculus rift. Theoretically they said, you would be able to fly your drone anywhere you'd have coverage. I think it's a bad idea to integrate with systems you have no control over, which is why wifi would be my preference.
If you assume a reasonable quality where you're not looking at smudges onscreen and 720p, then a static scene would give you about 2 Mbps. If you move around your bitrate can peak right into 12 Mbps. On average with some movement I found that 7-8 Mbps is a nice setting, but notice the peaks. This is assuming VBR. If you use CBR, people have said it "trains" your network better to deal with the throughput and it wouldn't generate the peaks but just degrade the quality more. My camera was not able to degrade quality that gracefully and it would fail as bad as a variable bitrate scheme with the peaks.
Hi Guys
I am also trying to make a digital video transmitter but at lower frequencies so it will peform better in NLOS (non line of sight). I have so far using a modified video transmitter operating at a frequency of 442 MHz that I can switch the channels by PWM. The idea was to use the method of OFDM to transmit data from an FPGA that did the compression and analog to digital converting. The camera that I am using so far is 680TVL CCTV analog camera.
I found that an FPGA would be to demanding and expensive so I will have to try with a beagle black bone or something that you might recommend ?
Btw my goal with this project is to make a digital video transmitter that takes a normal camera input and uses OFDM to have a long video transmisssion distance.
hi tillman. very good you, can you share the code with github, so everyone can help everyone?
Very good job, indeed
max
@Randy - thanks for that. I have the 4 cores doing recognition of the photos we are taking so I don't have much left and that may be the problem. I used multiprocessing to do this and it works well.
We aren't trying to fly FPV so it does not really matter too much.
Finding using the keying as you suggest could be then coordinated with a secondary signal from the telemetry radio the base station can compare timedata stamps and could also provide a video latency number. Going to a digital system that could be agnostic to the transmission while using the base station to coordinate all of these signals really could be amazing.
To the point of WiFi 2.4 using channel 13 it is feasible however we have dropped video down to 1.3 GHz and have used notch filters for simultaneous 2.4 rc control and WiFi bidirectional data transmission however it seems due to the congestion it has been a bit unreliable.
If the keying of data frames works and telemetry can be send via 2 sources 1 on the video channel and then on either a bidirectional UHF or 2.4 compatible system the redundancy or ability for a very simple setup could be completed.
I see allot of possibilities pm me and we can discuss what you may need from my side
I agree with JAB's idea that the best idea is to combine the telemetry and video into one feed and then intelligently prioritize the controls over the video. Easier said than done though!
@Stephen, In my tests I saw lag of about 1.2seconds if I only captured the image from the camera 5 times per second. If instead the pythong/opencv program running on the Odroid captured the image 15 times per second the lag dropped almost to zero. So in short, pull the image faster from the camera and the lag will go away. The issue of course is that if your small linux computer isn't very fast it may not be able to capture the image and process it that quickly. For the balloon finder I got over that problem by making use of 2 cores on the Odroid (it has 4 in total) using Python's "Multiprocessing". So the image/video capture would run in a separate process all on it's own so it could run much faster.
-
1
-
2
-
3
-
4
of 4 Next