StereoPi - companion computer on Raspberry Pi with stereo video support

Hello!
I’d like to share with you the details of our last project on Compute Module 3 for playing with stereo video and OpenCV. It could be interesting for those who study computer vision or make drones and robots (3D FPV).

It works with a stock Raspbian, you only need to put a dtblob.bin file to a boot partition for enabling second camera. It means you can use raspivid, raspistill and other traditional tools for work with pictures and video.

JFYI stereo mode supported in Raspbian from 2014, you can read implementation story on Raspberry forum.

Before diving into the technical details let me show you some real work examples.

1. Capture image:

raspistill -3d sbs -w 1280 -h 480 -o 1.jpg

and you get this:


You can download original file here.

2. Capture video:


raspivid -3d sbs -w 1280 -h 480 -o 1.h264


and you get this:


You can download original captured video fragment (converted to mp4) here.

3. Using Python and OpenCV you can experiment with depth map:

For this example I used slightly modified code from my previous project 3Dberry (https://github.com/realizator/3dberry-turorial)

I used this pair of cameras for taking the pictures in examples above:

For video livestream from drone I use wide angle (160 degrees) cameras like this:

Now – to hardware part.

Front view:

Top view:

Dimensions: 90x40 mm
Camera: 2 x CSI 15 lanes cable
GPIO: 40 classic Raspberry PI GPIO
USB: 2 x USB type A, 1 USB on a pins
Ethernet: RJ45
Storage: Micro SD (for CM3 Lite)
Monitor: HDMI out
Power: 5V DC
Supported Raspberry Pi: Raspberry Pi Compute Module 3, Raspberry Pi CM 3 Lite, Raspberry Pi CM 1
Supported cameras: Raspberry Pi camera OV5647, Raspberry Pi camera Sony IMX 237, HDMI In (single mode)
Firmware update: MicroUSB connector
Power switch: Yes! No more connect-disconnect MicroUSB cable for power reboot!
Status: we have fully tested ready-to-production samples

That’s all that I wanted to cover today. If you have any questions I will be glad to answer.

Project website is http://stereopi.com

Views: 1505

Comment by Auturgy on June 29, 2018 at 2:29pm
Cool! Has been on my to-do list for too long.
How did you solve the time sync issues between the cameras? I looked at this some time ago. iirc the sync pins on the Pi cam aren’t exposed, and you can’t deterministically trigger the image capture because of how i2c is implemented.
Comment by Eugene Pomazov on June 30, 2018 at 2:10am

Auturgy, sync question was the main point for us to analyze when we start do develop the first generation of the board on CM1 in 2015. On Pi forums user 6by9 mentioned this question several times in stereoscopic forum thread. There are no ability to absolutely sync them, but usually capturing time differs in just few ms. I was afraid of risk of error accumulation during long-time capture, that's why we did a lot of long-run tests. We make HD image capture, stream it over UDP and then analyze it (usually for 48 hours). We did not find any issues causing depth map artifacts.

Comment by Auturgy on June 30, 2018 at 2:52am
have you tested with a moving camera? The other issue is being able to accurately time stamp the images: there is a difference between when image capture is commanded and when it is actually executed, which is not consistent (between the cameras or between frames for a particular camera).
For applications such as VIO this creates a lot of noise in the results.
I’m not trying to be critical: just trying to get a feel for whether I should dig my cm3 out again :)
Comment by Eugene Pomazov on June 30, 2018 at 3:59am

Of course yes, moving camera is our main usage scenario. The only case is a low-light conditions, you need to set FPS priority while capturing video.

At that moment we finished hardware part of a project, and now focused on software part. Nearest step is ROS and obstacle avoidance. We prefer to make a practical stress-tests to find all unstable parts in our solutions.

Auturgy, what is your project on CM3? Did you try to make own PCB or used Pi's devboard?

Comment by Greg Dronsky on June 30, 2018 at 4:06am

Nice project! You write you support HDMI input camer. How do you connect the camera to the board?

Comment by Eugene Pomazov on June 30, 2018 at 4:14am

For HDMI we use Auvidea HDMI to CSI2 adapter. Unfortunately Toshiba chip in this adapter has only unofficial support, that's why it is impossible to use two HDMI capture modules at once. We use one HDMI in and one camera to make two independent video streams.

Comment by Auturgy on June 30, 2018 at 4:15am
Both. Custom carrier was designed (minimalist: basically just exposing USB, sdcard, the csi’s and uart, with some power management), but not fabricated.
Started with the dev board.
Comment by Eugene Pomazov on June 30, 2018 at 4:21am

"Both. Custom carrier was designed (minimalist: basically just exposing USB, sdcard, the csi’s and uart, with some power management), but not fabricated.
Started with the dev board."

Why do you stopped? It does not fit your tasks?

Our first generation (on CM1) was for stereo livestream only, and also has additional 16PWM chip onboard. Current generation more focused on CV.

Here's previous one:

Comment by Auturgy on June 30, 2018 at 4:57am
I just had too many little projects on the go and it dropped off...
Comment by Eugene Pomazov on June 30, 2018 at 6:00am

Auturgy, if you will reanimate it - let's compare our results and share experience!

Comment

You need to be a member of DIY Drones to add comments!

Join DIY Drones

© 2018   Created by Chris Anderson.   Powered by

Badges  |  Report an Issue  |  Terms of Service