Experiments with an industrial USB webcam

Above is a fullsize image of a 5MP webcam. A webcam usually has a CMOS sensor and commercial ones support up to 720p resolution. 720p is a result of the usb bus bandwidth available to stream the images and since many computers still use USB 2.0, this is the maximum resolution for reasonable framerates of 25fps or so. At USB 3.0 you can achieve 30fps with uncompressed data, or 1080p@15. With MJPEG streams this increases to 55fps/30fps respectively with "full size"  resolutions somewhere between 2-10fps.
I have a raspi and a piNoir cam and I was disappointed by the severe amount of rolling shutter. This is an industrial USB cam with an M12 lens mount.

I created this image by just waving the camera module around a lot. Notice the umbrella. A uav probably doesn't have such extreme movements, but there will be vibration involved.


The chip that's onboard is a Cypress CX3. The CX3 is an ARM processor with multi-lane image sensor readout capability, a USB 3.0 interface and a development environment that facilitates development of everything that you can stream over USB. The chip only has 512kB memory for code, memory, stack etc., yet the JPEG images (or YUV) are at least 1MB, so there's no way to buffer the image data. Uncompressed it's more than twice and for encoding you need a bit of memory too. This means that data flows through the following pipeline:

Host <--USB--> CX3 <--MIPI--> sensor

So it's probably unable to stream the data out quick enough, which means it needs to hold out on streaming more lines of data from the sensor. That's a theory, because I'm not sure how this process works in detail. It is likely though that using USB 3.0 will reduce the effect; usb-2.0 is 480 Mb/s and 3.0 is 5 Gb/s (not all of which usable for data, there's a lot of framing overhead).

These cameras also have an auto-managed shutter time. It sounds great if they claim it achieves 1/1600, but given the pipeline above, that only applies to the lines being read at the moment and since they're not read fast enough, you still have huge rolling shutter effects. Those effects are difficult to remove, because if vibration is involved or you have non-constant angular velocity, it's not a simple operation.

So why the interest in this camera?

  • It's only 20g and very small, very easy to attach
  • The lens is very easily replaceable; IR, IR-cut, fov, filters, gels, etc.
  • It has a GPIO pin to initiate the capture and this is accurate. You literally "arm" the camera
  • It has a GPIO pin that apparently indicates when a capture is taken for "strobe" use.
  • It can take up to 8 frames per second.
  • Two of these cameras can be hooked up, shoot together to 1ms precise.


The disadvantages:

  • You need a host computer with USB 3.0 interface. Not sure if embedded systems are fast enough to process the data and this must fit on the vehicle and be light.
  • It's unlikely that an embedded system exists with two independent 3.0 usb hubs.
  • The shutter speed is probably not as good as a P&S camera, which impacts blur.


There's also a 13MP usb cam, but this would increase the data to be transferred by a factor of 3, increasing the rolling shutter.

Probably the better choice is just grabbing a gopro, exchange the lens and then store the images onboard. The gopro includes a processor and probably temporary memory for processing. Anything custom built is unlikely to be lighter than 90g.

The P&S cam probably reads the sensor as fast as it can with fast memory inbetween for temporary storage, which is read out and processed prior to storing on an SD card. Perhaps an expert can chime in here.

There are interesting CMOS cameras that use a mechanical shutter and special CMOS sensors that hold their charge for much longer. This means that the sensor is exposed at the same time, but the readout process (in a dark area) can then take a bit longer. There are also the global shutter CMOS sensors. However, those are $3k or so without camera body.

Views: 3159

Comment by v01d on July 3, 2015 at 7:07pm

pointgrey and other manufacturers sell relatively cheap industrial cameras, which are 2.0 and have global shutter CMOS sensor, although only with 640x480. they have nice features although they can be a bit of a pain to use sometimes

Comment by Loren Zimmer on July 4, 2015 at 3:28am

What about the ODROID-XU3?

Comment by Laurent Eschenauer on July 4, 2015 at 8:06am

I'm not sure about your use case, but why not use a small embedded linux and a MIPI camera board directly ? You can easily find MIPI boards with M12 lens holders. Raspberry PI2 is capable of H264 encoding a 30fps 1080p stream.


Developer
Comment by Randy on July 4, 2015 at 11:29pm

I'm not an expert in this area but I agree with Laurent that it might be better to skip the USB interface completely and use CSI-2 interface (from the MIPI Alliance) which is what the RPi camera is and what most cellphones use (I think).  The issue with these CSI-2 cameras though is that they're not easy to find in a hackable form at a reasonable price as far as I can tell.  The RPi camera is a bit of an exception and I think it only works with the RPi and the RPi only works with this particular variety of CSI-2 camera.

Comment by Gerard Toonstra on July 5, 2015 at 2:23am

Well, I did some searching around and these low level interfaces are great when you have hardware that interfaces with it and know what to do (have an oscilloscope, plenty of patience, etc). The raspi cam is specifically for the raspi and the csi-2 is an interface specification. I have a raspi noir cam and the rolling shutter is terrible even in daylight conditions. So part of this experiment is to see if that can be improved. I have no idea if raspi uses the maximum bandwidth available, etc. Also, one of the potential issues I considered is that the miniature lens may not allow higher shutter times that could speed this up.

Anyway, building a camera or something that interfaces with it is hard. The signals are high-speed, which takes this out of the realm of hobbyist electronics. The MIPI standard specs are protected by the MIPI Alliance, the sensors are not available for sale in 1 unit quantities, etc.

Then what I also wanted was a method to control the trigger and some other properties.

The raspi cam works, but there are specific client apps that allow you to interface with it, it's not a common interface at all. The camera I use has a USB uvc standard interface, which makes it easier to grab images from a variety of apps, plus a standardized interface for parameter discovery.

The ODROID is indeed a USB-3 embedded platform. At 72g with cable and cam it would make the weight around 120g or so (more than a gopro).

So... until now most people worked with P&S canon cameras with chdk to get their things going. The gopro should have more processing power and abilities for taking a picture, except that it's not clear how to hook it up to an AP. Now that gopro is at v3/4, it should be possible to get cheap v1/2 gopro's, screw a different lens on, figure out how to trigger it remotely and use that instead of a P&S?

Comment by DAniel Dumitru on May 6, 2016 at 4:15am

Could you please tell us what camera did you use ? is from http://www.e-consystems.com/  ?

BR,

Daniel

Comment

You need to be a member of DIY Drones to add comments!

Join DIY Drones

Groups

Season Two of the Trust Time Trial (T3) Contest 
A list of all T3 contests is here. The current round, the Vertical Horizontal one, is here

© 2019   Created by Chris Anderson.   Powered by

Badges  |  Report an Issue  |  Terms of Service