No bandwidth bottlenecks

Embedded Stereo Vision based on Intel Arria 10 SOM

To address the growing 3D stereo vision market, which is expected to reach USD 5,46bn by 2022, Dream Chip will release their Arria 10 System on Module (SOM) with dedicated stereo vision support. The module is an ideal fit for all embedded imaging applications supported with an OpenCL BSP for quick development cycles.

The Arria 10 SOM has two dedicated LVDS BCON image sensor connectors, each having four LVDS lanes plus various control signals and can handle up to 1.200fps for Full HD resolution. (Bild: Dream Chip Technologies GmbH)

Two separate memory interfaces allow the ARM Cortex A9 subsystem and the FPGA to handle bandwidth intensive applications such as frame buffering and object detection to proceed in parallel without bandwidth bottlenecks, especially for high frame rate, stereo vision and 4K applications. The system has two dedicated LVDS BCON image sensor connectors, each having four LVDS lanes plus various control signals, allowing the direct connection to image sensor extension boards such as the Basler dart camera, which enables 3D stereo vision and OpenCL algorithm analysis. The Pylon camera control software runs on the ARM subsystem. The SOM itself can handle up to 1.200fps for Full HD resolution, the 32 versatile LVDS lanes on the module connectors, allow a variety of customer specific high speed differential data IO with e.g. image sensors, displays, ADC/DACs etc. The twelfe RX/TX Gigabit transceivers can be used to implement high bandwidth video standards such as 12G SDI or DisplayPort. All components on the module are certified for industrial temperature grade (-40 to +85°C), and the module connectors are ready for vibration intensive environments. To ease the integration into industrial environments, where 3,3-1,8V IO is still common, the module offers level shifters with user provided reference I/O voltage for the ARM/HPS I2C, SPI and GPIO interface signals.

Why the dart BCON camera with LVDS interface?

Stereo vision implies four major requirements regarding the camera:

• • Appropriate image sensor: For stereo vision applications, typically mono sensors are used to reduce computational complexity of the matching algorithm. However, in some cases, color information might be useful as additional feature that helps to find corresponding points. For moving object detection, global shutter sensors are recommended to avoid rolling shutter artefacts. Since high resolution correlates with high computational effort, low resolutions from VGA (640×480 pixels) up to 5MP are used for stereo vision applications. The same principle applies for the framerate. For moving objects, up to 60fps are used, which requires fast image processing on the customer’s processing board. Nevertheless, for most customers, computational time of the matching algorithm is the bottleneck of the application, limiting both framerate and resolution. The dart camera with BCON for LVDS interface utilizes the Aptina AR0134 (1,3MP, 54fps), the EV76C570 (1,9MP, 60fps) and the MT9P031 (5MP, 14fps) as image sensor, making the camera suitable for stereo vision applications. Furthermore, the pylon Camera Software Suite offers a variety of functions to control image resolution and framerate, allowing for an individual adaption of the camera to the application specific requirements.
• • Exact synchronization: Exact synchronization is one of the crucial aspects for reliable extraction of 3D information. The used camera has a dedicated hardware trigger, which can be used for rapid image acquisition. The trigger signal can be sent over the flat flex cable (FFC), so that there is no additional cable connection. One trigger signal can be used for both cameras. Hardware triggers are faster than software triggers by several orders of magnitude. Another advantage of the camera is the immediate trigger mode, which is implemented as firmware feature. If the Immediate trigger mode is set, the trigger delay can be improved significantly. This way, it is possible to reach trigger delays down to 2µs (measured with AR0134 sensor).
• • Low CPU load on customer’s processing board: Algorithms for calculating depth information based on two 2D images are complex and require high computational resources. Especially on low-cost processing boards, these resources are very limited. Unlike other camera modules in the market, the dart BCON delivers a perfect image, not just raw sensor data. For color sensors, advanced image processing, like sharpening filters and debayering algorithms as well as gamma and pixel correction is processed on the camera-internal FPGA before the image is transferred over the BCON interface. This saves highly limited computational resources for customer algorithms. The camera is connected to an FPGA on a processing board. In stereo application algorithms, image processing algorithms like edge detection or pixel correlation can be calculated directly on the FPGA, saving time and CPU resources.
• • Mechanical stability for camera alignment: Calculation of depth information is based on spatial correlation of objects to each other. For stereo vision, the cameras need to know the distance to each other (baseline) as well as the spatial orientation in order to work as expected. This results in highly precise calibration processes, after which the mechanical setup is supposed to stay as it is. The camera comes with pre-drilled screw holes, which can be used for solid attachment of the camera to the underlying construction. Another advantage is the flexible flat flex cable. The 28-pin ZIF connector leaves little room for cable displacement. Also, through its flexibility, applied force to the cable is not transferred to the camera.

The high count of LVDS lanes and the high memory bandwidth make this System on Module the right choice for demanding vision applications. Up to six image sensors with 6,4Gbit/s video data each can e.g. be connected for ADAS applications such as surround view for larger vehicles like busses or trucks. Due to the large amount of available logic of up to 480kLE, multiple 3D stereo vision applications with computing intense algorithms implemented in OpenCL are possible on just one module.