Anzeige
Anzeige
Anzeige

Event-Based Vision

Event-based vision is poised to take over from the frame-based approach used by traditional film, digital and mobile phone cameras in many machine-vision applications.

Image 1 | Prophesee has developed an image sensor containing an array of autonomously operating pixels that combine an asynchronous level-crossing detector with a separate exposure measurement circuit. Each exposure measurement by an individual pixel is triggered by a level-crossing event. (Bild: Prophesee)

Image 1 | Prophesee has developed an image sensor containing an array of autonomously operating pixels that combine an asynchronous level-crossing detector with a separate exposure measurement circuit. Each exposure measurement by an individual pixel is triggered by a level-crossing event. (Image: Prophesee)

The mode of operation of state-of-the-art image sensors is useful for exactly one thing: photography, i.e. for taking an image of a still scene. Exposing an array of pixels for a defined amount of time to the light coming from such a scene is an adequate procedure for capturing its visual content. Such an image is a snapshot taken at one point in time and contains zero dynamic information. Nonetheless, this method of acquiring visual information is also used in practically all machine vision systems for capturing and understanding dynamic scenes. This approach is seemingly supported by the way movies are made for human observers. The observation that visual motion appears smooth and continuous if viewed above a certain frame rate is, however, more related to characteristics of the human eye and brain than to the quality of the acquisition and encoding of the visual information as a series of still images. As soon as change or motion is involved, which is the case for almost all machine vision applications, the paradigm of visual frame acquisition becomes fundamentally flawed. If a camera observes a dynamic scene, no matter where you set your frame rate to, it will always be wrong. As different parts of a scene usually have different dynamic contents, a single sampling rate governing the exposure of all pixels in an imaging array will naturally fail to yield adequate acquisition of these different scene dynamics present at the same time.

Individual Pixel’s Sampling Points in Time

An ´ideal´ image sensor samples parts of the scene that contain fast motion and changes at high sampling rates and slow changing parts at slow rates, all at the same time – with the sampling rate going to zero if nothing changes. Obviously, this will not work using one common single sampling rate, the frame rate, for all pixels of a sensor. Conversely, one wants to have as many sampling rates as there are pixel in the sensor – and let each pixel’s sampling rate adapt to the part of the scene it sees. To achieve this requires putting each individual pixel in control of adapting its own sampling rate to the visual input it receives. This is done by introducing into each pixel a circuit that reacts to relative changes of the amount of incident light, so defining the individual pixel’s sampling points in time. As a consequence, the entire image data sampling process is no longer governed by a fixed (artificial) timing source (the frame clock) but by the signal to be sampled itself, or more precisely by the variations over time of the signal in the amplitude domain. The output generated by such a camera is no longer a sequence of images but a time-continuous stream of individual pixel data, generated and transmitted conditionally, based on what is happening in the scene.

Image 2 | The result of the exposure measurement is asynchronously output off the sensor together with the pixel’s coordinates in the sensor array. (Image: Prophesee)

Autonomously Operating Pixels

Following this paradigm, Prophesee has developed an image sensor containing an array of autonomously operating pixels that combine an asynchronous level-crossing detector with a separate exposure measurement circuit. Each exposure measurement by an individual pixel is triggered by a level-crossing event. Inspired by biology, every pixel in these sensors optimizes its own sampling depending on the visual information it sees. In case of rapid changes, the pixel samples at a high rate. On the contrary, if nothing happens, the pixel stops acquiring redundant data and goes idle until things start to happen again in its field of view. Hence each pixel independently samples its illuminance upon detection of a change of a certain magnitude in this same luminance, thus re-measuring its new light level after it has changed. The result of the exposure measurement (i.e. the new gray level) is asynchronously output off the sensor together with the pixel’s coordinates in the sensor array. As a result, image information is not acquired and transmitted frame-wise but continuously, and conditionally only from parts of the scene where there is new visual information. Or in other words, only information that is relevant – because unknown – is acquired, transmitted, stored and processed by machine vision algorithms. This way, both the acquisition of highly redundant and useless data by over-sampling static or slow parts of the scene, and the under-sampling of fast scene dynamics due to fixed frame rates, can be eliminated.

Image 3 | The results are no longer a sequence of images but a time-continuous stream of individual pixel data, generated and transmitted conditionally, based on what is happening in the scene. (Bild: Prophesee)

Image 3 | The results are no longer a sequence of images but a time-continuous stream of individual pixel data, generated and transmitted conditionally, based on what is happening in the scene. (Image: Prophesee)

Pixel acquisition and readout times of milliseconds to microseconds are achieved, resulting in temporal resolutions equivalent to conventional sensors running at tens to hundreds of thousands of frames per second. Now, for the first time, the strict temporal resolution vs. data rate tradeoff that limits all frame-based vision acquisition can be overcome. As the temporal resolution of the image data sampling process is no longer governed by a fixed clock driving all pixels, the data volume of the sensor output, independently of the temporal resolution available for the acquisition at the single pixel, is only depending on the dynamic contents of the visual scene. Visual data acquisition simultaneously becomes fast and sparse, leading to ultra-high-speed acquisition combined with reduced power consumption, transmission bandwidth and memory requirements.

Anzeige

Empfehlungen der Redaktion

Das könnte Sie auch interessieren

Die auf der CIS basierenden Line Scan Bars gibt es nun mit neuen Längen von 1.064, 1.247 und 1.688mm, die eine höhere Scanbreite als bisher abdecken. ‣ weiterlesen

de.mitsubishielectric.com

Anzeige

Die Allpixa Evo ist die erste Zeilenkamera mit Dual-10GigE-Schnittstelle. Sie liefert je nach Modell 10K oder 15K horizontale Auflösung bei vier Zeilen und ist als Farb- oder Monochromversion erhältlich.‣ weiterlesen

www.chromasens.de

Auf Basis der CoaXPress-V2.0-Schnittstelle liefert die Highspeed-Kamera Eosens 1.1CXP2 bei 1,1MP Auflösung bis zu 3.600fps. ‣ weiterlesen

www.mikrotron.de

Der hochauflösende 3D-Sensor DSMax bietet 2.000 Profilpunkte und Scanraten von bis zu 18kHz bei vollem Messbereich. Durch die Kombination aus telezentrischen Objektiv und gerichteter Laserlinie werden die abgeschatteten Bereiche reduziert.‣ weiterlesen

www.i-mation.de

Die 10GigE-Modelle der LX-Serie von Baumer ermöglichen die Ansteuerung von Autofokusobjektiven mit Flüssiglinsen zur schnellen und flexiblen Fokussierung bei sich ändernden Arbeitsabständen. ‣ weiterlesen

www.baumer.com

Erster Sensor-Neuzugang für die GigE-Vision-Kameras von Matrix Vision ist der 31MP-Sensor IMX342 von Sony. In Verbindung mit dem SmartFrameRecall Feature können auch die Auflösungen von 6.480x4.856 Pixel mit GigE und Dual GigE genutzt werden. ‣ weiterlesen

www.matrix-vision.de

Anzeige
Anzeige
Anzeige