The Dynamic and Active Pixel Vision Sensor (DAVIS), developed by iniLabs and INI sensors group represented a significant advancement in neuromorphic vision technology. It combined the asynchronous event-based sensing capabilities of the Dynamic Vision Sensor (DVS) with conventional active pixel sensor (APS) imaging, enabling hybrid functionality that delivers both event streams and frames. I can’t stop feeling some nostalgia while writing this article for the memories of having work so hard on the development of the whole DAVIS range, and also for having coined the DAVIS name myself, but I promise I’ll the rest of the post professional.
Introduction to Neuromorphic Vision
Neuromorphic engineering draws inspiration from the architecture and functioning of the human brain to design electronic systems. In the context of vision, neuromorphic sensors aim to replicate the efficiency and adaptability of biological vision systems. Traditional cameras capture images at a fixed frame rate, which can lead to redundant data and inefficiencies, especially in dynamic environments. In contrast, neuromorphic vision sensors like the DAVIS camera detect and respond to changes in the visual scene as they occur, significantly reducing data redundancy and power consumption.
The DAVIS Camera: A Hybrid Approach
The DAVIS camera is a hybrid sensor that combines the strengths of both frame-based and event-based vision systems. It integrates a Dynamic Vision Sensor (DVS) with an Active Pixel Sensor (APS) within the same pixel array. This dual functionality allows the DAVIS camera to capture both asynchronous events and synchronous frames, providing a comprehensive view of the environment.
Active Pixel Sensor (APS): The APS component captures traditional frames at a fixed rate, providing detailed spatial information about the scene. This is useful for applications that require high-resolution images or when the scene is static.
Dynamic Vision Sensor (DVS): The DVS component of the DAVIS camera detects changes in light intensity for each pixel independently and asynchronously. This results in a stream of events that represent the temporal dynamics of the scene, making it highly efficient for capturing fast-moving objects or changes in illumination.
What Sets DAVIS Apart from DVS?
The Dynamic Vision Sensor (DVS) is a neuromorphic sensor that detects changes in brightness asynchronously at the pixel level. Each pixel generates an event when there is a logarithmic change in light intensity, allowing DVS to achieve low latency (<1 ms), high dynamic range (>120 dB), and reduced data rates compared to traditional cameras. However, DVS lacks the ability to capture absolute illumination levels or conventional image frames.
DAVIS addresses these limitations by integrating APS technology into the same pixel array as the DVS. This integration enables DAVIS to provide:
- Absolute Illumination Frames: Conventional images captured at regular intervals choosing in between rolling or global shutter by software.
- Event Streams: Asynchronous brightness change events generated independently by each pixel.
This hybrid functionality allows DAVIS to encode visual information between frames using asynchronous events, bridging the gap between traditional frame-based imaging and event-based sensing.
How DAVIS Pixels Work
Each pixel in DAVIS combines two sensing mechanisms:
- Active Pixel Sensor (APS): Captures grayscale images by measuring absolute illumination levels during a fixed exposure time. This is similar to conventional cameras.
- Dynamic Vision Sensor (DVS): Detects relative changes in brightness asynchronously. When the logarithmic change in intensity exceeds a threshold, an “ON” or “OFF” event is generated based on the polarity of the change.
The APS provides spatial context and absolute brightness values, while the DVS offers high temporal resolution for detecting motion or changes in the scene. This dual-mode operation ensures that DAVIS can adapt to diverse lighting conditions and dynamic environments.
Address-Event Representation (AER) Bus
The information generated by DAVIS pixels is transmitted using the Address-Event Representation (AER) bus, a communication protocol commonly used in neuromorphic sensors. The AER bus encodes events as asynchronous spikes with unique addresses corresponding to the pixel location and polarity of brightness change .
Key features of AER transmission include:
- Asynchronous Communication: Events are transmitted only when they occur, minimizing data redundancy.
- High Temporal Precision: Events are timestamped with microsecond resolution, allowing accurate tracking of fast-moving objects.
- Sparse Data Encoding: Only pixels experiencing brightness changes generate events, reducing bandwidth requirements compared to frame-based systems.
This efficient data transmission mechanism enables DAVIS to handle high-speed scenarios while maintaining low power consumption.
Technical Comparison: DAVIS vs. DVS
| Feature | DAVIS | DVS |
|---|---|---|
| Output Modes | Grayscale frames + asynchronous events | Asynchronous events only |
| Dynamic Range | 130 dB | >120 dB |
| Latency | <1 ms | <1 ms |
| Pixel Complexity | ~45 transistors per pixel | Simpler design (~20 transistors) |
| Fill Factor | ~20% due to complex design | Higher fill factor |
| Applications | Hybrid tasks (e.g., feature tracking) | Motion detection |
While DVS excels in scenarios requiring minimal data output and high-speed motion detection, DAVIS provides additional spatial context through grayscale frames, making it more versatile for applications such as robotics, autonomous vehicles, and surveillance.
DAVIS Sensor Variants
The DAVIS family includes several variants, each tailored for specific applications and performance requirements. Below is an overview of the main DAVIS sensors:
| Feature | DAVIS 240C | DAVIS 346 | DAVIS 640 | DAVIS 640 BSI |
|---|---|---|---|---|
| Resolution | 240 x 180 pixels | 346 x 260 pixels | 640 x 480 pixels | 640 x 480 pixels |
| Dynamic Range | 120 dB | 120 dB | 120 dB | 120 dB |
| Color Sensitivity | Yes (Bayer filter) | Yes (Bayer filter) | No | No |
| IMU Integration | No | Yes | No | No |
| Low-Light Performance | Standard | Standard | Standard | Enhanced (BSI technology) |
| Use Cases | Robotics, color-sensitive applications | High-resolution imaging, motion tracking | Detailed spatial imaging | Low-light imaging, enhanced sensitivity |
Applications of the DAVIS Camera
The unique capabilities of the DAVIS camera make it suitable for a wide range of applications, including:
- Robotics: The low latency and high dynamic range of the DAVIS camera make it ideal for robotic vision systems that require real-time processing and adaptation to changing environments.
- Automotive: The DAVIS camera’s ability to capture fast-moving objects and adapt to varying lighting conditions makes it suitable for advanced driver assistance systems (ADAS) and autonomous vehicles.
- Surveillance: The event-based nature of the DAVIS camera allows for efficient monitoring of dynamic scenes, reducing data redundancy and power consumption in surveillance systems.
- Scientific Research: The DAVIS camera’s hybrid approach provides valuable insights into both temporal and spatial dynamics, making it a powerful tool for scientific research in fields such as neuroscience and computer vision.
Conclusion
The DAVIS sensor represents a breakthrough in neuromorphic vision technology by combining the strengths of DVS and APS into a single device. Its ability to output both event streams and conventional frames makes it ideal for tasks requiring high temporal resolution alongside spatial fidelity. With its efficient AER bus communication and hybrid sensing capabilities, DAVIS is paving the way for advanced machine vision systems across diverse fields such as space exploration, autonomous driving, and medical imaging.
References
