Professor Evan Y. Peng, from the Department of Electrical and Electronic Engineering and the Department of Computer Science, and Professor Jia Pan from the Department of Computer Science and their team, conducted research on the topic “Embodied Neuromorphic Synergy for Lighting-Robust Machine Vision to See in Extreme Bright Conditions.” The research findings were published by Nature Communications on 30 December, 2024.
Details of the publication:
Embodied neuromorphic synergy for lighting-robust machine vision to see in extreme bright
Shijie Lin, Guangze Zheng, Ziwei Wang, Ruihua Han, Wanli Xing, Zeqing Zhang, Yifan Peng & Jia Pan
Article in Nature Communications
https://www.nature.com/articles/s41467-024-54789-8
Abstract
Proper exposure settings are crucial for modern machine vision cameras to accurately convert light into clear images. However, traditional auto-exposure solutions are vulnerable to illumination changes, splitting the continuous acquisition of unsaturated images, which significantly degrades the overall performance of underlying intelligent systems. Here we present the neuromorphic exposure control (NEC) system. This system effectively alleviates the longstanding saturation problem at its core by exploiting bio-principles found in peripheral vision to compute a trilinear event double integral (TEDI). This approach enables accurate connections between events and frames in the physics space for swift irradiance prediction, ultimately facilitating rapid control parameter updates. Our experimental results demonstrate the remarkable efficiency, low latency, superior generalization capability, and bio-inspired nature of the NEC in delivering timely and robust neuromorphic synergy for lighting-robust machine vision across a wide range of real-world applications. These applications encompass autonomous driving, mixed-reality, and three-dimensional reconstruction.