Capturing and reconstructing transient images with single-photon avalanche diodes (SPAD) at interactive rates.

Abstract

Active imaging at the picosecond timescale reveals transient light transport effects otherwise not accessible by com- puter vision and image processing algorithms. For example, analyzing the time of flight of short laser pulses emitted into a scene and scattered back to a detector allows for depth imaging, which is crucial for autonomous driving and many other applications. Moreover, analyzing or removing global light transport effects from photographs becomes feasible. While several transient imaging systems have recently been proposed using various imaging technologies, none is capable of acquiring transient images at interactive frame rates. In this paper, we present an imaging system that leverages single-photon avalanche diodes together with a pulsed picosecond laser to record transient images with up to 25 Hz at a low spatial resolution of 64 x 80 pixels or 1 Hz at a moderate resolution of 256 x 250 pixels. We show several transient video clips recorded with this system and demonstrate transient imaging applications, including direct-global light transport separation and enhanced depth imaging.

Presentation

Video

Dataset

• Raw Data
• Processing Code

Acknowledgments

The authors are grateful to Edoardo Charbon, Pierre-Yves Cattin, and Samuel Burri for providing the LinoSPAD sensor used in this work and continued support of it. O’Toole is supported by the Government of Canada through the Banting Postdoctoral Fellowships program, and Wetzstein is supported by a National Science Foundation CAREER award (IIS 1553333), a Terman Faculty Fellowship, a Sloan Fellowship, the Center for Automotive Research at Stanford (CARS), and by the KAUST Office of Sponsored Research through the Visual Computing Center CCF grant.

Citation

@article{Lindell:2018:SPAD,
author = {D. Lindell and M. O’Toole and G. Wetzstein},
title = {{Towards Transient Imaging at Interactive Rates with Single-Photon Detectors}},
journal = {Proc. IEEE ICCP},
year = {2018},
}