3D Fluorescence Molecular Tomography (FMT) is an important pre-clinical tool for quantifying and characterizing molecular markers for different diseases. Herein, we demonstrate, for the first time, 3D K-Space FMT reconstructions using a large time-gated SPAD camera, SwissSPAD2 (SS2). The 3D reconstruction results are obtained using traditional and Deep Neural Network-based inverse solvers. Moreover, the 3D reconstruction performance with SS2 is benchmarked against that obtained with a gated-ICCD camera. The reconstruction results obtained using SS2 are in good agreement with those obtained from the gated-ICCD.
The SwissSPAD2/3 camera family is based on quarter megapixel single-photon avalanche diode (SPAD) time gated imagers. The 16.38-µm low-noise pixels feature a single-bit memory and built-in all-solid-state nanosecond time gating without the need for external image intensifiers. Microlenses have also been made available to increase the overall system sensitivity, including for high NA applications. SwissSPAD2/3 are coupled to FPGA platforms enabling a virtually noiseless streaming at up to 100 kpfs. A 1-bit accumulation of frames to reconfigurable number of bits was programmed on the FPGA for applications such as fluorescence lifetime imaging microscopy (FLIM). In other applications, a burst-mode read-out of 130,000 binary frames to a DDD3 memory of one sensor half was programmed on one FPGA for applications requiring full bitplanes. These initial configurations were extended to dual-FPGA systems capable of streaming data at near 100 kfps in continuous mode for long acquisition times. In such configuration one FPGA streams data from one sensor half to the other FPGA, which then sends the combined data stream to a host PC over PCIe at up to 3 GB/s. The eight PCIe lanes require careful design with differential routing and controlled impedance and the whole development presented significant hardware and firmware challenges. We also achieved full synchronization of two SwissSPAD2 camera systems over PCIe and characterized the pixel-to-pixel exposure timing alignment error to better than 150 ps with a time gate of 10 ns. The resulting platforms are unique enablers for quantum imaging applications, such as plenoptic maging, quantum LIDAR or quanta burst photography.
SignificanceFluorescence guidance is used clinically by surgeons to visualize anatomical and/or physiological phenomena in the surgical field that are difficult or impossible to detect by the naked eye. Such phenomena include tissue perfusion or molecular phenotypic information about the disease being resected. Conventional fluorescence-guided surgery relies on long, microsecond scale laser pulses to excite fluorescent probes. However, this technique only provides two-dimensional information; crucial depth information, such as the location of malignancy below the tissue surface, is not provided.AimWe developed a depth sensing imaging technique using light detection and ranging (LiDAR) time-of-flight (TOF) technology to sense the depth of target tissue while overcoming the influence of tissue optical properties and fluorescent probe concentration.ApproachThe technology is based on a large-format (512×512 pixel), binary, gated, single-photon avalanche diode (SPAD) sensor with an 18 ps time-gate step, synchronized with a picosecond pulsed laser. The fast response of the sensor was developed and tested for its ability to quantify fluorescent inclusions at depth and optical properties in tissue-like phantoms through analytical model fitting of the fast temporal remission data.ResultsAfter calibration and algorithmic extraction of the data, the SPAD LiDAR technique allowed for sub-mm resolution depth sensing of fluorescent inclusions embedded in tissue-like phantoms, up to a maximum of 5 mm in depth. The approach provides robust depth sensing even in the presence of variable tissue optical properties and separates the effects of fluorescence depth from absorption and scattering variations.ConclusionsLiDAR TOF fluorescence imaging using an SPAD camera provides both fluorescence intensity images and the temporal profile of fluorescence, which can be used to determine the depth at which the signal is emitted over a wide field of view. The proposed tool enables fluorescence imaging at a higher depth in tissue and with higher spatial precision than standard, steady-state fluorescence imaging tools, such as intensity-based near-infrared fluorescence imaging, optical coherence tomography, Raman spectroscopy, or confocal microscopy. Integration of this technique into a standard surgical tool could enable rapid, more accurate estimation of resection boundaries, thereby improving the surgeon’s efficacy and efficiency, and ultimately improving patient outcomes.
SwissSPAD3 is the latest of a family of widefield time-gated SPAD imagers developed for fluorescence lifetime imaging (FLI) applications. Its distinctive features are (i) the ability to define shorter gates than its predecessors (width W < 1 ns), (ii) support for laser repetition rates up to at least 80 MHz and (iii) a dual-gate architecture providing an effective duty cycle of 100%. We present widefield macroscopic FLI measurements of short lifetime NIR dyes, analyzed using the phasor approach. The results are compared with those previously obtained with SwissSPAD2 and to theoretical predictions.
Semiconductor nanocrystals feature multiply-excited states that display intriguing physics and significantly impact nanocrystal-based technologies. Fluorescence supplies a natural probe to investigate these states. Still, direct observation of multiexciton fluorescence has proved elusive to existing spectroscopy techniques. Heralded Spectroscopy is a new tool based on a breakthrough single-particle, single-photon, sub-nanosecond spectrometer that utilizes temporal photon correlations to isolate multiexciton emission. This proceedings paper introduces Heralded Spectroscopy and reviews some of the novel insights it uncovered into exciton–exciton interactions within single nanocrystals. These include weak exciton–exciton interactions and their correlation with quantum confinement, biexciton spectral diffusion, multiple biexciton species and biexciton emission polarization.
Silicon-based single-photon avalanche diodes (SPADs) implemented in front-side illuminated arrays and imagers have often suffered from fill factor limitations. The corresponding reduced sensitivity can be sometimes traded off with longer acquisition times thanks to SPAD’s noiseless read-out. The use of SPADs can however be critically affected in many applications, especially when photon-starved, or when several photons need to be detected in coincidence. The fill factor loss can be recovered by employing microlens arrays, which are difficult to build with relatively large pitch (> 10 μm) and low native SPAD fill factor (as low as 10%). To address these challenges, we have developed several generations of refractive microlenses by photoresist reflow used to fabricate molds. These structures were used to imprint UV-curable hybrid polymer microlenses on SPAD arrays. Replications were successfully carried out on large SPAD arrays with very thin residual layers (~10 μm), as required for higher numerical aperture (NA > 0.25). Replications were also carried out for the first time in a multi-chip operation regime at the wafer reticle level. By optimizing the lens sag and residual layer thickness, concentration factors (CFs) within 15-20% of the theoretical maxima were obtained for the smaller arrays (32×32 and 512×1). The spectral response was flat above 400 nm. CF values up to 4.2 with good uniformity were measured on large 512×512 arrays with 16 μm pixel pitch and a native fill factor of 10.5%. This result was confirmed by simulations when using the actual measured lens shape. We thus demonstrated good spectral and spatial uniformity and high CF, while moving to higher NAs and larger sensor sizes with respect to previous work.
The LinoSPAD2 camera combines a 512×1 linear single-photon avalanche diode (SPAD) array with an FPGA-based photon-counting and time-stamping platform, to create a reconfigurable sensing system capable of detecting single photons. The read-out is fully parallel, where each SPAD is connected to a different FPGA input. The hardware can be reconfigured to achieve different functionalities, such as photon counters, time-to-digital converter (TDC) arrays and histogramming units. Time stamping is performed by an array of 64 TDCs, with 20 ps resolution (LSB), serving 256 channels by means of 4:1 sharing. At sensor level, the pixel pitch is 26.2 μm with a fill factor of 25.1%. The median dark count rate of each SPAD at room temperature is below 100 cps at 6V excess bias, the single-photon timing resolution (SPTR) of each channel is 50 ps FWHM, and the peak photon detection probability reaches ~50% at 510 nm at the same excess bias. The fill factor can be increased by 2.3× by means of microlenses, with good spatial uniformity and flat spectral response above 400 nm. At system level, the average instrument response function (IRF) is 135 ps FWHM. The LinoSPAD2 camera enables a wide range of time-of-flight and time-resolved applications, including 3D imaging, fluorescence lifetime imaging microscopy (FLIM), heralded spectroscopy, and compressive Raman imaging, to name a few. Thanks to its features, LinoSPAD2 is a novel generation of reconfigurable single-photon image sensors capable of adapting their read-out and processing to match application-specific requirements, and combining SPAD arrays with advanced, massively-parallel computational functionalities.
Since the introduction of the intensity interferometer by Hanbury Brown and Twiss in the '50s and its subsequent adaptations to quantum optic experiments, photon correlations have played a significant role in optical research. An emerging detector technology, SPAD arrays, presents new opportunities to scale these experiments and exploit photon correlations in new regimes. I will describe a new temporal-spectral photon correlation technique based on these detectors, single-particle heralded spectroscopy. I will present new insights into the photophysics of multiply-excited semiconductor quantum dots uncovered by this method, hitherto inaccessible to existing approaches.
KEYWORDS: In vivo imaging, Fluorescence lifetime imaging, Sensors, In vitro testing, Visible radiation, Tumors, Near infrared, Luminescence, Fluorescence resonance energy transfer, Detector development
Near-Infrared wide-field Fluorescence Lifetime Imaging (FLI) has become an increasingly popular method due to its unique specificity in sensing the cellular micro-environment and/or protein-protein interactions via FRET, but the approach is still challenging due to inefficient detection modules. Here, we report on the characterization of a large gated SPAD array, SwissSPAD2, towards in vivo preclinical imaging of FLI-FRET. Fluorescence decay fitting as well as phasor analysis are used to demonstrate the ability of SwissSPAD2 to accurately quantify short lifetimes and associated lifetime parameters in both in vitro and in vivo experiments, in full agreement with gated ICCD measurements.
Classical fluorescence guided surgery uses long laser pulses to excite fluorescent tracers, only providing the practitioner with 2D information. In this work, we develop and test a 3D imaging technique using LiDAR technology to sense target tissue depths up to 5 mm below the surface. Imaging through highly scattering medium like tissues implies that temporal information used for 3D reconstruction is convolved with diffuse dynamics of the medium. Using diffuse reflectance as an indicator of the medium’s optical properties, correction of the time-of-flight is made possible. Time-of-flight fluorescence imaging, captures intensity as well as temporal profile of fluorescence excited by ultra-short pulses.
The performance of SwissSPAD2 (SS2), a large scale, widefield time-gated CMOS SPAD imager developed for fluorescence lifetime imaging, has recently been described in the context of visible range and fluorescence lifetime imaging microscopy (FLIM) of dyes with lifetimes in the 2.5 – 4 ns range. Here, we explore its capabilities in the NIR regime relevant for small animal imaging, where its sensitivity is lower and typical NIR fluorescent dye lifetimes are much shorter (1 ns or less). We carry out this study in a simple macroscopic imaging setup based on a compact NIR picosecond pulsed laser, an engineered diffuser-based illumination optics, and NIR optimized imaging lens suitable for well-plate or small animal imaging. Because laser repetition rates can vary between models, but the synchronization signal frequency accepted by SS2 is fixed to 20 MHz, we first checked that a simple frequency-division scheme enables data recording for different laser repetition rates. Next, we acquired data using different time gate widths, including gates with duration longer than the laser period, and analyzed the resulting data using both standard nonlinear least-square fit (NLSF) and phasor analysis. We show that the fixed synchronization rate and large gate widths characterizing SS2 (10 ns and over) are not an obstacle to accurately extracting lifetime in the 1 ns range and to distinguishing between close lifetimes. In summary, SS2 and similar very large gated SPAD imagers appear as a versatile alternative to other widefield time-resolved detectors for NIR fluorescence lifetime imaging, including preclinical molecular applications.
We introduce a new generation of 3D imaging devices based on quantum plenoptic imaging. Position-momentum entanglement and photon number correlations are exploited to provide a scan-free 3D image after post-processing of the collected light intensity signal. We explore the steps toward designing and implementing quantum plenop- tic cameras with dramatically improved performances, unattainable in standard plenoptic cameras, such as diffraction-limited resolution, large depth of focus, and ultra-low noise. However, to make these new types of devices attractive to end-users, two main challenges need to be tackled: the reduction of the acquisition times, that for the commercially available high-resolution cameras would be from tens of seconds to a few minutes, and a speed-up in processing the large amount of data that are acquired, in order to retrieve 3D reconstructions or refocused 2D images. To address these challenges, we are employing high-resolution SPAD (single photon avalanche diode) arrays and high-performance low-level programming of ultra-fast electronics, combined with compressive sensing and quantum tomography algorithms, with the aim of reducing both the acquisition and the elaboration time by one or possibly two orders of magnitude. Moreover, in order to achieve the quantum limit and further increase the volumetric resolution beyond the Rayleigh diffraction limit, we explored dedicated pro- tocols based on quantum Fisher information. Finally, we discuss how this new generation of quantum plenoptic devices could be exploited in different fields of research, such as 3D microscopy and space imaging.
We present a 1 megapixel single-photon avalanche diode (SPAD) camera featuring 3.8 ns time gating and 24 kfps frame rate for 1-bit images, fabricated in 180 nm CMOS image sensor technology. The SPAD sensor was used to capture 2D and 3D scenes over 2 m with depth resolution of 5.4 mm and precision better than 7.8 mm (rms). We demonstrate extended dynamic range in dual exposure operation mode and show spatially overlapped multi-object detection in single-photon time-gated time-of-flight experiments. We further demonstrate applications of the megapixel SPAD camera for fluorescence lifetime imaging microscopy (FLIM) and light-in-flight imaging.
Developing large arrays of single-photon avalanche diodes (SPADs) with on-chip time-correlated single-photon counting (TCSPC) capabilities continues to be a difficult task due to stringent silicon real estate constraints, high data rates and system complexity. As an alternative to TCSPC, time-gated architectures have been proposed, where the numbers of photons detected within different time gates are used as a replacement to the usual time-resolved luminescence decay. However, because of technological limitations, the minimum gate length implement is on the order of nanoseconds, longer than most fluorophore lifetimes of interest. However, recent FLIM measurements have shown that it is mainly the gate step and rise/fall time, rather than its length, which determine lifetime resolution. In addition, the large number of photons captured by longer gates results in higher SNR. In this paper, we study the effects of using long, overlapping gates on lifetime extraction by phasor analysis, using a recently developed 512×512 time-gated SPAD array. The experiments used Cy3B, Rhodamine 6G and Atto550 dyes as test samples. The gate window length was varied between 11.3 ns and 23 ns while the gate step was varied between 17.86 ps and 3 ns. We validated the results with a standard TCSPC setup and investigated the case of multi-exponential samples through simulations. Results indicate that lifetime extraction is not degraded by the use of longer gates, nor is the ability to resolve multi-exponential decays.
Single-photon avalanche diode (SPAD) imagers can perform fast time-resolved imaging in a compact form factor, by exploiting the processing capability and speed of integrated CMOS electronics. Developments in SPAD imagers have recently made them compatible with widefield microscopy, thanks to array formats approaching one megapixel and sensitivity and noise levels approaching those of established technologies. In this paper, phasor-based FLIM is demonstrated with a gated binary 512×512 SPAD imager, which can operate with a gate length as short as 5.75 ns, a minimum gate step of 17.9 ps, and up to 98 kfps readout rate (1-bit frames). Lifetimes of ATTO 550 and Rhodamine 6G (R6G) solutions were measured across a 472×256 sub-array using phasor analysis, acquiring data by shifting a 13.1 ns gate window across the 50 ns laser period. The measurement accuracy obtained when employing such a scheme based on long, overlapping gates was validated by comparison with TCSPC measurements and fitting analysis results based on a standard Levenberg-Marquardt algorithm (<90% accuracy for the lifetime of R6G and ATTO 550). This demonstrates the ability of the proposed method to measure short lifetimes without minimum gate length requirements. The FLIM frame rate of the overall system can be increased up to a few fps for phasor-based widefield FLIM (moving closer to real-time operation) by FPGA-based parallel computation with continuous acquisition at the full speed of 98 kfps.
Single-photon avalanche diodes (SPADs) are direct photon-to-digital detectors that enable scalable arrays with Poisson-limited signal-to-noise ratio and picosecond timing resolution. However, SPAD detectors require a guard-ring structure to prevent lateral edge breakdown. The guard ring, in addition to pixel electronics, reduces the sensitive area within the pixel, often below 50%. We present the simulation, design and characterization of microlens structures to increase the effective fill factor and SPAD photon detection efficiency. The main challenges in designing microlenses for SPADs are a relatively large SPAD pitch and a low native fill factor, requiring high microlens efficiency over a wide angular distribution of light. In addition, we addressed the requirements of several designs in the same technology, featuring native fill factors which range from 10.5% to 28%, by carrying out the microlens fabrication at wafer reticle level. The fabrication process starts with creating a photoresist microlens master, used to fabricate a mould for microlens imprints. After dispensing a UV curable hybrid polymer on top of the SPAD array, the mould is used to imprint the microlens array shape, and then cured with UV exposure. By using microlenses, we were able to increase the initial fill factor to more than 84% effective fill factor for a 28.5 μm pixel pitch. We also explore the influence of the passivation layer on the SPAD photon detection efficiency.
Single-photon avalanche diode (SPAD) arrays have recently emerged as promising detectors for many wide-field fluorescence lifetime imaging microscopy (FLIM) applications, thanks to their picosecond range temporal resolution, single-photon sensitivity and fast parallel readout. However, ultra-fast SPAD cameras with very large number of pixels (> 100,000) equipped with time-correlated single-photon counting (TCSPC) capabilities for each pixel have not been developed to date, due to the difficulty with implementing compact in-pixel ps-timestamping. Here, we report a time-resolved image sensor for real-time FLIM comprising the largest SPAD array reported so far. Thanks to its in-pixel time-gated architecture, this 512×512 pixels sensor can perform time-resolved photon-counting at a maximum of 97 kfps. The state-of-the-art photon detection probability and dark count rate of the SPAD allow the camera to collect enough photons for lifetime extraction in low light conditions at video rate. To reach the timing performance obtained by conventional time-to-digital converters (TDC) architectures, we used a time-gated technique based on large (5 ns) time-gates shifted by picosecond delays. Using a finite number of overlapping gates, fluorescence lifetime can be accurately recovered. This architecture and the corresponding FLIM data is ideally suited for phasor analysis, a powerful and computationally simple method to extract fluorescence lifetime information, which allows real-time monitoring of FLIM information provided by such large imagers.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.