This PDF file contains the front matter associated with SPIE Proceedings Volume 12846, including the Title Page, Copyright information, Table of Contents, and Conference Committee information.

We developed a semi-supervised deep-learning-based system classifying different types of red blood cells (RBCs) images based on their shape, texture, and size. Specifically, pre-training a convolutional neural network was done on over 35,000 brightfield images of RBCs acquired with an imaging flow cytometer from a post-COVID-19 patient cohort. The system utilizes object localization powered by a YOLO-inspired block for cell identification and a de-blurring CNN block based on FocalNet. A series of convolutional and fully connected layers classifies images into side-view, biconcave, elongated, and additional categories for reticulocytes and erythrocytes. Fine-tuning was done using 7,000 manually labeled brightfield images. Consequent evaluation on a test dataset of 3,000 samples yielded an accuracy of 98.2%. This system can be used for other cell analysis tasks, not requiring large fine-tuning datasets while maintaining high efficiency.

Highly multiplexed fluorescence microscopy is an emerging technology that allows for spatial analysis of increasingly more classes of cells within human tissue—state-of-the-art methods are now probing up to 60 different protein markers within an image. This level of phenotypic resolution is ideal for uncovering the spatial underpinnings of immune cell interactions. However, defining cell types from this high-plex data is non-trivial. We present a method that borrows from hyperspectral image analysis to improve the accuracy and efficiency of immune cell classification in highly multiplexed fluorescence microscopy images. Treating the protein marker image channels as the spectral dimension of the images, we define reference “pseudospectra” representative of the ideal marker expression for all cell types of interest probed by the marker panel. Cosine similarity is computed for each reference pseudo-spectra to create class maps for each cell type in question. Features are extracted from these class maps—rather than the fluorescence images. We compare these methods to a decision-tree based classification method for classifying immune cells and unsupervised K-means clustering of mean pixel intensities across all image channels. We demonstrate that pSAM performs comparably, and potentially outperforms methods with similar levels of supervision.

Breast cancer cell analysis has traditionally focused on cell and intracellular organelle morphology. Recent research has demonstrated that organelle topology-based cancer cell classification is considerably more accurate when using handcrafted feature extraction and machine learning-based classifiers on fluorescent confocal microscopy images. However, feature extraction and classification through this methodology requires manual segmentation and computational organelle rendering. Herein, we employ convolutional neural networks (CNN) and Gradient-weighted Class Activation Mapping (GradCAM) for fast end-to-end classification and visual interpretation of confocal fluorescent microscopy images based on spatial organelle features. First, raw 3D images are filtered and preprocessed into 2D image patches for the CNN. To replicate feature analysis of the surface-surface contact area, marginal intermediate fusion CNN is implemented to classify each patch. GradCAM is then used post hoc to generate a representative heatmap of important areas used for classification. All relevant heatmap patches are then reconstructed based on the extraction of their respective patches to obtain an overall heatmap of the entire microscopy image. Furthermore, finer-grained heatmaps were obtained through the use of patch overlap and weighting during initial patch preprocessing. On a dataset consisting of 6 different breast cancer cell lines, this methodology resulted in a classification accuracy of 95.7% while also providing visualization of areas indicative of certain cancer cell lines. These findings demonstrate the efficacy of using deep learning and GradCAM for fast and interpretable organelle-based cancer cell classification.

Small extracellular vesicles (sEVs), which are nanoparticles around 100 nm, have been widely studied in recent years in many interesting areas, such as cancer detection and drug delivery. Bulk analysis of extracellular vesicles provides average information about the EV population. However, single EV characterization enables a profound understanding of the biophysical properties of EV subpopulations, establishing an insightful view of the EVs function and composition. It is worth to explore light scattering imaging method for the analysis of single sEVs. We introduce here the deep-learning-based light scattering imaging method for analyzing label-free sEVs (DeepEVAnalyzer), which has been applied to measure the size of single sEVs. We also report our recent development of a light scattering imaging method to address the inverse problem, which is demonstrated to differentiate the label-free sEVs from healthy mice and those injected with malignant cells. Light scattering imaging together with machine learning for sEVs analysis may have potential diagnostic and therapeutic applications.

Optical coherence tomography (OCT) has become a promising tool for studying anatomical and functional dynamics of the cerebral cortex, offering advantages such as label-free imaging, high resolution, and non-invasive optical biopsy. However, observing the brains of non-anesthetized and freely moving mice has been a long-standing challenge for OCT. In this study, we designed a wearable OCT probe to observe the vascular morphology of the mouse brain and track short-term vascular changes after thrombosis. We utilized a microelectromechanical system (MEMS) scanning mirror for three-dimensional scanning. Compared to traditional OCT systems, this wearable imaging probe features miniaturization, low cost, portability, and stability, allowing for imaging of the mouse brain in a non-anesthetized and freely moving state. The entire probe weighs 8 g and achieves a lateral resolution of 5.5 μm, a longitudinal resolution of 12 μm, and an effective imaging area of 4 mm × 4 mm. We evaluated the performance of the probe through phantom experiments and imaging of the mouse brain's vascular network, and successfully monitored local vascular morphological changes in the mouse brain shortly after stroke under awake conditions. We believe that the wearable probe can be applied in various fields such as ophthalmology, dermatology, and dentistry, and due to its portability and non-invasiveness, the wearable OCT probe is expected to have wide clinical research applications.

In the context of biomaterials, small-molecules and drugs testing, intravital microscopy allows to quantify in-vivo the immune reaction, reducing the number of laboratory animals required to statistically validate the product. However, fluorescence microscopy is affected by limited tissue penetration due to light scattering and by optical aberrations, induced on focused beams, by the animal tissue surrounding the implant. In this framework, we developed a system of microlenses coupled to microscaffolds, both incorporated in a miniaturized imaging window. The chip is designed to act as an in-situ microscope objective with the aim to overcome the restrictions of in-vivo imaging (i.e. spherical aberrations) and to allow multiple biological observations in the same animal (by including fluorescent beacons). The device is fabricated by two-photon polymerizing a biocompatible photoresist called SZ2080. The microlenses are manufactured by the concentric polar scanning of the laser beam to realize their outer surface, followed by the UV bulk polymerization of their inner SZ2080. We preliminarily characterized the imaging capabilities of our implantable system on live cells cultured on flat substrates and 3D microscaffolds by coupling it to low magnification objectives. The microlenses optical quality is sufficient to induce non-linear excitation and collect two-photon excitation images with the same level of laser intensity and signal-to-noise ratio. Remarkably, they allow to efficiently excite the fluorescence of labelled human fibroblasts collecting high resolution magnified images. These results will open the way to the application of implanted micro-optics for the real-time and continuous in-vivo observation of complex biological processes.

Imaging flow cytometry (IFC) has been widely applied in biomedical research due to its numerous advantages, including multiparametric analysis, microscopic imaging and high-throughput detection. Previous research in our lab has demonstrated the effectiveness of two-dimensional light scattering (LS) and brightfield (BF) dual-modality imaging techniques for detecting and distinguishing unlabeled cells. As fluorescence (FL) imaging techniques are sensitive to specifically labeled cells, here we introduce a single-detector IFC enabling simultaneous imaging of LS signals and BF/FL signals for automatic single-cell analysis with deep learning. The special optical design with a knife-edge right angle (KERA) prism is adopted to simultaneously capture corresponding LS patterns in defocus and BF/FL patterns in focus on a single detector. The LS and BF dual-modality flow imaging results of 2 μm and 3.87 μm unlabeled microspheres can be obtained by our system, which can also simultaneously acquire LS and FL results for fluorescent microspheres of 2 μm and 4 μm in diameter. The results of these beads demonstrate excellent agreement between LS patterns and Mie scattering simulations. The obtained LS and BF dual-modality cell images of A2780 and Hey cells are analyzed using a visual geometry group 19 (VGG19) deep learning method through feature extraction and fusion to show accurate classification of ovarian cancer cell subtypes. In conclusion, our development of a single-detector imaging flow cytometer enables the simultaneous capture of two-dimensional light-scattering and fluorescence/brightfield images, where an automatic analysis with deep learning can be performed, showcasing potential wide applications in biomedicine.

Autofluorescent metabolic measurements of an optical redox ratio, NAD(P)H/FAD, have been utilized as a means of measuring cancer progression, treatment impact, subtype determination, and more. This optical redox ratio is traditionally measured through intensity with microscopy, but there is potential to adapt this technique for high throughput analysis using time-resolved flow cytometry with autofluorescence lifetime measurements. A fluorescence lifetime approach to these measurements allows for fluorophore concentration independent measurements that can provide new information to the field. The two fluorescent metabolites of interest that allow for redox analysis are NAD(P)H/NAD(P)⁺ and FADH₂/FAD. Variations in the redox state and binding of these metabolites to their respective coenzymes, have been correlated to the cycles in which cells metabolize glucose into ATP, either oxidative phosphorylation (OXPHOS) or glycolysis. This technique will be used herein to study the metabolism of MCF-7 tamoxifen resistant and sensitive breast cancer cells using flow cytometry, first on a fluorescence intensity basis of the two metabolites. Our results show there is a clear shift towards an increased redox ratio for tamoxifen resistant cells, indicating a greater reliance on glycolysis as a means of metabolism. Future work will focus on adapting the intensity based redox ratio approach through high-throughput flow cytometry used here to a fluorescence lifetime based measurement.

We demonstrate the capability of a time-resolved autofluorescence lifetime imaging setup for discriminating, perilesional and tumor tissues in freshly excised liver samples. In particular, freshly excised liver biopsies were collected from the surgery room and imaged within 30 minutes using 445 nm as excitation wavelength and one spectral window for detection. Differences in mean fluorescence lifetime were observed among the examined tissue types, potentially allowing their discrimination and classification. Interestingly, the obtained autofluorescence lifetime values were significantly different when comparing primary tumors of liver with colorectal tumor metastasis to the liver, underlying the classification capability of this experimental setup. The presented approach offers real-time acquisition and processing, optical flexibility, as well as the possibility to acquire autofluorescence data under bright background conditions. Hence, it meets all the requirements for label-free diagnostics and surgical guidance in various clinical and histopathological applications.

Hyperspectral imaging (HSI) technologies have enabled a range of experimental techniques and studies in the fluorescence microscopy field. Unfortunately, a drawback of many HSI microscope platforms is increased acquisition time required to collect images across many spectral bands, as well as signal loss due to the need to filter or disperse emitted fluorescence into many discrete bands. We have previously demonstrated that an alternative approach of scanning the fluorescence excitation spectrum can greatly improve system efficiency by decreasing light losses associated with emission filtering. Our initial system was configured using an array of thin-film tunable filters (TFTFs, VersaChrome, Semrock) mounted in a tiltable filter wheel (VF-5, Sutter) that required ~150-200 ms to switch between wavelengths. Here, we present a new configuration for high-speed switching of TFTFs to allow rapid time-lapse HSI microscopy. A TFTF array was mounted in a custom holder that was attached to a piezoelectric rotation mount (ThorLabs), allowing high-speed rotation. Switching between adjacent filters was achieved using the internal optics of a DG-4 lightsource (Sutter Instrument), including a pair of off-axis parabolic mirrors and galvanometers. Output light was coupled to a liquid lightguide and into an inverted widefield fluorescence microscope (TI-2, Nikon Instruments). Initial tests indicate that the HSI system provides a 15-20 nm bandwidth tunable excitation band and ~10-20 ms wavelength switch time, allowing for high-speed HSI imaging of dynamic cellular events. This work was supported by NIH P01HL066299, R01HL169522, NIH TL1TR003106, and NSF MRI1725937.

We demonstrate an early-stage prototype of an image-guided laser ablation system for precise 3D sampling of fresh-frozen tissue samples for subsequent analysis of biomolecules. Utilizing an integrated reflected light microscope and optical coherence tomography, pre-recorded image data (e.g. of characterized tissue slices) can be registered and mapped to the coordinate system for precise aiming and ablation. With this system, we are analyzing a variety of tissue samples from our many collaborators at the University Cancer Center Hamburg (UCCH) for cancer research and demonstrate the system’s feasibility through initial use cases.

Near-field optical tweezers on a chip based on silicon-on-insulator (SOI) platform have drawn significant attention as prominent particle manipulation tools in chemistry, biology, and materials science. In this work, we demonstrate a lowloss tapered silicon waveguide with a high intensity gradient to trap microparticles. The high transmittance of this waveguide makes it easy to cascade many traps along the direction of light propagation. Optical forces in all three dimensions are analyzed using an in-house modeling toolkit. Experimentally, an integrated waveguide-based optical tweezer with a cascade of five trapping units is fabricated by electron beam lithography and reactive ion etching. Yeast cells are successfully trapped at different trapping units in the cascaded tapered waveguide using only around 20 mW of optical power at 1550-nm wavelength. We believe that the proposed scheme exhibits great potential for applications in biological analysis and optical detection.

Mitochondria are vital organelles responsible for energy production and they undergo dynamic morphological changes influenced by the cell’s metabolic state. Mitochondrial membrane potential is an essential component in the process of energy storage in the cells. The membrane potential plays a crucial role in the functioning of mitochondria, and it can influence mitochondrial velocity through various mechanisms. Here we report correlation studies on the mitochondrial dynamics and mitochondrial membrane potential in mammalian cells using our home-built sMx-SPIM imaging system. The mitochondria are tracked manually by a self-developed algorithm to find the speed. The speed of mitochondria was determined by targeting mitochondria stained with MitoTracker Deep Red CMXRos dye and measurement of mitochondrial membrane potential was done by tetramethylrhodamine methyl ester (TMRM). Experimental studies were conducted on mammalian cells in alive conditions, and results are provided.