Accelerating the measurement for discrimination of samples, such as classification of cell phenotype, is crucial when faced with significant time and cost constraints. Spontaneous Raman microscopy offers label-free, rich chemical information but suffers from long acquisition time due to extremely small scattering cross-sections. One possible approach to accelerate the measurement is by measuring necessary parts with a suitable number of illumination points. However, how to design these points during measurement remains a challenge. To address this, we developed an imaging technique based on a reinforcement learning in machine learning (ML). This ML approach adaptively feeds back “optimal” illumination pattern during the measurement to detect the existence of specific characteristics of interest, allowing faster measurements while guaranteeing discrimination accuracy. Here accurate discrimination means that a user can determine an allowance error rate δ a priori to ensure that the diagnosis can be accurately accomplished with probability greater than (1 − δ) × 100%. We present our algorithm and our simulation studies using Raman images in the diagnosis of follicular thyroid carcinoma, and show that this protocol can accelerate in speedy and accurate diagnoses faster than the point scanning Raman microscopy that requires the full detailed scanning over all pixels. Given a descriptor based on Raman signals to quantify the degree of the predefined quantity to be evaluated, e.g., the degree of cancers, anomaly or defects of materials, the on-the-fly Raman image microscopy evaluates the upper and lower confidence bounds in addition to the sample average of that quantity based on finite point illuminations, and then the bandit algorithm feedbacks the desired illumination pattern to accelerate the detection of the anomaly, during the measurement to the microscope. Several updated realizations of the programmable illumination microscope using a spatial light modulator and line illumination will be presented.
Raman hyperspectral microscopy is a valuable tool in biological and biomedical imaging. Because Raman scattering is often weak in comparison to other phenomena, prevalent spectral fluctuations and contaminations have brought advancements in analytical and chemometric methods for Raman spectra. These chemometric advances have been key contributors to the applicability of Raman imaging to biological systems. As studies increase in scale, spectral contamination from extrinsic background, intensity from sources such as the optical components that are extrinsic to the sample of interest, has become an emerging issue. Although existing baseline correction schemes often reduce intrinsic background such as autofluorescence originating from the sample of interest, extrinsic background is not explicitly considered, and these methods often fail to reduce its effects. Here we show that extrinsic background can significantly affect a classification model using Raman images, yielding misleadingly high accuracies in the distinction of benign and malignant samples of follicular thyroid cell lines. To mitigate its effects, we develop extrinsic background correction (EBC) and demonstrate its use in combination with existing methods on Raman hyperspectral images. EBC isolates regions containing the smallest amounts of sample materials that retain extrinsic contributions that are specific to the device or environment. We perform classification both with and without the use of EBC, and we find that EBC retains biological characteristics in the spectra while significantly reducing extrinsic background. We also address its possible generalization for inhomogeneous illumination profile.
We developed a fast Raman spectroscopic discrimination system based on a slit-scanning confocal microscope and machine learning. The speed of discrimination was improved by reducing the number of measurements, without measuring all points in the field of view. During discrimination, the system continues to evaluate the spectra already obtained, which guarantees the accuracy of the discrimination and enables early detection of anomalies by optimizing the measurement positions. We performed discrimination using a mixture of polystyrene (PS) and polymethyl methacrylate (PMMA) microbeads as a sample to mimic cancer tissue and that of fatty liver tissue using mouse liver tissue samples. The results showed that the discrimination was about 2-11 times faster than that by slit scanning confocal microscopy.
KEYWORDS: Raman spectroscopy, Light sources and illumination, Machine learning, Medical research, Random forests, Microscopy, Microscopes, Engineering, Diagnostics, Decision trees
We propose a method that combines high-speed Raman imaging with a machine learning technique, multi-armed bandit, to achieve rapid and accurate identification of samples under observation. First, our method dvides the field of view of the sample into small sections, and it returns either ’positive’ or ’negative’ based on whether the sections with high anomaly indices exceed a certain proportion. Moreover, the points to be measured are determined dynamically and automatically generating a series of optimal illumination patterns.
We developed spontaneous Raman microscopy using Bandit algorithm to realize fast diagnosis of the existence of anomalies or not with guaranteeing accuracy. The algorithm evaluates obtained Raman spectra during measurement to judge if the diagnosis is completed with ensuring an allowance error rate that users decided and also to generate optimal illumination patterns for the next irradiation which are optimized to accelerate the detection of anomaly. We present our simulation and experimental studies to show that our system can accelerate more than a few tens times faster than line-scanning Raman microscopy which requires full scanning over all pixels.
We present our recent study combined multi-armed bandits algorithm in reinforcement learning with spontaneous Raman measurements for the acceleration of the measurements by designing and generating optimal illumination pattern “on the fly” during the measurements while keeping the accuracy of the diagnosis. Here accurate diagnosis means that a user can determine an allowance error rate δ a priori to ensure that the diagnosis can be accurately accomplished with probability greater than (1 −δ)×100%. We present our algorithm and our simulation studies using Raman images in the diagnosis of follicular thyroid carcinoma, and show that this protocol can accelerate in speedy and accurate diagnoses faster than the point scanning Raman microscopy that requires the full detailed scanning over all pixels. The on-the-fly Raman image microscopy is the first Raman microscope design to accelerate measurements by combining one of reinforcement learning techniques, multi-armed bandit algorithm utilized in the Monte Carlo tree search in alpha-GO. Given a descriptor based on Raman signals to quantify the degree of the predefined quantity to be evaluated, e.g., the degree of cancers, anomaly or defects of materials, the on-the-fly Raman image microscopy evaluates the upper and lower confidence bounds in addition to the sample average of that quantity based on finite point illuminations, and then the bandit algorithm feedbacks the desired illumination pattern to accelerate the detection of the anomaly, during the measurement to the microscope. The realization of the programmable illumination microscope using a spatial light modulator will be presented.
We present our recent study combined multi-armed Bandits algorithm in reinforcement learning with spontaneous Raman microscope for the acceleration of the measurements by designing and generating optimal illumination pattern “on the fly” during the measurements while keeping the accuracy of diagnosis. We present our simulation and experimental studies using Raman images in the diagnosis of follicular thyroid carcinoma and non-alcoholic fatty liver disease, and show that this protocol can accelerate more than a few tens times in speedy and accurate diagnoses faster than line-scanning Raman microscope that requires the full detailed scanning over all pixels.
The on-the-fly Raman image microscopy designs to accelerate measurements by combining one of reinforcement machine learning techniques, bandit algorithm utilized in the Monte Carlo tree search in alpha-GO, and a programmable illumination system. Given a descriptor based on Raman signals to quantify the likelihood of the predefined quantity to be evaluated, e.g., the degree of cancers, the on-the-fly Raman image microscopy evaluates the upper and lower confidence bounds in addition to the sample average of that quantity based on finite point/line illuminations, and then the bandit algorithm feedbacks the desired illumination pattern to accelerate the detection of the anomaly, during the measurement to the microscope.
Most conventional bandit algorithms assume that infinite number of measurements or samples provides us with 100% accuracy. However, in Raman measurements we should develop both a Raman descriptor to quantify the degree of anomaly, and a new algorithm to take into account the finite accuracy lower than 100%. This microscope can also be applied to other problems, besides detection of cancer cells, such as anomaly or defects of materials. The algorithm itself is also beneficial and transferrable to the other microscopes such as infrared image microscope.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.