The Reflectance Confocal Microscopy – Optical Coherence Tomography (RCM-OCT) device has demonstrated its effectiveness in the in vivo detection and depth assessment of basal cell carcinoma (BCC), though its interpretation can be challenging for novices. Artificial intelligence (AI) has the potential to assist in identifying BCC and measuring its depth in these images. Our goal was to develop an AI model capable of generating 3D volumetric representations of BCC to enhance its detection and depth measurement. We developed AI models trained on OCT images of biopsy-confirmed BCC to detect BCC, generate 3D volumetric representations, and automatically assess tumor depth. These models were then tested on a separate dataset containing images of BCC, benign lesions, and normal skin. The effectiveness of the AI models was evaluated through a blinded reader study and by comparing tumor depth measurements with those obtained from histopathology. The addition of AI-generated 3D renders of BCC improved BCC detection rates, with sensitivity increasing from 73.3% to 86.7% and specificity from 45.5% to 48.5%. A Pearson Correlation coefficient r2 = 0.59 (p=0.02) was achieved in comparing tumor depth measurements between AI -generated renders and histopathology slides. Incorporating AI-generated 3D renders has the potential to improve the diagnosis of BCC and the automated measurement of tumor depth in OCT images, reducing reader dependent variability and standardizing diagnostic accuracy.
Ex vivo confocal microscopy (EVCM) images tissues rapidly at near histological resolution without the need for histological processing. Other health institutes can benefit by sending their tissues to confocal experts in formalin, a readily available media that preserves tissue integrity. Fresh tissues imaged with an EVCM device at various timepoints after being put in formalin exhibited easier tissue flattening, improved visualization of the epidermis, reduced tissue movement (due to fat fixation), improved image contrast, and lack of photobleaching (due to dye fixation). Normal skin structures and tumors were readily identified at all TPs by an expert in real-time.
Cutaneous metastases are relatively rare and often require an invasive biopsy for diagnosis. A novel, non-invasive RCM-OCT device combines the advantage of reflectance confocal microscopy (RCM) and optical coherence tomography (OCT) by providing RCM high-resolution images in the horizontal plane and OCT low-resolution images in the transverse plane. We describe RCM and OCT characteristics of cutaneous metastases using this device to elucidate its utility for diagnosis and management. Seven patients with clinically suspicious cutaneous metastases from breast cancer were imaged using the RCM-OCT device. We found that the RCM-OCT device can detect cutaneous metastases and aid in non-invasive diagnosis.
In this study, we trained a convolutional neural network (CNN) utilizing a mix of recent CNN architectural design strategies. Our goals are to leverage these modern techniques to improve the binary classification of kidney tumor images obtained using Multi-Photon Microscopy (MPM). We demonstrate that incorporating these newer model design elements, coupled with transfer learning, image standardization, and data augmentation, leads to significantly increased classification performance over previous results. Our best model averages over 90% sensitivity, specificity, accuracy, area under the receiver operating characteristic curve (AUROC) in image-level classification across cross-validation folds, superior to the previous best in all four metrics.
Melanoma is the most aggressive skin cancer with the highest associated mortality, early diagnosis ensures high survival rates. Currently, in vivo morphological imaging such as reflectance confocal microscopy (RCM) is associated with high sensitivity but moderate specificity. Addition of molecular imaging using PARPi-FL (PARP1-targeted fluorophore) can improve distinction between malignant/potentially malignant lesions. Towards multimodal imaging in vivo, we first investigated differential PARP1 expression in the spectrum of melanocytic lesions. Higher PARP area positivity and intensity were found in melanoma as compared to benign nevi. Thus, PARPi-FL in association with RCM can potentially improve melanoma diagnosis non-invasively in patients.
We present the initial findings of two ML algorithms developed to automate reflectance confocal microscopy (RCM) of skin. On a retrospective test set of 141 pigmented lesions collected at MSKCC between 2011 and 2020, our DEJ detection algorithm identified the DEJ with a median precision of 3 “slices”. The algorithm was less precise on melanomas and on facial lesions. On a retrospective test set of 302 RCM mosaics, the segmentation algorithm identified nonspecific patterns with a sensitivity of 0.75 and specificity of 0.79. Prospectively, on 31 benign pigmented lesions, the DEJ detection algorithm was performed with a median precision of 6.18µm.
In this paper, we demonstrate deep learning-based denoising of high-speed (180 fps) confocal images obtained with our low-cost SECM device. The CARE network was trained with 3090 high- and low-SNR image pairs on the Google Colab platform and tested with 45 unseen image pairs. The CARE prediction showed significant increase of SSIM and PSNR, and reduction of the banding noise while maintaining the cellular details. The preliminary results show the potential of using a deep learning-based denoising approach to enable high-speed SECM imaging.
Basal cell carcinoma (BCC) is the most common skin cancer worldwide. In the diagnosis process million benign biopsies are performed annually, increasing morbidity and healthcare costs. Noninvasive in vivo technologies such as multiphoton microscopy (MPM) can reduce biopsies. We explored the potential of MPM to differentiate collagen changes associated with BCC and surrounding normal skin structures using quantitative analysis (Fast Fourier transformation and Integrated optical density using ImageJ software, and its CurveAlign and CT-FIRE fiber analysis plugins) on second harmonic generation images. Our results showed that collagen distribution is more aligned surrounding BCCs when compared to the skin normal structures, showing the feasibility of detecting BCC in a quantitative way. Our initial results are limited to a small number of samples therefore, large-scale studies are needed to validate these collagen analysis methods.
Convolutional neural networks (CNN) are a class of machine learning model that are especially well suited for imagebased tasks. In this study, we design and train a CNN on tissue samples imaged using Multi-Photon Microscopy (MPM) and show that the model can distinguish between chromophobe renal cell carcinoma (chRCC) and oncocytoma. We demonstrate the method to train a model using simple max-pooling vote fusion, and use the model to highlight regions of the input that cause a positive classification. The model can be tuned for higher sensitivity at the cost of specificity with a constant threshold and little impact to accuracy overall. Several numerical experiments were run to measure the model’s accuracy on both image and patient level analysis. Our models were designed with a dropout parameter that biases the model towards higher sensitivity or specificity. Our best performance model, as measured by area under the receiver operating characteristic curve (AUC of ROC, or AUROC) on patient level classification, is measured with a 94% AUROC and 88% accuracy, along with 100% sensitivity and 75% specificity.
A clear distinction between oncocytoma and chromophobe renal cell carcinoma (chRCC) is critically important for clinical management of patients. But it may often be difficult to distinguish the two entities based on hematoxylin and eosin (H and E) stained sections alone. In this study, second harmonic generation (SHG) signals which are very specific to collagen were used to image collagen fibril structure. We conduct a pilot study to develop a new diagnostic method based on the analysis of collagen associated with kidney tumors using convolutional neural networks (CNNs). CNNs comprise a type of machine learning process well-suited for drawing information out of images. This study examines a CNN model’s ability to differentiate between oncocytoma (benign), and chRCC (malignant) kidney tumor images acquired with second harmonic generation (SHG), which is very specific for collagen matrix. To the best of our knowledge, this is the first study that attempts to distinguish the two entities based on their collagen structure. The model developed from this study demonstrated an overall classification accuracy of 68.7% with a specificity of 66.3% and sensitivity of 74.6%. While these results reflect an ability to classify the kidney tumors better than chance, further studies will be carried out to (a) better realize the tumor classification potential of this method with a larger sample size and (b) combining SHG with two-photon excited intrinsic fluorescence signal to achieve better classification.
KEYWORDS: Skin, In vivo imaging, Melanoma, Reflectivity, Confocal microscopy, Diagnostics, Standards development, Imaging systems, 3D image processing, Cancer
Motivation and background: Melanoma, the fastest growing cancer worldwide, kills more than one person every hour in the United States. Determining the depth and distribution of dermal melanin and hemoglobin adds physio-morphologic information to the current diagnostic standard, cellular morphology, to further develop noninvasive methods to discriminate between melanoma and benign skin conditions.
Purpose: To compare the performance of a multimode dermoscopy system (SkinSpect), which is designed to quantify and map in three dimensions, in vivo melanin and hemoglobin in skin, and to validate this with histopathology and three dimensional reflectance confocal microscopy (RCM) imaging.
Methods: Sequentially capture SkinSpect and RCM images of suspect lesions and nearby normal skin and compare this with histopathology reports, RCM imaging allows noninvasive observation of nuclear, cellular and structural detail in 1-5 m-thin optical sections in skin, and detection of pigmented skin lesions with sensitivity of ~ 90-95% and specificity of ~ 70-80%. The multimode imaging dermoscope combines polarization (cross and parallel), autofluorescence and hyperspectral imaging to noninvasively map the distribution of melanin, collagen and hemoglobin oxygenation in pigmented skin lesions.
Results: We compared in vivo features of ten melanocytic lesions extracted by SkinSpect and RCM imaging, and correlated them to histopathologic results. We present results of two melanoma cases (in situ and invasive), and compare with in vivo features from eight benign lesions. Melanin distribution at different depths and hemodynamics, including abnormal vascularity, detected by both SkinSpect and RCM will be discussed.
Conclusion: Diagnostic features such as dermal melanin and hemoglobin concentration provided in SkinSpect skin analysis for melanoma and normal pigmented lesions can be compared and validated using results from RCM and histopathology.
Reflectance confocal microscopy (RCM) is a non-invasive device that images skin lesions in vivo at a cellular resolution
to guide management of patient care. While previous studies have demonstrated high accuracy of RCM in diagnosing
skin cancers, most of these studies were performed by experts as a blinded analysis off-site and does not reflect true
clinical scenario. We assessed the diagnostic potential of a novice RCM reader, in clinical settings, at the bedside. Over a
period of 15 months (August 2015- November 2016), 168 lesions (from 128 cases) were imaged with RCM to determine
BCC and or melanoma in dermoscopically equivocal lesions. To evaluate the learning curve of the novice reader,
diagnostic accuracy was evaluated at the end of 15 months, as well as during the first half (8 months) and latter half
(seven months) of the study. Histopathological diagnosis was available in 95/168 lesions, including 38 melanocytic
lesions (ML: 13 melanomas and 25 nevi) and 57 non-melanocytic lesions (NML: 26 BCCs, 4 SCCs and 27 benign). The
remaining 73/168 lesions (43.45%) were not biopsied (received topical treatment, monitoring). On RCM, 22/26 (84.61%)
BCCs and 11/13 (84.61%) melanomas were correctly diagnosed. BCC was missed in 3/26 (11.53%) lesions and
melanoma in 2/13 (15.38%) lesions; these lesions were diagnosed mostly as superficial BCCs and focal epidermal
changes overlying deeply situated melanoma nodule on histopathology, respectively. False positive diagnosis of BCC
was obtained in 6/52 (11.54%) benign lesions and of melanoma in 2/52 (3.85%) lesions; these were diagnosed mostly as
benign inflamed keratosis and moderately atypical dysplastic nevus on histopathology, respectively. In 6 lesions BCC or
melanoma could not be ruled out. An increase in the sensitivity and specificity was noticed between the two halves of the
study. A high sensitivity and specificity of 83.33% and 76.60%, respectively in diagnosing skin cancers was obtained.
Based on this study, we identified some current limitations and potential pitfalls of RCM. The fact that the diagnostic
accuracy of the novice reader increased with time, indicates a learning curve reading RCM images. Additionally, current
technical limitations of RCM such as inability to differentiate various cell types, sampling error, and, shallow depth of
imaging also lead to false diagnosis. Efforts are ongoing to overcome these challenges by building US based teachingtraining
program and through a multimodal imaging approach for better diagnosis and patient care.
Confocal mosaicking microscopy (CMM) enables rapid imaging of large areas of fresh tissue ex vivo without the processing that is necessary for conventional histology. When performed in fluorescence mode using acridine orange (nuclear specific dye), it enhances nuclei-to-dermis contrast that enables detection of all types of basal cell carcinomas (BCCs), including micronodular and thin strands of infiltrative types. So far, this technique has been mostly validated in research settings for the detection of residual BCC tumor margins with high sensitivity of 89% to 96% and specificity of 99% to 89%. Recently, CMM has advanced to implementation and testing in clinical settings by “early adopter” Mohs surgeons, as an adjunct to frozen section during Mohs surgery. We summarize the development of CMM guided imaging of ex vivo skin tissues from bench to bedside. We also present its current state of application in routine clinical workflow not only for the assessment of residual BCC margins in the Mohs surgical setting but also for some melanocytic lesions and other skin conditions in clinical dermatology settings. Last, we also discuss the potential limitations of this technology as well as future developments. As this technology advances further, it may serve as an adjunct to standard histology and enable rapid surgical pathology of skin cancers at the bedside.
Distinguishing chromophobe renal cell carcinoma (chRCC) from oncocytoma on hematoxylin and eosin images may be difficult and require time-consuming ancillary procedures. Multiphoton microscopy (MPM), an optical imaging modality, was used to rapidly generate sub-cellular histological resolution images from formalin-fixed unstained tissue sections from chRCC and oncocytoma.Tissues were excited using 780nm wavelength and emission signals (including second harmonic generation and autofluorescence) were collected in different channels between 390 nm and 650 nm. Granular structure in the cell cytoplasm was observed in both chRCC and oncocytoma. Quantitative morphometric analysis was conducted to distinguish chRCC and oncocytoma. To perform the analysis, cytoplasm and granules in tumor cells were segmented from the images. Their area and fluorescence intensity were found in different channels. Multiple features were measured to quantify the morphological and fluorescence properties. Linear support vector machine (SVM) was used for classification. Re-substitution validation, cross validation and receiver operating characteristic (ROC) curve were implemented to evaluate the efficacy of the SVM classifier. A wrapper feature algorithm was used to select the optimal features which provided the best predictive performance in separating the two tissue types (classes). Statistical measures such as sensitivity, specificity, accuracy and area under curve (AUC) of ROC were calculated to evaluate the efficacy of the classification. Over 80% accuracy was achieved as the predictive performance. This method, if validated on a larger and more diverse sample set, may serve as an automated rapid diagnostic tool to differentiate between chRCC and oncocytoma. An advantage of such automated methods are that they are free from investigator bias and variability.
Background: Routine urological surgery frequently requires rapid on-site histopathological tissue evaluation either
during biopsy or intra-operative procedure. However, resected tissue needs to undergo processing, which is not only time
consuming but may also create artifacts hindering real-time tissue assessment. Likewise, pathologist often relies on
several ancillary methods, in addition to H&E to arrive at a definitive diagnosis. Although, helpful these techniques are
tedious and time consuming and often show overlapping results. Therefore, there is a need for an imaging tool that can
rapidly assess tissue in real-time at cellular level. Multiphoton microscopy (MPM) is one such technique that can
generate histology-quality images from fresh and fixed tissue solely based on their intrinsic autofluorescence emission,
without the need for tissue processing or staining.
Design: Fresh tissue sections (neoplastic and non-neoplastic) from biopsy and surgical specimens of bladder and kidney
were obtained. Unstained deparaffinized slides from biopsy of medical kidney disease and oncocytic renal neoplasms
were also obtained. MPM images were acquired using with an Olympus FluoView FV1000MPE system. After imaging,
fresh tissues were submitted for routine histopathology.
Results: Based on the architectural and cellular details of the tissue, MPM could characterize normal components of
bladder and kidney. Neoplastic tissue could be differentiated from non-neoplastic tissue and could be further classified as
per histopathological convention. Some of the tumors had unique MPM signatures not otherwise seen on H&E sections.
Various subtypes of glomerular lesions were identified as well as renal oncocytic neoplasms were differentiated on
unstained deparaffinized slides.
Conclusions: We envision MPM to become an integral part of regular diagnostic workflow for rapid assessment of
tissue. MPM can be used to evaluate the adequacy of biopsies and triage tissues for ancillary studies. It can also be used
as an adjunct to frozen section analysis for intra-operative margin assessment. Further, it can play an important role for
pathologist for guiding specimen grossing, selecting tissue for tumor banking and as a rapid ancillary diagnostic tool.
Confocal mosaicing microscopy (CMM) enables rapid imaging of large areas of fresh tissue ex vivo without the processing that is necessary for conventional histology. When performed with fluorescence mode using acridine orange (nuclear specific dye) it enhances nuclei-to-dermis contrast that enables detection of all types of BCCs including thin strands of infiltrative basal cell carcinomas (BCCs). Thus far, this technique has been mostly validated in research setting for the analysis of BCC tumor margins. Recently, CMM has been adopted and implemented in real clinical settings by some surgeons as an alternative tool to frozen section (FS) during Mohs surgery. In this review article we summarize the development of CMM guided imaging of ex vivo tissues from bench to bedside. We also present its current state of application in routine clinical workflow not only for the assessment of BCC margin but also for other skin cancers such as melanoma, SCC, and some infectious diseases where FS is not routinely performed. Lastly, we also discuss the potential limitations of this technology as well as future developments. As this technology advances further, it may serve as an adjunct to standard histology and enable rapid surgical pathology of skin cancers at the bedside.
In clinical practice, histopathological analysis of biopsied tissue is the main method for bladder cancer diagnosis and
prognosis. The diagnosis is performed by a pathologist based on the morphological features in the image of a
hematoxylin and eosin (HE) stained tissue sample. This manuscript proposes algorithms to perform morphometric
analysis on the HE images, quantify the features in the images, and discriminate bladder cancers with different grades,
i.e. high grade and low grade. The nuclei are separated from the background and other types of cells such as red blood
cells (RBCs) and immune cells using manual outlining, color deconvolution and image segmentation. A mask of nuclei
is generated for each image for quantitative morphometric analysis. The features of the nuclei in the mask image
including size, shape, orientation, and their spatial distributions are measured. To quantify local clustering and alignment
of nuclei, we propose a 1-nearest-neighbor (1-NN) algorithm which measures nearest neighbor distance and nearest
neighbor parallelism. The global distributions of the features are measured using statistics of the proposed parameters. A
linear support vector machine (SVM) algorithm is used to classify the high grade and low grade bladder cancers. The
results show using a particular group of nuclei such as large ones, and combining multiple parameters can achieve better
discrimination. This study shows the proposed approach can potentially help expedite pathological diagnosis by triaging
potentially suspicious biopsies.
Multiphoton microscopy can instantly visualize cellular details in unstained tissues. Multiphoton probes with clinical potential have been developed. This study evaluates the suitability of multiphoton gradient index (GRIN) endoscopy as a diagnostic tool for prostatic tissue. A portable and compact multiphoton endoscope based on a 1-mm diameter, 8-cm length GRIN lens system probe was used. Fresh ex vivo samples were obtained from 14 radical prostatectomy patients and benign and malignant areas were imaged and correlated with subsequent H&E sections. Multiphoton GRIN endoscopy images of unfixed and unprocessed prostate tissue at a subcellular resolution are presented. We note several differences and identifying features of benign versus low-grade versus high-grade tumors and are able to identify periprostatic tissues such as adipocytes, periprostatic nerves, and blood vessels. Multiphoton GRIN endoscopy can be used to identify both benign and malignant lesions in ex vivo human prostate tissue and may be a valuable diagnostic tool for real-time visualization of suspicious areas of the prostate.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.