The high computational demand of deep neural networks for computer vision must be alleviated by hardware-based strategies to facilitate applications in resource constrained systems. A potential solutions is to offload computations onto a front-end analog optical preprocessor, which could perform low-level feature encoding operations instantaneously as images are captured. To that end, this study showcases incoherent, broadband, low-noise optical edge encoding for thermal imaging of real-world scenes, which is achieved using a hybrid system of a 24-mm, inversely-designed metasurface and a refractive lens. Using an inverse design approach, the metasurface is optimized for Laplacian-based edge detection across the 7.5 – 13.5 µm LWIR imaging band. This work could be expanded to enable optically-encoded feature maps for accelerating convolutional neural networks for image segmentation and classification.
KEYWORDS: Image segmentation, Transformers, Kidney, Performance modeling, Education and training, Visual process modeling, Deep learning, Data modeling, Tissues, Pathology
The segmentation of kidney layer structures, including cortex, outer stripe, inner stripe, and inner medulla within human kidney whole slide images (WSI) plays an essential role in automated image analysis in renal pathology. However, the current manual segmentation process proves labor-intensive and infeasible for handling the extensive digital pathology images encountered at a large scale. In response, the realm of digital renal pathology has seen the emergence of deep learning-based methodologies. However, very few, if any, deep learning based approaches have been applied to kidney layer structure segmentation. Addressing this gap, this paper assesses the feasibility of performing deep learning based approaches on kidney layer structure segmetnation. This study employs the representative convolutional neural network (CNN) and Transformer segmentation approaches, including Swin-Unet, Medical-Transformer, TransUNet, U-Net, PSPNet, and DeepLabv3+. We quantitatively evaluated six prevalent deep learning models on renal cortex layer segmentation using mice kidney WSIs. The empirical results stemming from our approach exhibit compelling advancements, as evidenced by a decent Mean Intersection over Union (mIoU) index. The results demonstrate that Transformer models generally outperform CNN-based models. By enabling a quantitative evaluation of renal cortical structures, deep learning approaches are promising to empower these medical professionals to make more informed kidney layer segmentation.
Podocytes, specialized epithelial cells that envelop the glomerular capillaries, play a pivotal role in maintaining renal health. The current description and quantification of features on pathology slides are limited, prompting the need for innovative solutions to comprehensively assess diverse phenotypic attributes within Whole Slide Images (WSIs). In particular, understanding the morphological characteristics of podocytes, terminally differentiated glomerular epithelial cells, is crucial for studying glomerular injury. This paper introduces the Spatial Pathomics Toolkit (SPT) and applies it to podocyte pathomics. The SPT consists of three main components: (1) instance object segmentation, enabling precise identification of podocyte nuclei; (2) pathomics feature generation, extracting a comprehensive array of quantitative features from the identified nuclei; and (3) robust statistical analyses, facilitating a comprehensive exploration of spatial relationships between morphological and spatial transcriptomics features. The SPT successfully extracted and analyzed morphological and textural features from podocyte nuclei, revealing a multitude of podocyte morphomic features through statistical analysis. Additionally, we demonstrated the SPT’s ability to unravel spatial information inherent to podocyte distribution, shedding light on spatial patterns associated with glomerular injury. By disseminating the SPT, our goal is to provide the research community with a powerful and user-friendly resource that advances cellular spatial pathomics in renal pathology. The toolkit’s implementation and its complete source code are made openly accessible at the GitHub repository: https://github.com/hrlblab/spatial_pathomics.
We demonstrate meta-optic based accelerators that can off-load computationally expensive operations into high-speed and low-power optics. The key to these architectures are the new freedoms afforded by metasurfaces such as optical edge isolation, polarization discrimination, and the ability to spatially multiplex, and demultiplex, information channels. I will discuss how these freedoms can be utilized for accelerating optical segmentation networks and objection classifiers, both based on incoherent illumination. This approach could enable compact, high-speed, and low-power image and information processing systems for a wide range of applications in machine-vision and artificial intelligence.
Eosinophilic esophagitis (EoE) is an immune-mediated, clinicopathologic disease of the esophagus. EoE is histologically characterized by the accretion of eosinophils in the esophageal epithelium. The current practice involving manual identification of the small-scale histologic features of EoE relative to the size of the esophageal biopsies can be burdensome and prone to interpreter errors. The existing automatic, computer-assisted EoE identification approaches are typically designed as a train-from-scratch setting, which is prone to overfitting. In this study, we propose to use transfer deep-learning via both the ImageNet pre-trained ResNet50 as well as the more recent Big Transfer (BiT) model to achieve automated EoE feature identification on whole slide images. As opposed to existing deep-learning-based approaches that typically focus on a single pathological phenotype, our study investigates five EoE-relevant histologic features including basal zone hyperplasia, dilated intercellular spaces, eosinophils, lamina propria fibrosis, and normal lamina propria simultaneously. From the results, the model achieved a promising testing balanced accuracy of 61.9%, which is better than that of its trained-from-scratch counterparts.
KEYWORDS: Data modeling, Performance modeling, Parallel computing, Image analysis, Instrument modeling, Process modeling, Pathology, Neural networks, Data processing, Skin cancer
Contrastive learning, a recent family of self-supervised learning, leverages pathological image analysis by learning from large-scale unannotated data. However, the state-of-the-art contrastive learning methods (e.g., SimCLR, BYOL) are typically limited by the more expensive computational hardware (with large GPU memory) as compared with traditional supervised learning approaches in achieving large training batch size. Fortunately, recent advances in the machine learning community provide multiple approaches to reduce GPU memory usage, such as (1) activation compressed training, (2) In-place activation, and (3) mixed precision training. Yet, such approaches are currently deployed independently without systematical assessments for contrastive learning. In this work, we applied these memory-efficient approaches into a self-supervised framework. The contribution of this paper is three-fold: (1) We combined previously independent GPU memory-efficient methods with self-supervised learning framework; (2) Our experiments are to maximize the memory efficiency via limited computational resources (a single GPU); (3) The self-supervised learning framework with GPU memory-efficient method allows a single GPU to triple the batch size that typically requires three GPUs. From the experimental results, contrastive learning model with larger batch size leads to higher accuracy enabled by GPU memory-efficient method on single GPU.
The unsupervised segmentation is an increasingly popular topic in biomedical image analysis. The basic idea is to approach the supervised segmentation task as an unsupervised synthesis problem, where the intensity images can be transferred to the annotation domain using cycle-consistent adversarial learning. The previous studies have shown that the macro-level (global distribution level) matching on the number of the objects (e.g., cells, tissues, protrusions etc.) between two domains resulted in better segmentation performance. However, no prior studies have exploited whether the unsupervised segmentation performance would be further improved when matching the exact number of objects at micro-level (mini-batch level). In this paper, we propose a deep learning based unsupervised segmentation method for segmenting highly overlapped and dynamic sub-cellular microvilli. With this challenging task, both micro-level and macro-level matching strategies were evaluated. To match the number of objects at the micro-level, the novel uorescence-based micro-level matching approach was presented. From the experimental results, the micro-level matching did not improve the segmentation performance, compared with the simpler macro-level matching.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.