KEYWORDS: Image segmentation, Muscles, Magnetic resonance imaging, Education and training, Deep learning, 3D modeling, Deformation, 3D image processing, Tissues, Anatomy
PurposeSegmentation is essential for tissue quantification and characterization in studies of aging and age-related and metabolic diseases and the development of imaging biomarkers. We propose a multi-method and multi-atlas methodology for automated segmentation of functional muscle groups in three-dimensional (3D) thigh magnetic resonance images. These groups lie anatomically adjacent to each other, rendering their manual delineation a challenging and time-consuming task.ApproachWe introduce a framework for automated segmentation of the four main functional muscle groups of the thigh, gracilis, hamstring, quadriceps femoris, and sartorius, using chemical shift encoded water–fat magnetic resonance imaging (CSE-MRI). We propose fusing anatomical mappings from multiple deformable models with 3D deep learning model–based segmentation. This approach leverages the generalizability of multi-atlas segmentation (MAS) and accuracy of deep networks, hence enabling accurate assessment of volume and fat content of muscle groups.ResultsFor segmentation performance evaluation, we calculated the Dice similarity coefficient (DSC) and Hausdorff distance 95th percentile (HD-95). We evaluated the proposed framework, its variants, and baseline methods on 15 healthy subjects by threefold cross-validation and tested on four patients. Fusion of multiple atlases, deformable registration models, and deep learning segmentation produced the top performance with an average DSC of 0.859 and HD-95 of 8.34 over all muscles.ConclusionsFusion of multiple anatomical mappings from multiple MAS techniques enriches the template set and improves the segmentation accuracy. Additional fusion with deep network decisions applied to the subject space offers complementary information. The proposed approach can produce accurate segmentation of individual muscle groups in 3D thigh MRI scans.
Segmentation is an essential tool for quantification and characterization of tissue properties, with applications ranging from assessment of body composition, disease diagnosis, to development of imaging biomarkers. In this work, we propose a multi-method and multi-atlas methodology for automated segmentation of functional muscle groups in 3D Dixon MR images of the mid-thigh. The functional muscle groups addressed in this paper lie anatomically close to each other, that makes segmentation an arduous task for accuracy. We propose an approach that uses anatomical mappings enabling delineation of adjacent muscle groups that are difficult to separate using conventional intensity-based patterns only. We segment the four functional muscle groups of the thigh in both legs by multi-atlas anatomical mappings and fuse the labels to improve delineation accuracy. We investigate the fusion of segmentation from multiple atlases and multiple deformable registration methods. For performance evaluation we applied cross-validation by excluding the scans that served as templates in our framework and report DSC values on the remaining test scans. We evaluated four individual deformable models, free-form deformation (FFD), symmetric normalization (SYN), symmetric diffeomorphic demons (SDD), and Voxelmorph (VXM), and the joint multi-method fusion. Multi-atlas and multi-method fusion produced the top average DSC of 0.795 over all muscles on the test scans.
KEYWORDS: Muscles, Image segmentation, Computed tomography, Magnetic resonance imaging, Education and training, Anatomy, Bone, Gallium nitride, Data modeling, Adversarial training
PurposeThigh muscle group segmentation is important for assessing muscle anatomy, metabolic disease, and aging. Many efforts have been put into quantifying muscle tissues with magnetic resonance (MR) imaging, including manual annotation of individual muscles. However, leveraging publicly available annotations in MR images to achieve muscle group segmentation on single-slice computed tomography (CT) thigh images is challenging.ApproachWe propose an unsupervised domain adaptation pipeline with self-training to transfer labels from three-dimensional MR to single CT slices. First, we transform the image appearance from MR to CT with CycleGAN and feed the synthesized CT images to a segmenter simultaneously. Single CT slices are divided into hard and easy cohorts based on the entropy of pseudo-labels predicted by the segmenter. After refining easy cohort pseudo-labels based on anatomical assumption, self-training with easy and hard splits is applied to fine-tune the segmenter.ResultsOn 152 withheld single CT thigh images, the proposed pipeline achieved a mean Dice of 0.888 (0.041) across all muscle groups, including gracilis, hamstrings, quadriceps femoris, and sartorius muscle.ConclusionsTo our best knowledge, this is the first pipeline to achieve domain adaptation from MR to CT for thigh images. The proposed pipeline effectively and robustly extracts muscle groups on two-dimensional single-slice CT thigh images. The container is available for public use in GitHub repository available at: https://github.com/MASILab/DA_CT_muscle_seg.
Diagnosis of breast cancer is often achieved through expert radiologist examination of medical images such as mammograms. Computer-aided diagnosis (CADx) methods can be useful tools in the medical field with applications such as aiding radiologists in making diagnosis decisions. However, such CADx systems require a sufficient amount of data to train on, in conjunction with efficient machine learning techniques. Our Spatially Localized Ensembles Sparse Analysis using Deep Features (DF-SLESA) machine learning model uses local information of features extracted from deep neural networks to learn and classify breast imaging patterns based on sparse approximations. We have also developed a new technique of patch sampling for learning sparse approximations and making classification decisions that we denote as PatchSample decomposition. The PatchSample method differs from our previous approach, our BlockBoost method, in that larger dictionaries are constructed that hold not just spatial-specific information, but a larger collective of visual information from all locations in the region of interest (ROI). Of note is that we trained and tested our method on a merged dataset of mammograms obtained from two sources. Experimental results have reached up to 67.80% classification accuracy (ACC) and 73.21% area under the ROC curve (AUC) using PatchSample decomposition on a merged dataset consisting of the MLO view regions of interest of the MIAS and CBIS-DDSM datasets.
KEYWORDS: Muscles, Image segmentation, Magnetic resonance imaging, Deformation, Silver, Image registration, Tissues, 3D image processing, 3D magnetic resonance imaging, Matrices
We introduce a multi-atlas-based image segmentation (MAIS) framework for the four functional muscle groups, gracilis, hamstring, quadriceps femoris, and sartorius, of the left and right thighs, using 3D chemical shift encoding-based MRI scans obtained from the MyoSegmenTUM database. We generated a statistical atlas and its silver truth by statistical approaches and employed block-matching and 3D filtering (BM3D) to deblur the statistical atlas. We segmented the four pairs of the functional muscle groups of the thigh using three templates, including the statistical atlas, and fused the labels using STAPLE. We validated the performance of our method by calculating the Dice similarity coefficient (DSC) between the delineated muscle group and its ground truth. We also compared the performance of four deformable models: free-form deformation (FFD), two versions of symmetric normalization (SYN and SYNO), and symmetric diffeomorphic demons (SDD). Our results show that SDD with STAPLE produced a mean DSC of 0.784 over all muscle groups. These results imply that the proposed technique has great potential for quantification and characterization of individual muscle groups.
KEYWORDS: Image segmentation, Bone, Computed tomography, Tissues, Data modeling, Neural networks, Medical imaging, 3D modeling, Performance modeling, Surgery
Purpose: Muscle, bone, and fat segmentation from thigh images is essential for quantifying body composition. Voxelwise image segmentation enables quantification of tissue properties including area, intensity, and texture. Deep learning approaches have had substantial success in medical image segmentation, but they typically require a significant amount of data. Due to the high cost of manual annotation, training deep learning models with limited human label data is desirable, but it is a challenging problem.
Approach: Inspired by transfer learning, we proposed a two-stage deep learning pipeline to address the thigh and lower leg segmentation issue. We studied three datasets, 3022 thigh slices and 8939 lower leg slices from the BLSA dataset and 121 thigh slices from the GESTALT study. First, we generated pseudo labels for thigh based on approximate handcrafted approaches using CT intensity and anatomical morphology. Then, those pseudo labels were fed into deep neural networks to train models from scratch. Finally, the first stage model was loaded as the initialization and fine-tuned with a more limited set of expert human labels of the thigh.
Results: We evaluated the performance of this framework on 73 thigh CT images and obtained an average Dice similarity coefficient (DSC) of 0.927 across muscle, internal bone, cortical bone, subcutaneous fat, and intermuscular fat. To test the generalizability of the proposed framework, we applied the model on lower leg images and obtained an average DSC of 0.823.
Conclusions: Approximated handcrafted pseudo labels can build a good initialization for deep neural networks, which can help to reduce the need for, and make full use of, human expert labeled data.
Quantification of the strength and quality of the muscle, adipose tissue, and bone is important for the characterization of the effects of normal aging, age-related diseases, metabolic disorders and neuromuscular diseases. The cost, duration and risks of medical imaging trials may render the task of generating a sufficient high number of quality data for the training of systems for clinical trials and evaluations impracticable. In this work, we developed a model of the human mid-thigh with structural representation of its muscles, adipose tissue and bone anatomy, for use in virtual clinical studies. This is a simulation-based approach to optimizing medical imaging systems. We simulated thigh phantoms based on the OpenVCT software framework, originally designed for digital breast imaging studies. We designed and generated phantoms of mid-thigh anatomical structures representing normal anatomy. Exploiting the flexibility of the system, we were able to generate a controlled population for which we varied, separately, different anatomical structures. We simulated regional mid-thigh muscle area degeneration that is frequently observed in diabetes patients and quantified the structural changes relatively to a healthy anatomy. This framework also allows us to simulate variations of anatomical structures – hence serving as a system for advanced data augmentation that may be used for training machine learning-based diagnostic methods, simulating the effects of diseases, and designing clinical studies.
Quantification and segmentation of the mid-thigh region has high clinical importance for assessment of muscle composition and adipose tissue depositions. Changes in body composition may characterize chronic diseases like obesity, metabolic disorders, type-2 diabetes, and osteoarthritis. Effective methods for segmentation of soft and hard tissues in the mid-thigh in help to understand and characterize changes caused by disease or normal aging. The purpose of our research is to develop a fully automated system for segmentation of hard and soft tissues from CT scans of the mid-thigh region. In particular, we aim to segment the muscle, intermuscular adipose tissue, and subcutaneous adipose tissue using a deep network. A major challenge in deep learning is to provide a rich and diverse set of data for training. Another limitation in tissue segmentation applications is class imbalance, because larger structures may dominate the training process and introduce classification bias. We propose an adaptive re-sampling method according to the tissue type to address class imbalance. We evaluated the segmentation accuracy of the network by cross-validation techniques using CT scans obtained from the BLSA study. We obtained an overall DSC score of 91.5% for segmentation of the mid-thigh regional tissues. Performance evaluation results leads to the observation that our method produces very good accuracy rates and is competitive with current methods used for quantification. This method applied deep learning to a meaningful clinical application that is not revisited frequently.
Muscle, bone, and fat segmentation of CT thigh slice is essential for body composition research. Voxel-wise image segmentation enables quantification of tissue properties including area, intensity and texture. Deep learning approaches have had substantial success in medical image segmentation, but they typically require substantial data. Due to high cost of manual annotation, training deep learning models with limited human labelled data is desirable but also a challenging problem. Inspired by transfer learning, we proposed a two-stage deep learning pipeline to address this issue in thigh segmentation. We study 2836 slices from Baltimore Longitudinal Study of Aging (BLSA) and 121 slices from Genetic and Epigenetic Signatures of Translational Aging Laboratory Testing (GESTALT). First, we generated pseudo-labels based on approximate hand-crafted approaches using CT intensity and anatomical morphology. Then, those pseudo labels are fed into deep neural networks to train models from scratch. Finally, the first stage model is loaded as initialization and fine-tuned with a more limited set of expert human labels. We evaluate the performance of this framework on 56 thigh CT scans and obtained average Dice of 0.979,0.969,0.953,0.980 and 0.800 for five tissues: muscle, cortical bone, internal bone, subcutaneous fat and intermuscular fat respectively. We evaluated generalizability by manually reviewing external 3504 BLSA single thighs from 1752 thigh slices. The result is consistent and passed human review with 5 failed thigh images, which demonstrates that the proposed method has strong generalizability.
Breast cancer is the second most common type of cancer of women in the U.S. behind skin cancer. Early detection and characterization of breast masses is critical for effective diagnosis and treatment of breast cancer. Computer-aided breast mass characterization methods would help to improve the accuracy of diagnoses, their reproducibility, and the throughput of breast cancer screening workflows. In this work, we introduce sparse representations of deep learning features for separation of malignant from benign breast masses in mammograms. We expect that the use of deep feature-based dictionaries will produce better benign/malignant class separation than straightforward sparse representation techniques, and fine-tuned convolutional neural networks (CNNs). We performed 10- and 30-fold cross-validation experiments for classification of benign and malignant breast masses on the MIAS and DDSM mammographic datasets. The results show that the proposed deep feature sparse analysis produces better classification rates than conventional sparse representations and fine-tuned CNNs. The top areas under the curve (AUC) for the receiver operating curve are 80.64% for 10-fold and 97.44% for 30-fold cross-validation in MIAS, and 77.29% for 10-fold and 76.02% for 30-fold cross-validation in DDSM. The main advantages of this approach are that it employs dictionaries of deep network features that are sparse in nature and that it alleviates the need for large volumes of training data and lengthy training procedures. The interesting results from this work prompt further exploration of the relationship between sparse optimization problems and deep learning.
Accurate and reproducible tissue identification techniques are essential for understanding structural and functional changes that either occur naturally with aging, or because of chronic disease, or in response to intervention therapies. These image analysis techniques are frequently utilized for characterization of changes in bone architecture to assess fracture risk, and for the assessment of loss of muscle mass and strength defined as sarcopenia. Peripheral quantitative computed tomography (pQCT) is widely employed for tissue identification and analysis. Advantages of pQCT scanners are compactness, portability, and low radiation dose. However, these characteristics imply limitations in spatial resolution and SNR. Therefore, there is still a need for segmentation methods that address image quality limitations and artifacts such as patient motion. In this paper, we introduce multi-atlas segmentation (MAS) techniques to identify soft and hard tissues in pQCT scans of the proximal tibia (~ 66% of tibial length) and to address the above factors that limit delineation accuracy. To calculate the deformation fields, we employed multi-grid free-form deformation (FFD) models with B-splines and a symmetric extension of the log-domain diffeomorphic demons (SDD). We then applied majority voting and Simultaneous Truth And Performance Level Estimation (STAPLE) for label fusion. We compared the results of our MAS methodology for each deformable registration model and each label fusion method, using Dice similarity coefficient scores (DSC). The results show that our technique utilizing SDD with STAPLE produces very good accuracy (DSC mean of 0.868) over all tissues, even for scans with considerable quality degradations caused by motion artifacts.
The analysis and characterization of imaging patterns is a significant research area with several applications to biomedicine, remote sensing, homeland security, social networking, and numerous other domains. In this paper we study and develop mathematical methods and algorithms for disease diagnosis and tissue characterization. The central hypothesis is that we can predict the occurrence of diseases with a certain level of confidence using supervised learning techniques that we apply to medical imaging datasets that include healthy and diseased subjects. We develop methods for calculation of sparse representations to classify imaging patterns and we explore the advantages of this technique over traditional texture-based classification. We introduce integrative sparse classifier systems that utilize structural block decomposition to address difficulties caused by high dimensionality. We propose likelihood functions for classification and decision tuning strategies. We performed osteoporosis classification experiments on the TCB challenge dataset. TCB contains digital radiographs of the calcaneus trabecular bone of 87 healthy and 87 osteoporotic subjects. The scans of healthy and diseased subjects show little or no visual differences, and their density histograms have significant overlap. We applied 30-fold crossvalidation to evaluate the classification performances of our methods, and compared them to a texture based classification system. Our results show that ensemble sparse representations of imaging patterns provide very good separation between groups of healthy and diseased subjects and perform better than conventional sparse and texture-based techniques.
We describe a systematic approach to image, track, and quantify the movements of HIV viruses embedded in human cervical mucus. The underlying motivation for this study is that, in HIV-infected adults, women account for more than half of all new cases and most of these women acquire the infection through heterosexual contact. The endocervix is believed to be a susceptible site for HIV entry. Cervical mucus, which coats the endocervix, should play a protective role against the viruses. Thus, we developed a methodology to apply time-resolved confocal microscopy to examine the motion of HIV viruses that were added to samples of untreated cervical mucus. From the images, we identified the viruses, tracked them over time, and calculated changes of the statistical mean-squared displacement (MSD) of each virus. Approximately half of tracked viruses appear constrained while the others show mobility with MSDs that are proportional to τα+ν2τ2, over time range τ, depicting a combination of anomalous diffusion (0<α<0.4) and flow-like behavior. The MSD data also reveal plateaus attributable to possible stalling of the viruses. Although a more extensive study is warranted, these results support the assumption of mucus being a barrier against the motion of these viruses.
Peripheral Quantitative Computed Tomography (pQCT) is a non-invasive imaging technology that is well-suited for quantification of bone structural and material properties. Because of its increasing use and applicability, the development of automated quantification methods for pQCT images is an appealing field of research. In this paper we introduce a software system for hard and soft tissue quantification in the lower leg using pQCT imaging data. The main stages of our approach are the segmentation and identification of bone, muscle and fat, and the computation of densitometric and geometric variables of each regional tissue type. Our system was validated against reference area and densitometric measurements over a set of test images and produced encouraging results.
Hypodense metastases are not always completely distinguishable from benign cysts in the liver using conventional
Computed Tomography (CT) imaging, since the two lesion types present with overlapping intensity distributions
due to similar composition as well as other factors including beam hardening and patient motion. This problem
is extremely challenging for small lesions with diameter less than 1 cm. To accurately characterize such lesions,
multiple follow-up CT scans or additional Positron Emission Tomography or Magnetic Resonance Imaging exam
are often conducted, and in some cases a biopsy may be required after the initial CT finding. Gemstone
Spectral Imaging (GSI) with fast kVp switching enables projection-based material decomposition, offering the
opportunity to discriminate tissue types based on their energy-sensitive material attenuation and density. GSI
can be used to obtain monochromatic images where beam hardening is reduced or eliminated and the images
come inherently pre-registered due to the fast kVp switching acquisition. We present a supervised learning
method for discriminating between cysts and hypodense liver metastases using these monochromatic images.
Intensity-based statistical features extracted from voxels inside the lesion are used to train optimal linear and
nonlinear classifiers. Our algorithm only requires a region of interest within the lesion in order to compute
relevant features and perform classification, thus eliminating the need for an accurate segmentation of the lesion.
We report classifier performance using M-fold cross-validation on a large lesion database with radiologist-provided
lesion location and labels as the reference standard. Our results demonstrate that (a) classification using a single
projection-based spectral CT image, i.e., a monochromatic image at a specified keV, outperforms classification
using an image-based dual energy CT pair, i.e., low and high kVp images derived from the same fast kVp
acquisition and (b) classification using monochromatic images can achieve very high accuracy in separating
benign liver cysts and metastases, especially for small lesions.
The topic of aerial image registration attracts considerable interest within the imaging research community due to its significance for several applications, including change detection, sensor fusion, and topographic mapping. Our interest is focused on finding the optimal transformation between two aerial images that depict the same visual scene in the presence of pronounced spatial, temporal, and sensor variations. We first introduce a stochastic edge estimation process suitable for geometric shape-based registration, which we also compare to intensity-based registration. Furthermore, we propose an objective function that weights the L2 distances of the edge estimates by the feature points' energy, which we denote by sum of normalized squared differences and compare to standard objective functions, such as mutual information and the sum of absolute centered differences. In the optimization stage, we employ a genetic algorithm scheme in a multiscale image representation scheme to enhance the registration accuracy and reduce the computational load. Our experimental tests, measuring registration accuracy, rate of convergence, and statistical properties of registration errors, suggest that the proposed edge-based representation and objective function in conjunction with genetic algorithm optimization are capable of addressing several forms of imaging variations and producing encouraging registration results.
The detection and classification of objects in complicated backgrounds represents a difficult image analysis problem. Previous methods have employed additional information from dynamic scene processing to extract the object of interest from its environment and have produced efficient results. However, the study of object detection based on the information provided uniquely by still images has not been comprehensively studied. In this work, a different approach is proposed, when dynamic information is not available for detection. The presented scheme consists of two main stages. The first one includes a still image segmentation approach that makes use of multi-scale information and graph-based grouping to partition the image scene into meaningful regions. This is followed by a texture-based classification algorithm, in which correspondence analysis is used for feature selection and optimisation purposes. The outcomes of this methodology provide representative results at each stage of the study, to indicate the efficiency and potential of this approach for classification/detection in the difficult task of object detection in camouflaged environments.
An automated multiscale segmentation approach for color images is presented. The scale-space stack is generated using the Perona-Malik diffusion approach and the watershed algorithm is employed to produce the regions at each scale. A minima-linking process by downward projection is carried out over the successive scales, and a region dissimilarity measure—combining scale, contrast, and homogeneity—is subsequently estimated on the finer scale (localization scale). The dissimilarity measure is estimated as a function of two different features, i.e., the dynamics of contours and the relative entropy of color region distributions, combined by means of a fuzzy-rule-based system. A region-merging process is also applied to the localization scale to produce the final regions. To validate the performance of the proposed multiscale segmentation, qualitative and quantitative results are provided in comparison to its single-scale counterpart. We also deal with the topic of localization scale selection. This stage is critical for the final segmentation results and can be used as a preprocessing step for higher level computer vision applications as well. A preliminary study of localization scale selection techniques is carried out. A scale selection method that originates from the evolution of the probability distribution of a region homogeneity measure across the generated scales is proposed next. The proposed algorithm is finally compared to a previously reported approach to indicate its efficiency.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.