PurposeOur study investigates the potential benefits of incorporating prior anatomical knowledge into a deep learning (DL) method designed for the automated segmentation of lung lobes in chest CT scans.ApproachWe introduce an automated DL-based approach that leverages anatomical information from the lung’s vascular system to guide and enhance the segmentation process. This involves utilizing a lung vessel connectivity (LVC) map, which encodes relevant lung vessel anatomical data. Our study explores the performance of three different neural network architectures within the nnU-Net framework: a standalone U-Net, a multitasking U-Net, and a cascade U-Net.ResultsExperimental findings suggest that the inclusion of LVC information in the DL model can lead to improved segmentation accuracy, particularly, in the challenging boundary regions of expiration chest CT volumes. Furthermore, our study demonstrates the potential for LVC to enhance the model’s generalization capabilities. Finally, the method’s robustness is evaluated through the segmentation of lung lobes in 10 cases of COVID-19, demonstrating its applicability in the presence of pulmonary diseases.ConclusionsIncorporating prior anatomical information, such as LVC, into the DL model shows promise for enhancing segmentation performance, particularly in the boundary regions. However, the extent of this improvement has limitations, prompting further exploration of its practical applicability.
KEYWORDS: Tumors, Breast, Ultrasonography, Image segmentation, 3D modeling, Radiomics, Education and training, Image classification, Deep learning, 3D image processing
The 3D breast ultrasound is a radiation-free and effective imaging technology for breast tumor diagnosis. However, checking the 3D breast ultrasound is time-consuming compared to mammograms. To reduce the workload of radiologists, we proposed a 2.5D deep learning-based breast ultrasound tumor classification system. First, we used the pre-trained STU-Net to finetune and segment the tumor in 3D. Then, we fine-tuned the DenseNet-121 for classification using the 10 slices with the biggest tumoral area and their adjacent slices. The Tumor Detection, Segmentation, and Classification on Automated 3D Breast Ultrasound (TDSC-ABUS) MICCAI Challenge 2023 dataset was used to train and validate the performance of the proposed method. Compared to a 3D convolutional neural network model and radiomics, our proposed method has better performance.
KEYWORDS: Deformation, Digital breast tomosynthesis, Object detection, Transformers, Convolution, Breast, Feature extraction, Education and training, Performance modeling, Information fusion
In this study, we adapted a transformer-based method to localize lesions in digital breast tomosynthesis (DBT) images. Compared with convolutional neural network-based object detection methods, the transformer-based method does not require non-maximum suppression postprocessing. Integrated deformable convolution detection transformers can better capture small-size lesions. We added transfer learning to tackle the issue of the lack of annotated data from DBT. To validate the superiority of the transformer-based detection method, we compared the results with deep-learning object detection methods. The experimental results demonstrated that the proposed method performs better than all comparison methods.
Segmentation of diagnostic radiography images using deep learning is progressively expanding, which sets demands on the accessibility, availability, and accuracy of the software tools used. This study aimed at evaluating the performance of a segmentation model for digital breast tomosynthesis (DBT), with the use of computer-simulated breast anatomy. We have simulated breast anatomy and soft tissue breast lesions, by utilizing a model approach based on the Perlin noise algorithm. The obtained breast phantoms were projected and reconstructed into DBT slices using a publicly available open-source reconstruction method. Each lesion was then segmented using two approaches: 1. the Segment Anything Model (SAM), a publicly available AI-based method for image segmentation and 2. manually by three human observers. The lesion area in each slice was compared to the ground truth area, derived from the binary mask of the lesion model. We found similar performance between SAM and manual segmentation. Both SAM and the observers performed comparably in the central slice (mean absolute relative error compared to the ground truth and standard deviation SAM: 4±3%, observers: 3±3%). Similarly, both SAM and the observers overestimated the lesion area in the peripheral reconstructed slices (mean absolute relative error and standard deviation SAM: 277±190%, observers: 295±182%). We showed that 3D voxel phantoms can be used for evaluating different segmentation methods. In preliminary comparison, tumor segmentation in simulated DBT images using SAM open-source method showed a similar performance as manual tumor segmentation.
In children, brain tumors are the leading cause of cancer-related death. The amount of labeled data in children is much lower than that for adult subjects. This paper proposes a new method to synthesize high-quality pathological pediatric MRI brain images from pathological adult ones. To realistically simulate the appearance of brain tumors, the proposed method considers the mass effect, i.e., the deformation induced by the tumor to the surrounding tissue. First, a probabilistic U-Net was trained to predict a deformation field that encodes the mass effect from the healthy-pathological image pair. Second, the learned deformation field was utilized to warp the healthy mask to simulate the mass effect. The tumor mask is also added to the warped mask. Finally, a label-to-image transformer, i.e., the SPADE GAN, was trained to synthesize a pathological image from the segmentation masks of gray matter, white matter, CSF and the tumor. The synthetic images were evaluated in two quantitative ways: i) three supervised segmentation pipelines were trained on datasets with and without synthetic images. Two pipelines show over 1% improvements in the Dice scores when the datasets were augmented with synthetic data. ii) The Fr´echet inception distance was measured between real and synthetic image distributions. Results show that SPADE outperforms the state-of-the-art Pix2PixHD method in both T1w and T2w modalities. The source code can be accessed on https://github.com/audreyeternal/pediatric-tumor-generation
A central research topic in medical image processing is the development of imaging biomarkers, i.e. image-based numeric measures of the degree (or probability) of disease. Typically, they rely on segmentation of an anatomical or pathological structure in a radiological image, followed by quantitative measurement. With much of traditional image processing methods being supplanted by machine learning techniques, the identification of new imaging biomarkers is also often made with such techniques, in particular deep learning. Successful examples include quantitative assessment of Alzheimer’s disease and Parkinson’s disease based on brain MRI data, as well as image-based brain age estimation.
Intensity inhomogeneity is a great challenge for automated organ segmentation in magnetic resonance (MR) images. Many segmentation methods fail to deliver satisfactory results when the images are corrupted by a bias field. Although inhomogeneity correction methods exist, they often fail to remove the bias field completely in knee MR images. We present a new iterative approach that simultaneously predicts the segmentation mask of knee structures using a 3D U-net and estimates the bias field in 3D MR knee images using partial convolution operations. First, the test images run through a trained 3D U-net to generate a preliminary segmentation result, which is then fed to the partial convolution filter to create a preliminary estimation of the bias field using the segmented bone mask. Finally, the estimated bias field is then used to produce bias field corrected images as the new inputs to the 3D U-net. Through this loop, the segmentation results and bias field correction are iteratively improved. The proposed method was evaluated on 20 proton-density (PD)-weighted knee MRI scans with manually created segmentation ground truth using 10 fold cross-validation. In our preliminary experiments, the proposed methods outperformed conventional inhomogeneity-correction-plus-segmentation setup in terms of both segmentation accuracy and speed.
In MRI neuroimaging, the shimming procedure is used before image acquisition to correct for inhomogeneity of the static magnetic field within the brain. To correctly adjust the field, the brain’s location and edges must first be identified from quickly-acquired low resolution data. This process is currently carried out manually by an operator, which can be time-consuming and not always accurate. In this work, we implement a quick and automatic technique for brain segmentation to be potentially used during the shimming. Our method is based on two main steps. First, a random forest classifier is used to get a preliminary segmentation from an input MRI image. Subsequently, a statistical shape model of the brain, which was previously generated from ground-truth segmentations, is fitted to the output of the classifier to obtain a model-based segmentation mask. In this way, a-priori knowledge on the brain’s shape is included in the segmentation pipeline. The proposed methodology was tested on low resolution images of rat brains and further validated on rabbit brain images of higher resolution. Our results suggest that the present method is promising for the desired purpose in terms of time efficiency, segmentation accuracy and repeatability. Moreover, the use of shape modeling was shown to be particularly useful when handling low-resolution data, which could lead to erroneous classifications when using only machine learning-based methods.
Percutaneous coronary intervention (PCI) uses x-ray images, which may give high radiation dose and high concentrations of contrast media, leading to the risk of radiation-induced injury and nephropathy. These drawbacks can be reduced by using lower doses of x-rays and contrast media, with the disadvantage of noisier PCI images with less contrast. Vessel-edge-preserving convolutional neural networks (CNN) were designed to denoise simulated low x-ray dose PCI images, created by adding artificial noise to high-dose images. Objective functions of the designed CNNs have been optimized to achieve an edge-preserving effect of vessel walls, and the results of the proposed objective functions were evaluated qualitatively and quantitatively. Finally, the proposed CNN-based method was compared with two state-of-the-art denoising methods: K-SVD and block-matching and 3D filtering. The results showed promising performance of the proposed CNN-based method for PCI image enhancement with interesting capabilities of CNNs for real-time denoising and contrast enhancement tasks.
Vascular segmentation plays an important role in the assessment of peripheral arterial disease. The segmentation is very challenging especially for arteries with severe stenosis or complete occlusion. We present a cascading algorithm for vascular centerline tree detection specializing in detecting centerlines in diseased peripheral arteries. It takes a three-dimensional computed tomography angiography (CTA) volume and returns a vascular centerline tree, which can be used for accelerating and facilitating the vascular segmentation. The algorithm consists of four levels, two of which detect healthy arteries of varying sizes and two that specialize in different types of vascular pathology: severe calcification and occlusion. We perform four main steps at each level: appropriate parameters for each level are selected automatically, a set of centrally located voxels is detected, these voxels are connected together based on the connection criteria, and the resulting centerline tree is corrected from spurious branches. The proposed method was tested on 25 CTA scans of the lower limbs, achieving an average overlap rate of 89% and an average detection rate of 82%. The average execution time using four CPU cores was 70 s, and the technique was successful also in detecting very distal artery branches, e.g., in the foot.
Cortical bone plays a big role in the mechanical competence of bone. The analysis of cortical bone requires accurate
segmentation methods. Level set methods are usually in the state-of-the-art for segmenting medical images. However,
traditional implementations of this method are computationally expensive. This drawback was recently tackled through the
so-called coherent propagation extension of the classical algorithm which has decreased computation times dramatically. In
this study, we assess the potential of this technique for segmenting cortical bone in interactive time in 3D images acquired
through High Resolution peripheral Quantitative Computed Tomography (HR-pQCT). The obtained segmentations are
used to estimate cortical thickness and cortical porosity of the investigated images. Cortical thickness and Cortical porosity
is computed using sphere fitting and mathematical morphological operations respectively. Qualitative comparison between
the segmentations of our proposed algorithm and a previously published approach on six images volumes reveals superior
smoothness properties of the level set approach. While the proposed method yields similar results to previous approaches
in regions where the boundary between trabecular and cortical bone is well defined, it yields more stable segmentations in
challenging regions. This results in more stable estimation of parameters of cortical bone. The proposed technique takes
few seconds to compute, which makes it suitable for clinical settings.
In this study we present a non-rigid point set registration for 3D curves (composed by 3D set of points). The method was evaluated in the task of registration of 3D superficial vessels of the brain where it was used to match vessel centerline points. It consists of a combination of the Coherent Point Drift (CPD) and the Thin-Plate Spline (TPS) semilandmarks. The CPD is used to perform the initial matching of centerline 3D points, while the semilandmark method iteratively relaxes/slides the points.
For the evaluation, a Magnetic Resonance Angiography (MRA) dataset was used. Deformations were applied to the extracted vessels centerlines to simulate brain bulging and sinking, using a TPS deformation where a few control points were manipulated to obtain the desired transformation (T1). Once the correspondences are known, the corresponding points are used to define a new TPS deformation(T2). The errors are measured in the deformed space, by transforming the original points using T1 and T2 and measuring the distance between them. To simulate cases where the deformed vessel data is incomplete, parts of the reference vessels were cut and then deformed. Furthermore, anisotropic normally distributed noise was added.
The results show that the error estimates (root mean square error and mean error) are below 1 mm, even in the presence of noise and incomplete data.
We aim at reconstructing superficial vessels of the brain. Ultimately, they will serve to guide the deformation methods to compensate for the brain shift. A pipeline for three-dimensional (3-D) vessel reconstruction using three mono-complementary metal-oxide semiconductor cameras has been developed. Vessel centerlines are manually selected in the images. Using the properties of the Hessian matrix, the centerline points are assigned direction information. For correspondence matching, a combination of methods was used. The process starts with epipolar and spatial coherence constraints (geometrical constraints), followed by relaxation labeling and an iterative filtering where the 3-D points are compared to surfaces obtained using the thin-plate spline with decreasing relaxation parameter. Finally, the points are shifted to their local centroid position. Evaluation in virtual, phantom, and experimental images, including intraoperative data from patient experiments, shows that, with appropriate camera positions, the error estimates (root-mean square error and mean error) are ∼1 mm.
For optimization and evaluation of image quality, one can use visual grading experiments, where observers rate some
aspect of image quality on an ordinal scale. To take into account the ordinal character of the data, ordinal logistic
regression is used in the statistical analysis, an approach known as visual grading regression (VGR). In the VGR model
one may include factors such as imaging parameters and post-processing procedures, in addition to patient and observer
identity. In a single-image study, 9 radiologists graded 24 cardiac CTA images acquired with ECG-modulated tube
current using standard settings (310 mAs), reduced dose (62 mAs) and reduced dose after post-processing. Image quality
was assessed using visual grading with five criteria, each with a five-level ordinal scale from 1 (best) to 5 (worst). The
VGR model included one term estimating the dose effect (log of mAs setting) and one term estimating the effect of postprocessing.
The model predicted that 115 mAs would be required to reach an 80% probability of a score of 1 or 2 for
visually sharp reproduction of the heart without the post-processing filter. With the post-processing filter, the
corresponding figure would be 86 mAs. Thus, applying the post-processing corresponded to a dose reduction of 25%.
For other criteria, the dose-reduction was estimated to 16-26%. Using VGR, it is thus possible to quantify the potential
for dose-reduction of post-processing filters.
This paper extends to gray-scale the method proposed by Hildebrand and Rüegsegger for estimating thickness of
trabecular bone, which is the most used in trabecular bone research, where local thickness at a point is defined
as the diameter of the maximum inscribed ball that includes that point. The proposed extension takes advantage
of the equivalence between this method and the opening function computed for the granulometry generated by
the opening operation of mathematical morphology with ball-shaped structuring elements of different diameter.
The proposed extension (a) uses gray-scale instead of binary mathematical morphology, (b) uses all values of the
pattern spectrum of the granulometry instead of the maximum peak as used for binary images, (c) corrects bias
on local thickness estimations generated by partial volume effects, and (d) uses the gray-scale as a weighting
function for global thickness estimation. The proposed extension becomes equivalent to the original method
when it is applied to binary images. A new non-flat structuring element is also proposed in order to reduce the
discretization errors generated by traditional flat structuring elements. Translation invariance can be attained
by up-sampling the images through interpolation by a factor of two. Results for synthetic and real images show
that the quality of the measurements obtained through the original method strongly depends on the binarization
process, whereas the measurements obtained through the proposed extension do not. Consequently, the proposed
extension is more appropriate for images with limited resolution where binarization is not trivial.
To analyze visual grading experiments, ordinal logistic regression (here called visual grading regression, VGR) may be
used in the statistical analysis. In addition to types of imaging or post-processing, the VGR model may include factors
such as patient and observer identity, which should be treated as random effects. Standard software does not allow
random factors in ordinal logistic regression, but using Generalized Linear Latent And Mixed Models (GLLAMM) this
is possible. In a single-image study, 9 radiologists graded 24 cardiac Computed Tomography Angiography (CTA)
images with reduced dose without and after post-processing with a 2D adaptive filter, using five image quality criteria.
First, standard ordinal logistic regression was carried out, treating filtering, patient and observer identity as fixed effects.
The same analysis was then repeated with GLLAMM, treating filtering as a fixed effect and patient and observer identity
as random effects. With both approaches, a significant effect (p<0.01) of the filtering was found for all five criteria. No
dramatic differences in parameter estimates or significance levels were found between the two approaches. It is
concluded that random effects can be appropriately handled in VGR using GLLAMM, but no major differences in the
results were found in a preliminary evaluation.
KEYWORDS: Image segmentation, Medical imaging, Image processing algorithms and systems, Partial differential equations, Detection and tracking algorithms, Radiology, Image visualization, Visualization, Information technology, 3D metrology
To accelerate level-set based abdominal aorta segmentation on CTA data, we propose a periodic monotonic speed
function, which allows segments of the contour to expand within one period and to shrink in the next period, i.e.,
coherent propagation. This strategy avoids the contour's local wiggling behavior which often occurs during the
propagating when certain points move faster than the neighbors, as the curvature force will move them backwards even
though the whole neighborhood will eventually move forwards. Using coherent propagation, these faster points will,
instead, stay in their places waiting for their neighbors to catch up. A period ends when all the expanding/shrinking
segments can no longer expand/shrink, which means that they have reached the border of the vessel or stopped by the
curvature force. Coherent propagation also allows us to implement a modified narrow band level set algorithm that
prevents the endless computation in points that have reached the vessel border. As these points' expanding/shrinking
trend changes just after several iterations, the computation in the remaining iterations of one period can focus on the
actually growing parts. Finally, a new convergence detection method is used to permanently stop updating the local level
set function when the 0-level set is stationary in a voxel for several periods. The segmentation stops naturally when all
points on the contour are stationary. In our preliminary experiments, significant speedup (about 10 times) was achieved
on 3D data with almost no loss of segmentation accuracy.
Magnetic Resonance Angiography (MRA) and Computed Tomography Angiography (CTA) data are usually presented using Maximum Intensity Projection (MIP) or Volume Rendering Technique (VRT), but these often fail to demonstrate a stenosis if the projection angle is not suitably chosen. In order to make vascular stenoses visible in projection images independent of the choice of viewing angle, a method is proposed to supplement these images with colors representing the local caliber of the vessel. After preprocessing the volume image with a median filter, segmentation is performed by thresholding, and a Euclidean distance transform is applied. The distance to the background from each voxel in the vessel is mapped to a color. These colors can either be rendered directly using MIP or be presented together with opacity information based on the original image using VRT. The method was tested in a synthetic dataset containing a cylindrical vessel with stenoses in varying angles. The results suggest that the visibility of stenoses is enhanced by the color information. In clinical feasibility experiments, the technique was applied to clinical MRA data. The results are encouraging and indicate that the technique can be used with clinical images.
Magnetic resonance angiography (MRA) images are usually presented as maximum intensity projections (MIP), and the choice of viewing direction is then critical for the detection of stenoses. We propose a presentation method that uses skeletonization and distance transformations, which visualizes variations in vessel width independent of viewing direction. In the skeletonization, the object is reduced to a surface skeleton and further to a curve skeleton. The skeletal voxels are labeled with their distance to the original background. For the curve skeleton, the distance values correspond to the minimum radius of the object at that point, i.e., half the minimum diameter of the blood vessel at that level. The following image processing steps are performed: resampling to cubic voxels, segmentation of the blood vessels, skeletonization ,and reverse distance transformation on the curve skeleton. The reconstructed vessels may be visualized with any projection method. Preliminary results are shown. They indicate that locations of possible stenoses may be identified by presenting the vessels as a structure with the minimum radius at each point.
A 3D image processing algorithm for separating vessels in datasets from Magnetic Resonance Angiography (MRA) and Computed Tomography Angiography (CTA) has been developed and tested on clinical MRA data. Relevant and irrelevant vessels are marked interactively by the user. The algorithm them processes the data, ideally yielding a 3D dataset representing only vessels of interest, while removing other structures. The result is projected to 2D images for visualization. In contrast to traditional segmentation methods, little greyscale information is lost in the process, and the amount of interaction required is relatively small. The classification of voxels utilizes a novel greyscale connectivity measure. A comparison based on the greyscale connectivity values with marked regions is made to decide whether a voxel is of interest for visualization or not. In the projection, those voxels are excluded where the connectivity value is smaller for the relevant vascular structure than for the irrelevant ones. In cases of ambiguity, morphological operations applied to unambiguously classified regions may be used as an additional criterium. In the implementation of the connectivity computation, an iterative propagation scheme is used, similar to that used in chamfer algorithms for distance transforms.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.