A new generic model-based segmentation scheme is presented, which can be trained from examples akin to the Active Shape Model (ASM) approach in order to acquire knowledge about the shape to be segmented and about the gray-level appearance of the object in the image. Because in the ASM approach the intensity and shape models are typically applied alternately during optimizing as first an optimal target location is selected for each landmark separately based on local gray-level appearance information only to which the shape model is fitted subsequently, the ASM may be misled in case of wrongly selected landmark locations. Instead, the proposed approach optimizes for shape and intensity characteristics simultaneously. Local gray-level appearance information at the landmark points extracted from feature images is used to automatically detect a number of plausible candidate locations for each landmark. The shape information is described by multiple landmark-specific statistical models that capture local dependencies between adjacent landmarks on the shape. The shape and intensity models are combined in a single cost function that is optimized non-iteratively using dynamic programming which allows to find the optimal landmark positions using combined shape and intensity information, without the need for initialization.

An automatic procedure for detecting and segmenting anatomical objects in 3-D images is necessary for achieving a high level of automation in many medical applications. Since today's segmentation techniques typically rely on user input for initialization, they do not allow for a fully automatic workflow. In this work, the generalized Hough transform is used for detecting anatomical objects with well defined shape in 3-D medical images. This well-known technique has frequently been used for object detection in 2-D images and is known to be robust and reliable. However, its computational and memory requirements are generally huge, especially in case of considering 3-D images and various free transformation parameters. Our approach limits the complexity of the generalized Hough transform to a reasonable amount by (1) using object prior knowledge during the preprocessing in order to suppress unlikely regions in the image, (2) restricting the flexibility of the applied transformation to only scaling and translation, and (3) using a simple shape model which does not cover any inter-individual shape variability. Despite these limitations, the approach is demonstrated to allow for a coarse 3-D delineation of the femur, vertebra and heart in a number of experiments. Additionally it is shown that the quality of the object localization is in nearly all cases sufficient to initialize a successful segmentation using shape constrained deformable models.

Models of geometry or appearance of three-dimensional objects may be used for locating and specifying object instances in 3D image data. Such models are necessary for segmentation if the object to be segmented is not separable based on image information only. They provide a-priori knowledge about the expected shape of the target structure. The success of such a segmentation task depends on the incorporated model knowledge. We present an automatic method to generate such a model for a given target structure. This knowledge is created in the form of a 3D Stable Mass-Spring Model (SMSM) and can be computed from a single sample segmentation. The model is built from different image features using a bottom-up strategy, which allows for different levels of model abstraction. We show the adequacy of the generated models in two practical medical applications: the anatomical segmentation of the left ventricle in myocardial perfusion SPECT, and the segmentation of the thyroid cartilage of the larynx in CT datasets. In both cases, the model generation was performed in a few seconds.

Active Shape Models (ASM) are widely employed for recognizing anatomic structures and for delineating them in medical images. In this paper, we present a novel strategy called Oriented Active Shape Models (OASM) in an attempt to overcome the following three major limitations of ASM: (1) poor delineation accuracy, (2) the requirement of a large number of landmarks, (3) the problem of sensitivity to search range to recognize the object boundary. OASM effectively combines the rich statistical shape information embodied in ASM with the boundary orientedness property and the globally optimal delineation capability of the live wire methodology of boundary segmentation. The latter allow live wire to effectively separate an object boundary from other non object boundaries with similar properties that come very close in the image domain. Our approach leads us to a 2-level dynamic programming method, wherein the first level corresponds to boundary recognition and the second level corresponds to boundary delineation. Our experiments in segmenting breast, liver, bones of the foot, and cervical vertebrae of the spine in MR and CT images indicate the following: (1) The accuracy of segmentation via OASM is considerably better than that of ASM. (2) The number of landmarks can be reduced by a factor of 3 in OASM over that in ASM. (3) OASM becomes largely independent of search range. All three benefits of OASM ensue mainly from the severe constraints brought in by the boundary-orientedness property of live wire and the globally optimal solution of dynamic programming.

This paper presents a new image segmentation algorithm using surface-to-image registration. The algorithm employs multi-level transformations and multi-resolution image representations to progressively register atlas surfaces (modeling anatomical structures) to subject images based on weighted external forces in which weights and forces are determined by gradients and local intensity profiles obtained from images. The algorithm is designed to prevent atlas surfaces converging to unintended strong edges or leaking out of structures of interest through weak edges where the image contrast is low. Segmentation of bone structures on MR images of rat knees analyzed in this manner performs comparably to technical experts using a semi-automatic tool.

Magnetic resonance (MR) imaging is frequently used to diagnose abnormalities in the spinal intervertebral discs. Owing to the non-isotropic resolution of typical MR spinal scans, physicians prefer to align the scanner plane with the disc in order to maximize the diagnostic value and to facilitate comparison with prior and follow-up studies. Commonly a planning scan is acquired of the whole spine, followed by a diagnostic scan aligned with selected discs of interest. Manual determination of the optimal disc plane is tedious and prone to operator variation. A fast and accurate method to automatically determine the disc alignment can decrease examination time and increase the reliability of diagnosis. We present a validation study of an automatic spine alignment system for determining the orientation of intervertebral discs in MR studies. In order to measure the effectiveness of the automatic alignment system, we compared its performance with human observers. 12 MR spinal scans of adult spines were tested. Two observers independently indicated the intervertebral plane for each disc, and then repeated the procedure on another day, in order to determine the inter- and intra-observer variability associated with manual alignment. Results were also collected for the observers utilizing the automatic spine alignment system, in order to determine the method's consistency and its accuracy with respect to human observers. We found that the results from the automatic alignment system are comparable with the alignment determined by human observers, with the computer showing greater speed and consistency.

Ankylosing Spondylitis is a disease of the vertebra where abnormal bone structures (syndesmophytes) grow at intervertebral disk spaces. Because this growth is so slow as to be undetectable on plain radiographs taken over years, it is necessary to resort to computerized techniques to complement qualitative human judgment with precise quantitative measures on 3-D CT images. Very fine segmentation of the vertebral body is required to capture the small structures caused by the pathology. We propose a segmentation algorithm based on a cascade of three level set stages and requiring no training or prior knowledge. First, the noise inside the vertebral body that often blocks the proper evolution of level set surfaces is attenuated by a sigmoid function whose parameters are determined automatically. The 1st level set (geodesic active contour) is designed to roughly segment the interior of the vertebra despite often highly inhomogeneous and even discontinuous boundaries. The result is used as an initial contour for the 2nd level set (Laplacian level set) that closely captures the inner boundary of the cortical bone. The last level set (reversed Laplacian level set) segments the outer boundary of the cortical bone and also corrects small flaws of the previous stage. We carried out extensive tests on 30 vertebrae (5 from each of 6 patients). Two medical experts scored the results at intervertebral disk spaces focusing on end plates and syndesmophytes. Only two minor segmentation errors at vertebral end plates were reported and two syndesmophytes were considered slightly under-segmented.

Rheumatoid Arthritis is one of the most common chronic diseases. Joint space width in hand radiographs is evaluated to assess joint damage in order to monitor progression of disease and response to treatment. Manual measurement of joint space width is time-consuming and highly prone to inter- and intra-observer variation. We propose a method for automatic extraction of finger bone boundaries using fast marching methods for quantitative evaluation of joint space width. The proposed algorithm includes two stages: location of hand joints followed by extraction of bone boundaries. By setting the propagation speed of the wave front as a function of image intensity values, the fast marching algorithm extracts the skeleton of the hands, in which each branch corresponds to a finger. The finger joint locations are then determined by using the image gradients along the skeletal branches. In order to extract bone boundaries at joints, the gradient magnitudes are utilized for setting the propagation speed, and the gradient phases are used for discriminating the boundaries of adjacent bones. The bone boundaries are detected by searching for the fastest paths from one side of each joint to the other side. Finally, joint space width is computed based on the extracted upper and lower bone boundaries. The algorithm was evaluated on a test set of 8 two-hand radiographs, including images from healthy patients and from patients suffering from arthritis, gout and psoriasis. Using our method, 97% of 208 joints were accurately located and 89% of 416 bone boundaries were correctly extracted.

Respiratory motion degrades image quality in PET and SPECT imaging. Patient specific information on the motion of structures such as the heart if obtained from CT slices from a dual-modality imaging system can be employed to compensate for motion during emission reconstruction. The CT datasets may not be contrast enhanced. Since each patient may have 100-120 coronal slices covering the heart, an automated but accurate segmentation of the heart is important. We developed and implemented an algorithm to segment the heart in non-contrast CT datasets. The algorithm has two steps. In the first step we place a truncated-ellipse curve on a mid-slice of the heart, optimize its pose, and then track the contour through the other slices of the same dataset. During the second step the contour points are drawn to the local edge points by minimizing an distance measure. The segmentation algorithm was tested on 10 patients and the boundaries were determined to be accurate to within 2 mm of the visually ascertained locations of the borders of the heart. The segmentation was automatic except for initial placement of the first truncated-ellipse and for having to re-initialize the contour for 3 patients for less than 3% (1-3 slices) of the coronal slices of the heart. These end-slices constituted less than 0.3% of the heart volume.

Segmentation of the left myocardium in four-dimensional (space-time) cardiac MRI data sets is a prerequisite of many diagnostic tasks. We propose a fully automatic method based on global minimization of an energy functional by means of the graphcut algorithm. Starting from automatically obtained segmentations of the left and right ventricles and a cardiac region of interest, a spatial model is constructed using simple and plausible assumptions. This model is used to learn the appearance of different tissue types by non parametric robust estimation. Our method does not require previously trained shape or appearance models. Processing takes 30-40s on current hardware. We evaluated our method on 11 clinical cardiac MRI data sets acquired using cine balanced fast field echo. Linear regression of the automatically segmented myocardium volume against manual segmentations (performed by a radiologist) showed an RMS error of about 12ml.

In the diagnosis of coronary artery disease, 3D-multi-slice computed tomography (MSCT) has recently become more and more important. In this work, an anatomical-based method for the segmentation of atherosclerotic coronary arteries in MSCT is presented. This technique is able to bridge severe stenosis, image artifacts or even full vessel occlusions. Different anatomical structures (aorta, blood-pool of the heart chambers, coronary arteries and their orifices) are detected successively to incorporate anatomical knowledge into the algorithm. The coronary arteries are segmented by a simulated wave propagation method to be able to extract anatomically spatial relations from the result. In order to bridge segmentation breaks caused by stenosis or image artifacts, the spatial location, its anatomical relation and vessel curvature-propagation are taken into account to span a dynamic search space for vessel bridging and gap closing. This allows the prevention of vessel misidentifications and improves segmentation results significantly. The robustness of this method is proven on representative medical data sets.

Osteoporosis is a metabolic bone disease leading to de-mineralization and increased risk of fracture. The two major factors that determine the biomechanical competence of bone are the degree of mineralization and the micro-architectural integrity. Today, modern imaging modalities exist that allow to depict structural details of trabecular bone tissue. Recently, non-linear techniques in 2D and 3D based on the scaling vector method (SVM) and the Minkowski functionals (MF) have been introduced, which show excellent performance in predicting bone strength and fracture risk. However, little is known about the performance of the various parameters with respect to monitoring structural changes due to progression of osteoporosis or as a result of medical treatment. We test and compare the two methodologies using realistic two-dimensional simulations of bone structures, which model the effect of osteoblasts and osteoclasts on the local change of relative bone density. Different realizations with slightly varying control parameters are considered. Our results show that even small changes in the trabecular structures, which are induced by variation of a control parameter of the system, become discernible by applying both the MF and the locally adapted scaling vector method. The results obtained with SVM are superior to those obtained with the Minkowski functionals. An additive combination of both measures drastically increases the sensitivity to slight changes in bone structures. These findings may be especially important for monitoring the treatment of patients, where the early recognition of (drug-induced) changes in the trabecular structure is crucial.

Stroke is the third leading cause of death in the western world and the major cause of disability in adults. The type and stenosis of extracranial carotid artery disease is often responsible for ischemic strokes, transient ischemic attacks (TIAs) or amaurosis fugax (AF). The identification and grading of stenosis can be done using gray scale ultrasound scans. The appearance of B-scan pictures containing various granular structures makes the use of texture analysis techniques suitable for computer assisted tissue characterization purposes. The objective of this study is to investigate the usefulness of variogram analysis in the assessment of ultrasound plague morphology. The variogram estimates the variance of random fields, from arbitrary samples in space. We explore stationary random field models based on the variogram, which can be applied in ultrasound plaque imaging leading to a Computer Aided Diagnosis (CAD) system for the early detection of symptomatic atherosclerotic plaques. Non-parametric tests on the variogram coefficients show that the cofficients coming from symptomatic versus asymptomatic plaques come from distinct distributions. Furthermore, we show significant improvement in class separation, when a log point-transformation is applied to the images, prior to variogram estimation. Model fitting using least squares is explored for anisotropic variograms along specific directions. Comparative classification results, show that variogram coefficients can be used for the early detection of symptomatic cases, and also exhibit the largest class distances between symptomatic and asymptomatic plaque images, as compared to over 60 other texture features, used in the literature.

High resolution MR-scanners working with magnetic field strengths of 3 Tesla are clinically available nowadays. They offer the possibility to obtain 3D images with unprecedented spatial resolution and/or signal-to-noise-ratio (SNR) allowing for an accurate visualization of the trabecular bone structure. It has been demonstrated that scaling indices are well suited to quantify these structures, especially to discriminate between plate-like and rod-like structural elements, which is crucial for the diagnosis of osteoporosis. Until now image quality has mainly been assessed by visual impression or by measures based on the SNR. In this work we present a methodology to assess different MR-sequences with respect to the texture measure that is used later in the image analysis. We acquired for a bone specimen HR-MR-sequences with different spatial resolution and signal to noise ratio. For these data sets we selected two volumes of interest (VOI) of same size located in the trabecular bone and in the background of the image. For both VOIs the scaling indices are calculated for different scale parameters. Subsequently the 'texture contrast' between structure and background is calculated by comparing the probability distributions of the scaling indices using a quadratic distance measure. By means of the contrast the optimal set of scale parameters is determined. By comparing the contrast for the different MR sequences the best suited ones are determined. It turns out that sequences with slightly lower spatial resolution but better signal to noise ration yield a better texture contrast than sequences with the best spatial resolution. The presented methodology offers the possibility to optimize simultaneously texture measures and MR-sequences, which will allow for an adapted and thus optimized analysis of image structures, e.g. trabecular bone, in the HR-MR data.

Automatic classification of the density of breast parenchyma is shown using a measure that is correlated to the human observer performance, and compared against the BI-RADS density rating. Increasingly popular in the United States, the Breast Imaging Reporting and Data System (BI-RADS) is used to draw attention to the increased screening difficulty associated with greater breast density; however, the BI-RADS rating scheme is subjective and is not intended as an objective measure of breast density. So, while popular, BI-RADS does not define density classes using a standardized measure, which leads to increased variability among observers. The adaptive thresholding technique is a more quantitative approach for assessing the percentage breast density, but considerable reader interaction is required. We calculate an objective density rating that is derived using a measure of local feature salience. Previously, this measure was shown to correlate well with radiologists' localization and discrimination of true positive and true negative regions-of-interest. Using conspicuous spatial frequency features, an objective density rating is obtained and correlated with adaptive thresholding, and the subjectively ascertained BI-RADS density ratings. Using 100 cases, obtained from the University of South Florida's DDSM database, we show that an automated breast density measure can be derived that is correlated with the interactive thresholding method for continuous percentage breast density, but not with the BI-RADS density rating categories for the selected cases. Comparison between interactive thresholding and the new salience percentage density resulted in a Pearson correlation of 76.7%. Using a four-category scale equivalent to the BI-RADS density categories, a Spearman correlation coefficient of 79.8% was found.

Numerous studies have investigated the relation between mammographic density and breast cancer risk. These studies indicate that women with high breast density have a four to six fold risk increase. An investigation of whether or not this relation is causal is important for, e.g., hormone replacement therapy (HRT), which has been shown to actually increase the density. No gold standard for automatic assessment of mammographic density exists. Manual methods such as Wolfe patterns and BI-RADS are helpful for communication of diagnostic sensitivity, but they are both time consuming and crude. They may be sufficient in certain cases and for single measurements, but for serial, temporal analysis it is necessary to be able to detect more subtle changes and, in addition, to be more reproducible. In this work an automated method for measuring the effect of HRT w.r.t. changes in biological density in the breast is presented. This measure is a novel measure, which provides structural information orthogonal to intensity-based methods. Hessian eigenvalues at different scales are used as features and a clustering of these is employed to divide a mammogram into four structurally different areas. Subsequently, based on the relative size of the areas, a density score is determined. In the experiments, two sets of mammograms of 50 patients from a double blind, placebo controlled HRT experiment were used. The change in density for the HRT group, measured with the new method, was significantly higher (p = 0.0002) than the change in the control group.

Early detection of structural damage to the optic nerve head (ONH) is critical in diagnosis of glaucoma, because such glaucomatous damage precedes clinically identifiable visual loss. Early detection of glaucoma can prevent progression of the disease and consequent loss of vision. Traditional early detection techniques involve observing changes in the ONH through an ophthalmoscope. Stereo fundus photography is also routinely used to detect subtle changes in the ONH. However, clinical evaluation of stereo fundus photographs suffers from inter- and intra-subject variability. Even the Heidelberg Retina Tomograph (HRT) has not been found to be sufficiently sensitive for early detection. A semi-automated algorithm for quantitative representation of the optic disc and cup contours by computing accumulated disparities in the disc and cup regions from stereo fundus image pairs has already been developed using advanced digital image analysis methodologies. A 3-D visualization of the disc and cup is achieved assuming camera geometry. High correlation among computer-generated and manually segmented cup to disc ratios in a longitudinal study involving 159 stereo fundus image pairs has already been demonstrated. However, clinical usefulness of the proposed technique can only be tested by a fully automated algorithm. In this paper, we present a fully automated algorithm for segmentation of optic cup and disc contours from corresponding stereo disparity information. Because this technique does not involve human intervention, it eliminates subjective variability encountered in currently used clinical methods and provides ophthalmologists with a cost-effective and quantitative method for detection of ONH structural damage for early detection of glaucoma.

We present a new algorithm for automatic detection of bright tubular structures and its performance for automatic segmentation of vessels in breast MR sequences. This problem is interesting because vessels are the main type of false positive structures when automatically detecting lesions as regions that enhance after injection of the contrast agent. Our algorithm is based on the eigenvalues of what we call the shape tensor. It is new in that it does not rely on image derivatives of either first order, like methods based on the eigenvalues of the mean structure tensor, or second order, like methods based on the eigenvalues of the Hessian. It is therefore more precise and less sensitive to noise than those methods. In addition, the smoothing of the output which is inherent to approaches based on the Hessian or structure tensor is avoided. The output of our filter does not present the typical over-smoothed look of the output of the two differential filters that affects both their precision and sensitivity. The scale selection problem appears also less difficult in our approach compared to the differential techniques. Our algorithm is fast, needing only a few seconds per sequence. We present results of testing our method on a large number of motion-corrected breast MR sequences. These results show that our algorithm reliably segments vessels while leaving lesions intact. We also compare our method to the differential techniques and show that it significantly out-performs them both in sensitivity and localization precision and that it is less sensitive to scale selection parameters.

Structural, functional, and clinical studies in schizophrenia have, for several decades, consistently implicated dysfunction of the prefrontal cortex in the etiology of the disease. Functional and structural imaging studies, combined with clinical, psychometric, and genetic analyses in schizophrenia have confirmed the key roles played by the prefrontal cortex and closely linked "prefrontal system" structures such as the striatum, amygdala, mediodorsal thalamus, substantia nigra-ventral tegmental area, and anterior cingulate cortices. The nodal structure of the prefrontal system circuit is the dorsal lateral prefrontal cortex (DLPFC), or Brodmann area 46, which also appears to be the most commonly studied and cited brain area with respect to schizophrenia.^{1, 2, 3, 4} In 1986, Weinberger et. al. tied cerebral blood flow in the DLPFC to schizophrenia.¹ In 2001, Perlstein et. al. demonstrated that DLPFC activation is essential for working memory tasks commonly deficient in schizophrenia. 2 More recently, groups have linked morphological changes due to gene deletion and increased DLPFC glutamate concentration to schizophrenia.^{3, 4} Despite the experimental and clinical focus on the DLPFC in structural and functional imaging, the variability of the location of this area, differences in opinion on exactly what constitutes DLPFC, and inherent difficulties in segmenting this highly convoluted cortical region have contributed to a lack of widely used standards for manual or semi-automated segmentation programs. Given these implications, we developed a semi-automatic tool to segment the DLPFC from brain MRI scans in a reproducible way to conduct further morphological and statistical studies. The segmenter is based on expert neuroanatomist rules (Fallon-Kindermann rules), inspired by cytoarchitectonic data and reconstructions presented by Rajkowska and Goldman-Rakic.⁵ It is semi-automated to provide essential user interactivity. We present our results and provide details on our DLPFC open-source tool.

The hippocampus (Hc) and the amygdala (Am) are two cerebral structures that play a central role in main cognitive processes. Their segmentation allows atrophy in specific neurological illnesses to be quantified, but is made difficult by the complexity of the structures. In this work, a new algorithm for the simultaneous segmentation of Hc and Am based on competitive homotopic region deformations is presented. The deformations are constrained by relational priors derived from anatomical knowledge, namely probabilities for each structure around automatically retrieved landmarks at the border of the objects. The approach is designed to perform well on data from diseased subjects. The segmentation is initialized by extracting a bounding box and positioning two seeds; total execution time for both sides is between 10 and 15 minutes including initialization for the two structures. We present the results of validation based on comparison with manual segmentation, using volume error, spatial overlap and border distance measures. For 8 young healthy subjects the mean volume error was 7% for Hc and 11% for Am, the overlap: 84% for Hc and 83% for Am, the maximal distance: 4.2mm for Hc and 3.1mm for Am; for 4 Alzheimer's disease patients the mean volume error was 9% for Hc and Am, the overlap: 83% for Hc and 78% for Am, the maximal distance: 6mm for Hc and 4.4mm for Am. We conclude that the performance of the proposed method compares favourably with that of other published approaches in terms of accuracy and has a short execution time.

The definition of regions of interests (ROIs), such as suspect cancer nodules or lymph nodes in 3D CT chest images, is often difficult because of the complexity of the phenomena that give rise to them. Manual slice tracing has been used widely for years for such problems, because it is easy to implement and guaranteed to work. But the manual method is extremely time-consuming, especially for high-solution 3D images which may have hundreds of slices, and it is subject to operator biases. Numerous automated image-segmentation methods have been proposed, but they are generally strongly application dependent, and even the "most robust" methods have difficulty in defining complex anatomical ROIs. To address this problem, the semi-automatic interactive paradigm referred to as "live wire" segmentation has been proposed by researchers. In live-wire segmentation, the human operator interactively defines an ROI's boundary guided by an active automated method which suggests what to define. This process in general is far faster, more reproducible and accurate than manual tracing, while, at the same time, permitting the definition of complex ROIs having ill-defined boundaries. We propose a 2D live-wire method employing an improved cost over previous works. In addition, we define a new 3D live-wire formulation that enables rapid definition of 3D ROIs. The method only requires the human operator to consider a few slices in general. Experimental results indicate that the new 2D and 3D live-wire approaches are efficient, allow for high reproducibility, and are reliable for 2D and 3D object segmentation.

Accurate segmentation of small pulmonary nodules (SPNs) on thoracic CT images is an important technique for volumetric doubling time estimation and feature characterization for the diagnosis of SPNs. Most of the nodule segmentation algorithms that have been previously presented were designed to handle solid pulmonary nodules. However, SPNs with ground-glass opacity (GGO) also affects a diagnosis. Therefore, we have developed an automated volumetric segmentation algorithm of SPNs with GGO on thoracic CT images. This paper presents our segmentation algorithm with multiple fixed-thresholds, template-matching method, a distance-transformation method, and a watershed method. For quantitative evaluation of the performance of our algorithm, we used the first dataset provided by NCI Lung Image Database Consortium (LIDC). In the evaluation, we employed the coincident rate which was calculated with both the computerized segmented region of a SPN and the matching probability map (pmap) images provided by LIDC. As the result of 23 cases, the mean of the total coincident rate was 0.507 +/- 0.219. From these results, we concluded that our algorithm is useful for extracting SPNs with GGO and solid pattern as well as wide variety of SPNs in size.

The pulmonary lobes are the five distinct anatomic divisions of the human lungs. The physical boundaries between the lobes are called the lobar fissures. Detection of lobar fissure positions in pulmonary X-ray CT images is of increasing interest for the early detection of pathologies, and also for the regional functional analysis of the lungs. We have developed a two-step automatic method for the accurate segmentation of the three pulmonary fissures. In the first step, an approximation of the actual fissure locations is made using a 3-D watershed transform on the distance map of the segmented vasculature. Information from the anatomically labeled human airway tree is used to guide the watershed segmentation. These approximate fissure boundaries are then used to define the region of interest (ROI) for a more exact 3-D graph search to locate the fissures. Within the ROI the fissures are enhanced by computing a ridgeness measure, and this is used as the cost function for the graph search. The fissures are detected as the optimal surface within the graph defined by the cost function, which is computed by transforming the problem to the problem of finding a minimum s-t cut on a derived graph. The accuracy of the lobar borders is assessed by comparing the automatic results to manually traced lobe segments. The mean distance error between manually traced and computer detected left oblique, right oblique and right horizontal fissures is 2.3 ± 0.8 mm, 2.3 ± 0.7 mm and 1.0 ± 0.1 mm, respectively.

In this paper, we present a pseudo wavelet-based tagline detection method. The tagged MR image is transformed to the wavelet domain, and the prominent tagline coefficients are retained while others are eliminated. Significant stripes are extracted via segmentation, which are mixtures of tags and anatomical boundary that resembles line. A refinement step follows such that broken lines or isolated points are grouped or eliminated. Without assumption on tag models, our method extracts taglines automatically regardless their width and spacing. In addition, founded on the multi-resolution wavelet analysis, our method reconstructs taglines precisely and shows great robustness to various types of taglines.

Digital image fusion is a useful technique that can be applied to achieve extended depth-of-field microscope imaging. The central idea is to incorporate from all input images the regions that contain most in-focus signals into a single composite image. The amount of signal content of a particular region is estimated by an activity measure in general. To our best knowledge, all existing approaches in the literatures rely on estimates based on the strength of high frequency signal components as the activity measure. However, such measure does not distinguish true image signals from noise. We propose to use a multiscale point-wise product as the activity measure, which does not amplify the effect of noise. The resulting scheme shows a significant improvement on imaging of cytological specimens in terms of both subjective and objective quality even under a noise-free environment. More importantly, the scheme has a significant advantage over existing methods in the presence of noise.

In magnetic resonance imaging (MRI), there are three bands of images ("MRI triplet") available, which are T1-, T2- and PD-weighted images. The three images of a MRI triplet provide complementary structure information and therefore it is useful for diagnosis and subsequent analysis to combine three-band images into one. We propose an advanced discrete wavelet transform (αDWT) for three-band MRI image fusion and the αDWT algorithm is further optimized utilizing two quantitative fusion metrics - the image quality index (IQI) and ratio spatial frequency error (rSFe). In the aDWT method, principle component analysis (PCA) and morphological processing are incorporated into a regular DWT fusion algorithm. Furthermore, the αDWT has two adjustable parameters - the level of DWT decomposition (L_d) and the length of the selected wavelet (L_w) that determinately affect the fusion result. The fused image quality can be quantitatively measured with the established metrics - IQI and rSFe. Varying the control parameters (L_d and L_w), an iterative fusion procedure can be implemented and running until an optimized fusion is achieved. We fused and analyzed several MRI triplets from the Visible Human Project^® female dataset. From the quantitative and qualitative evaluations of fused images, we found that (1) the αDWTi-IQI algorithm produces a smoothed image whereas the αDWTi-rSFe algorithm yields a sharpened image, (2) fused image "T1+T2" is the most informative one in comparison with other two-in-one fusions (PD+T1 and PD+T2), (3) for three-in-one fusions, no significant difference is observed among the three fusions of (PD+T1)+T2, (PD+T2)+T1 and (T1+T2)+PD, thus the order of fusion does not play an important role. The fused images can significantly benefit medical diagnosis and also the further image processing such as multi-modality image fusion (with CT images), visualization (colorization), segmentation, classification and computer-aided diagnosis (CAD).

In this paper, we evaluate different non-rigid image registration methodologies in the context of atlas-based brain image segmentation. Three non-rigid voxel-based registration regularization schemes (viscous fluid, elastic and curvature-based registration) combined with the mutual information similarity measure are compared. We conduct large-scale atlas-based segmentation experiments on a set of 20 anatomically labelled MR brain images in order to find the optimal parameter settings for each scheme. The performance of the optimal registration schemes is evaluated in their capability of accurately segmenting 49 different brain sub-structures of varying size and shape.

The importance of small animal imaging in fundamental and clinical research is growing rapidly. These studies typically involve micro PET, micro MR, and micro CT images as well as optical or fluorescence images. Histological images are also often used to complement and/or validate the in vivo data. As is the case for human studies, automatic registration of these imaging modalities is a critical component of the overall analysis process, but, the small size of the animals and thus the limited spatial resolution of the in vivo images present specific challenges. In this paper, we propose a series of methods and techniques that permit the inter-subject registration of micro MR and histological images. We then compare results obtained by registering directly MR volumes to each other using a non-rigid registration algorithm we have developed at our institution with results obtained by registering first the MR volumes to their corresponding histological volume, which we reconstruct from 2D cross-sections, and then registering histological volumes to each other. We show that the second approach is preferable.

During functional magnetic resonance imaging (fMRI) brain examinations, the signal extraction from a large number of images is used to evaluate changes in blood oxygenation levels by applying statistical methodology. Image registration is essential as it assists in providing accurate fractional positioning accomplished by using interpolation between sequentially acquired fMRI images. Unfortunately, current subvoxel registration methods found in standard software may produce significant bias in the variance estimator when interpolating with fractional, spatial voxel shifts. It was found that interpolation schemes, as currently applied during the registration of functional brain images, could introduce statistical bias, but there is a possible correction scheme. This bias was shown to result from the "weighted-averaging" process employed by conventional implementation of interpolation schemes. The most severe consequence of inaccurate variance estimators is the undesirable violation of the fundamental 'stationary' assumption required for many statistical methods and Gaussian random field analysis. Thus, this bias violates assumptions of the general linear model (GLM) and/or t-tests commonly used in fMRI studies. Using simulated data as well as actual human data in this, it was demonstrated that this artifact can significantly alter the magnitude and location of the resulting activation patterns/results. Further, the work detailed here introduces a bias correction scheme and evaluates the improved accuracy of its sample variance calculation and influence on fMRI results through comparison with traditional fMRI image registered data.

The endovascular repair of an abdominal aortic aneurysm is a minimal invasive therapy which has been established during the past 15 years. A stent-graft is placed inside the aorta in order to cover the weakened regions of its wall. During a time interval of one or more years the stent-graft can migrate and deform with the risk of the occlusion of one of its limbs or of the rupture of the aneurysm. In this work we developed several strategies to quantify the migration and deformation in order to assess the risk coming with these movements and especially to characterize appearing complications by them. We calculated the rigid movement of the stent-graft and the aorta relative to the spinal canal. For this purpose, firstly, we rigidly registered the spinal canals, extracted for the different points in time, in order to establish a fixed reference system. All objects have been segmented first and surface points have been determined before applying a rigid and non-rigid point set registration algorithm. The change in the residual error after registration of the stent-graft with an increasing number of degrees of freedom indicates the amount of change in the stent-graft's morphology. We investigated a sample of 9. Two cases could be clearly distinguished by the quantified parameters: a high global migration and a strong reduction of the residual error after non-rigid registration. In both cases, strong complications have been detected by the examination of clinical experts but only by means of the images acquired one year later.

One method of modelling respiratory motion of the abdomen is to acquire CT images at different points in the respiratory cycle and develop a deformation model that gives a mapping between corresponding anatomical points in the images. In this work, we use such a method, and the target application is radiosurgery, particularly radiosurgical treatment of lesions that move during respiration, for example those in the liver, lung, or pancreas. In order to accurately calculate the treatment dose, it is necessary to have a good deformation map both globally and locally (in the vicinity of the treatment target). We use a dual-resolution method in order to allow a more accurate deformation model to be computed in the region of interest. We also introduce a tissue stiffness constraint, along with an application of matrix algebra that allows this constraint to be applied in an effective way with respect to the control point values.

In this paper, we propose a multi-modal non-rigid 2D-3D registration technique. This method allows a non-rigid alignment of a patient pre-operatively computed tomography (CT) to few intra operatively acquired fluoroscopic X-ray images obtained with a C-arm system. This multi-modal approach is especially focused on the 3D alignment of high contrast reconstructed volumes with intra-interventional low contrast X-ray images in order to make use of up-to-date information for surgical guidance and other interventions. The key issue of non-rigid 2D-3D registration is how to define the distance measure between high contrast 3D data and low contrast 2D projections. In this work, we use algebraic reconstruction theory to handle this problem. We modify the Euler-Lagrange equation by introducing a new 3D force. This external force term is computed from the residual of the algebraic reconstruction procedures. In the multi-modal case we replace the residual between the digitally reconstructed radiographs (DRR) and observed X-ray images with a statistical based distance measure. We integrate the algebraic reconstruction technique into a variational registration framework, so that the 3D displacement field is driven to minimize the reconstruction distance between the volumetric data and its 2D projections using mutual information (MI). The benefits of this 2D-3D registration approach are its scalability in the number of used X-ray reference images and the proposed distance that can handle low contrast fluoroscopies as well. Experimental results are presented on both artificial phantom and 3D C-arm CT images.

Most of the suggested image registration methods are based on the optimization of an objective function. Drawbacks of this approach are the problem of local minima and the need to initialize the transformation close to the true solution. This paper presents a method for N-dimensional rigid and similarity image registration that is not optimization-based and consequently it doesn't involve local minima and initialization. Instead of obtaining the transformation parameters implicitly through an iterative optimization process, they are obtained explicitly. The proposed method has advantages over existing explicit methods. The explicit expressions for transformation parameters involve image integrals and no image derivatives, which makes the method robust to noise. It is shown that the method has a few desired properties including symmetry and transitivity, and that it is invariant to initial alignment of the images. The method has been tested on simulated and real brain 2D and 3D MR image pairs and the achieved average registration error was one voxel.

The representation of shapes by Fourier descriptors is a time-honored technique that has received relatively little attention lately. Nevertheless, it has its benefits and is suitable for describing a range of medical structures in two dimensions. Delineations in medical applications often consist of continuous outlines of structures, where no information of correspondence between samples exist. In this article, we discuss a Euclidean alignment method that works directly with the functional representation of Fourier descriptors, and that is optimal in a least-squares sense. With corresponding starting points, the alignment of one shape onto another consists of a single expression. If the starting points are arbitrary, we present a simple algorithm to bring a set of shapes into correspondence. Results are given for three different data sets; 62 outlines of the corpus callosum brain structure, 61 outlines of the brain ventricles, and 50 outlines of the right lung. The results show that even though starting points, translations, rotations and scales have been randomized, the alignment succeeds in all cases. As an application of the proposed method, we show how high-quality shape models represented by common landmarks can be constructed in an automatic fashion. If the aligned Fourier descriptors are inverse transformed from the frequency domain to the spatial domain, a set of roughly aligned landmarks are obtained. The positions of these are then adjusted along the contour of the objects using the minimum description length criterion, producing ample correspondences. Results on this are also presented for all three data sets.

Image registration is the process of aligning separate images into a common reference frame so that they can be compared visually or statistically. In order for this alignment to be accurate and correct it is important to identify the correct anatomical correspondences between different subjects. We propose a new approach for a feature-based, inter-subject deformable image registration method using a novel displacement field interpolation. Among the top deformable registration algorithms in the literature today is the work of Shen et al. called HAMMER. This is a feature-based, hierarchical registration algorithm, which introduces the novel idea of fusing feature and intensity matching. The algorithm presented in this paper is an implementation of that method, where significant improvements of some important aspects have been made. A new approach to the algorithm will be introduced as well as clarification of some key features of the work of Shen et al. which have not been elaborated in previous publications. The new algorithm, which is referred to as Mjolnir (Thor's hammer), was validated on both synthesized and real T1 weighted MR brain images. The results were compared with results generated by HAMMER and show significant improvements in accuracy with reduction in computation time.

In this article, we propose a new registration method, based on a statistical analysis of deformation fields. At first, a set of MRI brain images was registered using a viscous fluid algorithm. The obtained deformation fields are then used to calculate a Principal Component Analysis (PCA) based decomposition. Since PCA models the deformations as a linear combination of statistically uncorrelated principal components, new deformations can be created by changing the coefficients in the linear combination. We then use the PCA representation of the deformation fields to non-rigidly align new sets of images. We use a gradient descent method to adjust the coefficients of the principal components, such that the resulting deformation maximizes the mutual information between the deformed image and an atlas image. The results of our method are promising. Viscous fluid registrations of new images can be recovered with an accuracy of about half a voxel. Better results can be obtained by using a more extensive database of learning images (we only used 84). Also, the optimization method used here can be improved, especially to shorten computation time.

Regularized nonrigid medical image registration algorithms usually estimate the deformation by minimizing a cost function, consisting of a similarity measure and a penalty term that discourages "unreasonable" deformations. Conventional regularization methods enforce homogeneous smoothness properties of the deformation field; less work has been done to incorporate tissue-type-specific elasticity information. Yet ignoring the elasticity differences between tissue types can result in non-physical results, such as bone warping. Bone structures should move rigidly (locally), unlike the more elastic deformation of soft issues. Existing solutions for this problem either treat different regions of an image independently, which requires precise segmentation and incurs boundary issues; or use an empirical spatial varying "filter" to "correct" the deformation field, which requires the knowledge of a stiffness map and departs from the cost-function formulation. We propose a new approach to incorporate tissue rigidity information into the nonrigid registration problem, by developing a space variant regularization function that encourages the local Jacobian of the deformation to be a nearly orthogonal matrix in rigid image regions, while allowing more elastic deformations elsewhere. For the case of X-ray CT data, we use a simple monotonic increasing function of the CT numbers (in HU) as a "rigidity index" since bones typically have the highest CT numbers. Unlike segmentation-based methods, this approach is flexible enough to account for partial volume effects. Results using a B-spline deformation parameterization illustrate that the proposed approach improves registration accuracy in inhale-exhale CT scans with minimal computational penalty.

Nonrigid registration is a technique commonly used in the field of medical imaging. A drawback of most current nonrigid registration algorithms is that they model all tissue as being nonrigid. When a nonrigid registration is performed, the rigid objects in the image, such as bony structures or surgical instruments, may also transform nonrigidly. Other consequences are that tumour growth between follow-up images may be concealed, or that structures containing contrast material in one image and not in the other may be compressed by the registration algorithm. In this paper we propose a novel regularisation term, which is added to the cost function in order to penalise nonrigid deformations of rigid objects. This regularisation term can be used for any representation of the deformation field capable of modelling locally rigid deformations. By using a B-spline representation of the deformation field, a fast algorithm can be devised. We show on 2D synthetic data, on clinical CT slices, and on clinical DSA images, that the proposed rigidity constraint is successful, thus improving registration results.

Respiratory motion is a significant source of error in radiotherapy treatment planning. 4D-CT data sets can be useful to measure the impact of organ motion caused by breathing. But modern CT scanners can only scan a limited region of the body simultaneously and patients have to be scanned in segments consisting of multiple slices. For studying free breathing motion multislice CT scans can be collected simultaneously with digital spirometry over several breathing cycles. The 4D data set is assembled by sorting the free breathing multislice CT scans according to the couch position and the tidal volume. But artifacts can occur because there are no data segments for exactly the same tidal volume and all couch positions. We present an optical flow based method for the reconstruction of 4D-CT data sets from multislice CT scans, which are collected simultaneously with digital spirometry. The optical flow between the scans is estimated by a non-linear registration method. The calculated velocity field is used to reconstruct a 4D-CT data set by interpolating data at user-defined tidal volumes. By this technique, artifacts can be reduced significantly. The reconstructed 4D-CT data sets are used for studying inner organ motion during the respiratory cycle. The procedures described were applied to reconstruct 4D-CT data sets for four tumour patients who have been scanned during free breathing. The reconstructed 4D data sets were used to quantify organ displacements and to visualize the abdominothoracic organ motion.

Modern MDCT and micro-CT scanners are able to produce high-resolution three-dimensional (3D) images of anatomical trees, such as the airway tree and the heart and liver vasculature. An important problem arising in many contexts is the matching of trees depicted in two different images. Three basic steps are used in order to match two trees: (1) image segmentation, to extract the raw trees from a given pair of 3D images; (2) axial-analysis, to define the underlying centerline structure of the trees; and (3) tree matching, to match the centerline structures of the trees. We focus on step (3). This task is complicated by several problems associated with current segmentation and axial-analysis methods, including missing branches, false branches, and other topological errors in the extracted trees. We propose a model-based approach in which the extracted trees are assumed to arise from an initially unknown common structure corrupted by a sequence of modelled topological deformations. We employ a novel mathematical framework to directly incorporate this model into the matching problem. Under this framework, it is possible to define the set of matches that are consistent with a given deformation model. The optimal match is the member of this set that maximizes a user-definable similarity measure. We present several such similarity measures based upon geometrical attributes (e.g., branch lengths, branching angles, and relative branchpoint locations as measured from the 3D image data). We locate the globally optimal match via an efficient dynamic programming algorithm. Our primary analytical result is a set of sufficient conditions on the user-definable similarity measure such that our dynamic programming algorithm is guaranteed to locate an optimal match. Experimental results have been generated for 3D human CT chest scans and micro-CT coronary arterial-tree images of mice. The resulting matches are in good agreement with correspondences defined by human experts.

Little work has been done on comparing the performance of statistical model-based approaches and nonrigid registration algorithms. This paper deals with the qualitative and quantitative comparison of active appearance models (AAMs) and a nonrigid registration algorithm based on free-form deformations (FFDs). AAMs are known to be much faster than nonrigid registration algorithms. On the other hand nonrigid registration algorithms are independent of a training set as required to build an AAM. To obtain a further comparison of the two methods, they are both applied to automatically register multi-slice myocardial perfusion images. The images are acquired by magnetic resonance imaging, from infarct patients. A registration of these sequences is crucial for clinical practice, which currently is subjected to manual labor. In the paper, the pros and cons of the two registration approaches are discussed and qualitative and quantitative comparisons are provided. The quantitative comparison is obtained by an analysis of variance of landmark errors, i.e. point to point and point to curve errors. Even though the FFD-based approach does not include a training phase it gave similar accuracy as the AAMs in terms of point to point errors. For the point to curve errors the AAMs provided higher accuracy. In both cases AAMs gave higher precision due to the training procedure.

Tagged Magnetic Resonance Imaging (MRI) is currently the reference modality for myocardial motion and strain analysis. Mutual Information (MI) based non rigid registration has proven to be an accurate method to retrieve cardiac motion and overcome many drawbacks present on previous approaches. In a previous work¹, we used Wavelet-based Attribute Vectors (WAVs) instead of pixel intensity to measure similarity between frames. Since the curse of dimensionality forbids the use of histograms to estimate MI of high dimensional features, k-Nearest Neighbors Graphs (kNNG) were applied to calculate α-MI. Results showed that cardiac motion estimation was feasible with that approach. In this paper, K-Means clustering method is applied to compute MI from the same set of WAVs. The proposed method was applied to four tagging MRI sequences, and the resulting displacements were compared with respect to manual measurements made by two observers. Results show that more accurate motion estimation is obtained with respect to the use of pixel intensity.

Three-dimensional (3D) stress echocardiography is a novel technique for diagnosing cardiac dysfunction, by comparing wall motion of the left ventricle under different stages of stress. For quantitative comparison of this motion, it is essential to register the ultrasound data. We propose an intensity based rigid registration method to retrieve two-dimensional (2D) four-chamber (4C), two-chamber, and short-axis planes from the 3D data set acquired in the stress stage, using manually selected 2D planes in the rest stage as reference. The algorithm uses the Nelder-Mead simplex optimization to find the optimal transformation of one uniform scaling, three rotation, and three translation parameters. We compared registration using the SAD, SSD, and NCC metrics, performed on four resolution levels of a Gaussian pyramid. The registration's effectiveness was assessed by comparing the 3D positions of the registered apex and mitral valve midpoints and 4C direction with the manually selected results. The registration was tested on data from 20 patients. Best results were found using the NCC metric on data downsampled with factor two: mean registration errors were 8.1mm, 5.4mm, and 8.0° in the apex position, mitral valve position, and 4C direction respectively. The errors were close to the interobserver (7.1mm, 3.8mm, 7.4°) and intraobserver variability (5.2mm, 3.3mm, 7.0°), and better than the error before registration (9.4mm, 9.0mm, 9.9°). We demonstrated that the registration algorithm visually and quantitatively improves the alignment of rest and stress data sets, performing similar to manual alignment. This will improve automated analysis in 3D stress echocardiography.

Cardiac interventional procedures such as myocardial stem cell delivery and radiofrequency ablation require a high degree of accuracy and efficiency. Real-time, 2-D MR technology is being developed to guide such procedures; the associated challenges include the relatively low resolution and image quality in real-time images. Real-time MR guidance can be enhanced by acquiring a 4-D (3-D + phase) volume prior to the procedure and aligning it to the 2-D real-time images, so that corresponding features in the prior volume can be integrated into the real-time image visualization. This technique provides spatial context with high resolution and SNR. A left ventricular (LV) myocardial wall contour tracking system was developed to maintain spatial alignment of prior volume images to real-time MR images. Over 9 test images sequences, each comprising 100 frames of simulated respiratory motion, the tracker maintained alignment with a mean displacement error of 1.61mm in a region of interest around the LV, as compared to a mean displacement error of 5.2mm without tracking.

In this paper we develop a new filtering framework for tensor signal processing using the theory of vector spaces. From this point of view, signals are regarded as elements of vector spaces and operators as mappings from the input space to the output space. Hence, it is possible to generalize the principle of superposition to any operator defined on the signal spaces. Systems that obey that generalization of the principle of superposition are referred to as homomorphic and they can be decomposed in a cascade of three homomorphic subsystems: the first one operates on the input signal space, the second one is a linear system in the usual sense and the third one operates on the output signal space. Thus, suitable input and output subsystems can be chosen to deal with input signals, which defines a whole family of homomorphic filters. To apply this idea for DT-MRI signals, which consist of positive semi-definite matrices, we identify input and output signal spaces as the set of those real symmetric positive semi-definite matrices. Our homomorphic filtering framework not only guarantees a positive-semidefinite output tensor field whatever linear filter is used to regularize the noisy input, but also reduces the swelling effect produced by a faster regularization of diffusivities rather than orientations, as demonstrate the encouraging results that have been obtained.

Reconstruction methods for MR imaging of dynamic objects have traditionally been analyzed using the projection slice theorem. In this paper, we present a new theoretical framework for analyzing MR imaging of dynamic objects. Our framework reinterprets the object stationarity assumption in the MR reconstruction techniques as a combination of filtering and downsampling operations performed on the acquired k-space data. We have analyzed our results in x-f (spatial coordinate - temporal frequency) space using a time-sequential analysis. While the projection slice theorem has only be used to analyze the Cartesian sampling pattern, the new framework can analyze any arbitrary sampling pattern with a given reconstruction algorithm. Further, the new theoretical framework can be used to analyze the effect of relaxing the object stationarity assumption over the reconstructed MR images. We have demonstrated the use of our framework by analyzing two popular image reconstruction techniques, namely view-sharing and UNFOLD. In the analysis of view-sharing, we have confirmed the fact that interleaved and bit reversed k-space sampling patterns provide better artifact suppression for dynamic MR imaging. We propose using a different filter to further reduce artifacts in the reconstructed images. In the case of UNFOLD, we have analyzed the effect of relaxing the object stationarity assumption and have shown that it leads to an increase in motion artifacts.

Magnetic resonance images are often corrupted by intensity inhomogeneity (i.e., bias field effects), which manifests itself as slow intensity variations over the image domain. Such shading artifacts must be corrected before performing computerized analyses such as intensity-based segmentation and quantitative analysis. In this paper, we present a novel strategy in the fuzzy c-means (FCM) framework that simultaneously estimates the bias field while segmenting the image. An additive field term that models the bias field is incorporated into the FCM objective function. We propose a new term based on the spectral parameterization (i.e., wavelet coefficients) of the bias field that serves as a regularizer to enforce the smoothness of the estimated bias field. We also introduce a second regularization term that causes the labeling of each pixel to be influenced by its immediate neighborhood pixels. The latter regularization term renders the algorithm less sensitive to noise. We show that the novel objective functional could be optimized efficiently using an iterative process. The efficacy of the algorithm is demonstrated on synthesized images as well as on clinical breast MR images. With the synthesize images, segmentation accuracy using standard FCM is 89.07% while segmentation accuracy with the proposed algorithm is 99.95%.

High angular resolution diffusion imaging (HARDI) has recently been of great interest to characterize non-Gaussian diffusion process. In the white matter of the brain, this occurs when fiber bundles cross, kiss or diverge within the same voxel. One of the important goal is to better describe the apparent diffusion process in these multiple fiber regions, thus overcoming the limitations of classical diffusion tensor imaging (DTI). In this paper, we design the appropriate mathematical tools to describe noisy HARDI data. Using a meaningful modified spherical harmonics basis to capture the physical constraints of the problem, we propose a new regularization algorithm to estimate a smoother and closer diffusivity profile to the true diffusivities without noise. We exploit properties of the spherical harmonics to define a smoothing term based on the Laplace-Beltrami for functions defined on the unit sphere. An additional contribution of the paper is the derivation of the general transformation taking the spherical harmonics coefficients to the high order tensor independent elements. This allows the careful study of the state of the art high order anisotropy measures computed from either spherical harmonics or tensor coefficients. We analyze their ability to characterize the underlying diffusion process. We are able to recover voxels with isotropic, single fiber anisotropic and multiple fiber anisotropic diffusion. We test and validate the approach on diffusion profiles from synthetic data and from a biological rat phantom.

The multiplicity of sensors used in medical imaging leads to different noises. Non informative noise can damage the image interpretation process and the performance of automatic analysis. The method proposed in this paper allows compensating highly noisy image data from non informative noise without sophisticated modeling of the noise statistics. This generic approach uses jointly a wavelet decomposition scheme and a non-isotropic Total Variation filtering of the transform coefficients. This framework benefits from both the hierarchical capabilities of the wavelet transform and the well-posed regularization scheme of the Total Variation. This algorithm has been tested and validated on test-bed data, as well as different clinical MR and 3D ultrasound images, enhancing the capabilities of the proposed method to cope with different noise models.

In this paper we seek to improve the standard method of assessing the degree of calcification in the lumbar aorta visualized on lateral 2-D X-rays. The semiquantitative method does not take density of calcification within the individual plaques into account and is unable to measure subtle changes in the severity of calcification over time. Both of these parameters would be desirable to assess, since they are the keys to assessing important information on the impact of risk factors and candidate drugs aiming at the prevention of atherosclerosis. As a further step for solving this task, we propose a pixelwise inpainting-based refinement scheme that seeks to optimize the individual plaque shape by maximizing the signal-to-noise ratio. Contrary to previous work the algorithm developped for this study uses a sorted candidate list, which omits possible bias introduced by the choice of starting pixel. The signal-to-noise optimization scheme will be discussed in different settings using TV as well as Harmonic inpainting and comparing these with a simple averaging process.

In the past decade, statistical shape modeling has been widely popularized in the medical image analysis community. Predominantly, principal component analysis (PCA) has been employed to model biological shape variability. Here, a reparameterization with orthogonal basis vectors is obtained such that the variance of the input data is maximized. This property drives models toward global shape deformations and has been highly successful in fitting shape models to new images. However, recent literature has indicated that this uncorrelated basis may be suboptimal for exploratory analyses and disease characterization. This paper explores the orthomax class of statistical methods for transforming variable loadings into a simple structure which is more easily interpreted by favoring sparsity. Further, we introduce these transformations into a particular framework traditionally based on PCA; the Active Appearance Models (AAMs). We note that the orthomax transformations are independent of domain dimensionality (2D/3D etc.) and spatial structure. Decompositions of both shape and texture models are carried out. Further, the issue of component ordering is treated by establishing a set of relevant criteria. Experimental results are given on chest radiographs, magnetic resonance images of the brain, and face images. Since pathologies are typically spatially localized, either with respect to shape or texture, we anticipate many medical applications where sparse parameterizations are preferable to the conventional global PCA approach.

We propose a multi-step approach for representing and classifying tree-like structures in medical images. Examples of such tree-like structures are encountered in the bronchial system, the vessel topology and the breast ductal network. We assume that the tree-like structures are already segmented. To avoid the tree isomorphism problem we obtain the breadth-first canonical form of a tree. Our approach is based on employing tree encoding techniques, such as the depth-first string encoding and the Prüfer encoding, to obtain a symbolic representation. Thus, the problem of classifying trees is reduced to string classification where node labels are the string terms. We employ the tf-idf text mining technique to assign a weight of significance to each string term (i.e., tree node label). We perform similarity searches and k-nearest neighbor classification of the trees using the tf-idf weight vectors and the cosine similarity metric. We applied our approach to the breast ductal network manually extracted from clinical x-ray galactograms. The goal was to characterize the ductal tree-like parenchymal structures in order to distinguish among different groups of women. Our best classification accuracy reached up to 90% for certain experimental settings (k=4), outperforming on the average by 10% that of a previous state-of-the-art method based on ramification matrices. These results illustrate the effectiveness of the proposed approach in analyzing tree-like patterns in breast images. Developing such automated tools for the analysis of tree-like structures in medical images can potentially provide insight to the relationship between the topology of branching and function or pathology.

We have developed a novel system that provides total support for assessment of alveolar bone resorption, caused by periodontitis, based on three-dimensional (3-D) dental CT images. In spite of the difficulty in perceiving the complex 3-D shape of resorption, dentists assessing resorption location and severity have been relying on two-dimensional radiography and probing, which merely provides one-dimensional information (depth) about resorption shape. However, there has been little work on assisting assessment of the disease by 3-D image processing and visualization techniques. This work provides quantitative evaluation results and figures for our system that measures the three-dimensional shape and spread of resorption. It has the following functions: (1) measures the depth of resorption by virtually simulating probing in the 3-D CT images, taking advantage of image processing of not suffering obstruction by teeth on the inter-proximal sides and much smaller measurement intervals than the conventional examination; (2) visualizes the disposition of the depth by movies and graphs; (3) produces a quantitative index and intuitive visual representation of the spread of resorption in the inter-radicular region in terms of area; and (4) calculates the volume of resorption as another severity index in the inter-radicular region and the region outside it. Experimental results in two cases of 3-D dental CT images and a comparison of the results with the clinical examination results and experts' measurements of the corresponding patients confirmed that the proposed system gives satisfying results, including 0.1 to 0.6mm of resorption measurement (probing) error and fairly intuitive presentation of measurement and calculation results.

Minimizing the description length (MDL) is one of the most promising methods to automatically generate 3D statistical shape models. By modifying an initial landmark distribution according to the MDL cost function, points across the different training shapes are brought into correspondence. A drawback of the current approach is that the user has no influence on the final landmark positions, which often do not represent the modeled shape adequately. We extend an existing remeshing technique to work with statistical shape models and show how the landmark distribution can be modified anytime during the model construction phase. This procedure is guided by a control map in parameter space that can be set up to produce any desired point distribution, e.g. equally spaced landmarks. To compare our remeshed models with the original approach, we generalize the established generalization and specificity measures to be independent of the underlying landmark distribution. This is accomplished by switching the internal metric from landmark distances to the Tanimoto coefficient, a volumetric overlap measure. In a concluding evaluation, we generate models for two medical datasets with and without landmark redistribution. As the outcome reveals, redistributing landmarks to an equally spaced distribution during the model construction phase improves the quality of the resulting models significantly if the shapes feature prominent bulges or other complex geometry.

In this paper we describe the application of folding measures to tracking in vivo cortical brain development in premature neonatal brain anatomy. The outer gray matter and the gray-white matter interface surfaces were extracted from semi-interactively segmented high-resolution T1 MRI data. Nine curvature- and geometric descriptor-based folding measures were applied to six premature infants, aged 28-37 weeks, using a direct voxelwise iso-surface representation. We have shown that using such an approach it is feasible to extract meaningful surfaces of adequate quality from typical clinically acquired neonatal MRI data. We have shown that most of the folding measures, including a new proposed measure, are sensitive to changes in age and therefore applicable in developing a model that tracks development in premature infants. For the first time gyrification measures have been computed on the gray-white matter interface and on cases whose age is representative of a period of intense brain development.

Tissue engineering is an interdisciplinary effort aimed at the repair and regeneration of biological tissues through the application and control of cells, porous scaffolds and growth factors. The regeneration of specific tissues guided by tissue analogous substrates is dependent on diverse scaffold architectural indices that can be derived quantitatively from the microCT and microMR images of the scaffolds. However, the randomness of pore-solid distributions in conventional stochastic scaffolds presents unique computational challenges. As a result, image-based characterization of scaffolds has been predominantly qualitative. In this paper, we discuss quantitative image-based techniques that can be used to compute the metrological indices of porous tissue engineering scaffolds. While bulk averaged quantities such as porosity and surface are derived directly from the optimal pore-solid delineations, the spatially distributed geometric indices are derived from the medial axis representations of the pore network. The computational framework proposed (to the best of our knowledge for the first time in tissue engineering) in this paper might have profound implications towards unraveling the symbiotic structure-function relationship of porous tissue engineering scaffolds.

In clinical MRI examinations, the geometry of diagnostic scans is defined in an initial planning phase. The operator plans the scan volumes (off-centre, angulation, field-of-view) with respect to patient anatomy in 'scout' images. Often multiple plans are required within a single examination, distracting attention from the patient waiting in the scanner. A novel and robust method is described for automated planning of neurological MRI scans, capable of handling strong shape deviations from healthy anatomy. The expert knowledge required to position scan geometries is learned from previous example plans, allowing site-specific styles to be readily taken into account. The proposed method first fits an anatomical model to the scout data, and then new scan geometries are positioned with respect to extracted landmarks. The accuracy of landmark extraction was measured to be comparable to the inter-observer variability, and automated plans are shown to be highly consistent with those created by expert operators using clinical data. The results of the presented evaluation demonstrate the robustness and applicability of the proposed approach, which has the potential to significantly improve clinical workflow.

Multiscale analysis provides a complete hierarchical partitioning of images into visually plausible regions. Each of them is formally characterized by a feature vector describing shape, texture and scale properties. Consequently, object extraction becomes a classification of the feature vectors. Classifiers are trained by relevant and irrelevant regions labeled as object and remaining partitions, respectively. A trained classifier is applicable to yet uncategorized partitionings to identify the corresponding region's classes. Such an approach enables retrieval of a-priori unknown objects within a point-and-click interface. In this work, the classification pipeline consists of a framework for data selection, feature selection, classifier training, classification of testing data, and evaluation. According to the no-free-lunch-theorem of supervised learning, the appropriate classification pipeline is determined experimentally. Therefore, each of the steps is varied by state-of-the-art methods and the respective classification quality is measured. Selection of training data from the ground truth is supported by bootstrapping, variance pooling, virtual training data, and cross validation. Feature selection for dimension reduction is performed by linear discriminant analysis, principal component analysis, and greedy selection. Competing classifiers are k-nearest-neighbor, Bayesian classifier, and the support vector machine. Quality is measured by precision and recall to reflect the retrieval task. A set of 105 hand radiographs from clinical routine serves as ground truth, where the metacarpal bones have been labeled manually. In total, 368 out of 39.017 regions are identified as relevant. In initial experiments for feature selection with the support vector machine have been obtained recall, precision and F-measure of 0.58, 0.67, and 0,62, respectively.

In medical image processing, several attempts have been made to develop filters which enhance certain structures in 3D data based on analysis of the Hessian matrix. These filters also tend to respond to other structures, e.g. most vessel enhancement filters also enhance nodule-like objects. In this paper, we use pattern recognition techniques to design more optimal filters. The essential difference with previous approaches is that we provide a system with examples of what it should enhance and suppress. These examples are used to train a classifier that determines the probability that a voxel in an unseen image belongs to the desired structures. The advantages of such an approach are excellent performance and flexibility: it can be used for any structure by providing the appropriate examples. We evaluated our approach on enhancing pulmonary fissures, which appear as plate-like structures in 3D CT chest scans. We compared our approach to the results of a recently proposed fissure enhancement filter. The results show that both methods are able to enhance the fissures, but our approach shows better performance; the areas under the ROC curves are 0.9044 and 0.7650, respectively.

Wireless Capsule Endoscopy (WCE) is a relatively new technology (FDA approved in 2002) allowing doctors to view most of the small intestine. Other endoscopies such as colonoscopy, upper gastrointestinal endoscopy, push enteroscopy, and intraoperative enteroscopy could be used to visualize up to the stomach, duodenum, colon, and terminal ileum, but there existed no method to view most of the small intestine without surgery. With the miniaturization of wireless and camera technologies came the ability to view the entire gestational track with little effort. A tiny disposable video capsule is swallowed, transmitting two images per second to a small data receiver worn by the patient on a belt. During an approximately 8-hour course, over 55,000 images are recorded to a worn device and then downloaded to a computer for later examination. Typically, a medical clinician spends more than two hours to analyze a WCE video. Research has been attempted to automatically find abnormal regions (especially bleeding) to reduce the time needed to analyze the videos. The manufacturers also provide the software tool to detect the bleeding called Suspected Blood Indicator (SBI), but its accuracy is not high enough to replace human examination. It was reported that the sensitivity and the specificity of SBI were about 72% and 85%, respectively. To address this problem, we propose a technique to detect the bleeding regions automatically utilizing the Expectation Maximization (EM) clustering algorithm. Our experimental results indicate that the proposed bleeding detection method achieves 92% and 98% of sensitivity and specificity, respectively.

We propose a new method for the analysis of functional magnetic resonance imaging (fMRI) which is called functional feature subspace mapping (FFSM). We mainly focused on the experimental design with periodic stimuli which can be described by a number of Fourier coefficients in the spectral domain. Then the subspace is obtained through the dimension reduction technique. Finally, the presence of activated time series is identified by the clustering method. Experiments with simulated data and the real human experiments are conducted to demonstrate that the algorithm we proposed is feasible. Although we focus on analyzing periodic fMRI data, the approach could be extended to analyze non-periodic fMRI data (event-related fMRI) by replacing the spectral analysis with a wavelet analysis.

Myocardial perfusion MRI has emerged as a suitable imaging technique for the detection of ischemic regions of the heart. However, manual post-processing is labor intensive, seriously hampering its daily clinical use. We propose a novel, data driven analysis method based on Independent Component Analysis (ICA). By performing ICA on the complete perfusion sequence, physiologically meaningful feature images, representing events occurring during the perfusion sequence, can be factored out. Results obtained using our method are compared with results obtained using manual contouring by a medical expert. The estimated weight functions are correlated against the perfusion time-intensity curves from manual contours, yielding promising results.

Radiologists can use the differences between the left and right breasts, or asymmetry, in mammograms to help detect certain malignant breast cancers. An image similarity method is introduced to make use of this knowledge base to recognize breast cancer. Image similarity is determined using a contextual and then a spatial comparison. The mammograms are filtered to find the most contextually significant points, and then the resulting point set is analyzed for spatial similarity. We develop the analysis through a combination of modeling and supervised learning of model parameters. This process correctly classifies mammograms 80% of the time and thus asymmetry is a measure that can play an important role in significantly improving computer-aided breast cancer detection systems.

Automated detection of lung nodules in thoracic CT scans is an important clinical challenge. Blood vessels form a major source of false positives in automated nodule detection systems. Hence, the performance of such systems may be improved by enhancing nodules while suppressing blood vessels. Ideally, nodule enhancement filters should enhance nodules while suppressing vessels and lung tissue. A distinction between vessels and nodules is normally obtained through eigenvalue analysis of the Hessian matrix. The Hessian matrix is a second order differential quantity and so is sensitive to noise. Furthermore, by relying on principal curvatures alone, existing filters are incapable of distinguishing between nodules and vessel junctions, and are incapable of handling cases in which nodules touch vessels. In this paper we develop novel nodule enhancement filters that are capable of suppressing junctions and are capable of handling cases in which nodules appear to touch or even overlap with vessels. The proposed filters are based on optimized probabilistic models derived from eigenvalue analysis of the gradient correlation matrix which is a first order differential quantity and so are less sensitive to noise compared with known vessel enhancement filters. The proposed filters are evaluated and compared to known techniques both qualitatively, quantitatively. The evaluation includes both synthetic and actual clinical data.

The detection of lung nodules located in opaque areas including the mediastinum, retrocardiac lung, and lung projected below or on the diaphragm has been very difficult, because the contrast of these nodules is usually extremely low, and sometimes radiologists may not pay attention to these locations. In this study, we have developed a new computer-aided diagnostic (CAD) scheme designed specifically for the detection of these difficult-to-detect lung nodules located in opaque areas. We used 1,000 chest images with 1,076 lung nodules, which included 73 very difficult lung nodules in these opaque areas. In this new computerized scheme, opaque areas within a chest image were segmented by use of an adaptive multi-thresholding method based on edge-gradient values, and then the gray level and contrast of the chest image were adjusted for the opaque areas. Initial candidates were identified by use of the nodule-enhanced image obtained with the average radial-gradient (ARG) filtering technique based on radial gradient values. We employed a total of 35 image features for sequential application of artificial neural networks (ANNs) in order to reduce the number of false-positive candidates. The ANNs were trained and tested by use of a k-fold cross-validation test method (k=100), in which each of 100 different combinations of training and test image data sets included 990 and 10 chest images, respectively. The overall performance determined from the results of 100 test data sets indicated that the average sensitivity in detecting lung nodules was 52.1% with 1.89 false positives per image, which was considered "acceptable", because these nodules were very subtle and difficult to detect. By combination of this advanced CAD scheme with our standard CAD scheme for lung-nodule detection, the clinical usefulness of the CAD scheme would be improved significantly.

A novel method called local shape controlled voting has been developed for spherical object detection in 3D voxel images. By combining local shape properties into the global tracking procedure of normal overlap, the proposed method solved the ambiguities of normal overlap between a small size sphere and a possible large size cylinder, as the normal overlap technique can only measures the 'density' of normal overlapping, while how the normal vectors are distributed in 3D is not discovered. The proposed method was applied to computer aided detection of small size pulmonary nodules based on helical CT images. Experiments showed that this method attained a better performance compared to the original normal overlap technique.

We are developing a CAD system for automated detection of ureter abnormalities on multi-detector row CT urography, which potentially can assist radiologists in detecting ureter cancer. In the first stage of the CAD system, given an initial starting point, the ureter is tracked based on the CT values of the contrast-filled lumen. In the second stage, lesion candidates are detected using histogram and shape analysis to separate the abnormality from the background, which is the ureter filled with contrast material. A uniformity measure is designed to detect non-uniformity of the CT values within the ureter volume. If a ureter abnormality is present, the CT values uniformity will be distorted, resulting in a reduced uniformity measure. The smoothness of the ureter wall is also estimated using a shape measure. A rule-based system is used to combine the two measures. In this pilot study, a limited data set of 11 patients with biopsy-proven lesions was used. Nine patients had 12 ureter cancers and 6 benign lesions and the remaining two patients had 2 benign lesions. The average lesions size for the 12 cancers was 7.8mm (range: 2.1mm-9.5mm). The tracking program successfully tracked the ureters in 10 of the patients. Our system detected 75% (15/20) of the ureter lesions with 2.6 (28/11) false positives per patient. 83% (10/12) of the ureter cancers were detected. The preliminary results show that our detection system can track the ureter and detect ureter cancer of medium conspicuity and relatively small size.

In this paper, we present performance of biplane correlation imaging (BCI) on set of chest x-ray projections of human data. BCI significantly minimizes the number of false positives (FPs) when used in conjunction with computer aided detection (CAD) by eliminating non-correlated nodule candidates. Sixty-one low exposure posterior projections were acquired from more than 20 human subjects with small angular separations (0.32 degree) over a range of 20 degrees along the vertical axis. All patients were previously diagnosed for the presence of lung nodules based on computed tomography (CT) examination. Images were processed following two steps. First, all images were analyzed using our CAD routine for chest radiography. This process proceeded with a BCI processing in which the results of CAD on each single projection were examined in terms of their geometrical correlation with those found in the other 60 projections based on the predetermined shift of possible nodule locations in each projection. The suspect entities with a geometrical correlation that coincided with the known location of the lesions were selected as nodules; otherwise they were ignored. An expert radiologist with reference to the associated CT dataset determined the truth regarding nodule location and sizes, which were then used to determine if the found nodules are true positive or false positive. The preliminary results indicated that the best performance was obtained when the angular separation of the projection pair was greater than about 6.7 degrees. Within the range of optimum angular separation, the number of FPs per image was 0-1 without impacting the number of true positives (TPs), averaged around 92%.

Although CAD schemes can detect a high percentage of subtle cancers depicted on "prior" and false-negative cases, radiologists frequently discard most of CAD-cued subtle masses in the clinical environment. As a result of the relatively high false-positive detection rate and the fact that a large number of subtle masses are typically cued only on one view cause radiologists to rely less on (and often ignore) CAD-cued masses. In this study, we present a multi-view based method to increase the number of actual masses that are cued by the CAD scheme on both (ipso-lateral) views and at the same time limiting the overall "case-based" false-positive detection rate. The new scheme includes a traditional single-image based CAD scheme and a multi-view processing module. An image database from 435 examinations (or a total of 1,740 images) consisting 235 examinations depicting a verified malignant mass each and 200 negative examinations was used in this study. The single-image based CAD scheme with a fixed operating threshold (i.e. 0.55) was applied to all images. For each CAD identified region (i.e. with detection score ≥ 0.55), the multi-view processing module defined a matched strip on the corresponding ipso-lateral image and identified all "matched" regions located inside the strip (including those with detection score < 0.55). All matched regions are cued on both views and unmatched regions were discarded. CAD scheme initially detected 172 true masses and 576 false-positive regions. Of the 172 masses, 90 were detected on two views (52%) and 82 were detected only on one view. Of the 576 false-positive detections, only 72 pairs (14%) were considered "matched" and 432 were not, resulting in 504 case-based ("independent") cues. Case-based sensitivity and false-positive rate of the CAD scheme were 73.2% and 1.16 per case. When only matched region pairs were cued, 160 masses (68.1%) and 308 false-positive detections (0.71 per case) were identified on two views. Reducing the original operating threshold from 0.55 to 0.49 resulted in 172 masses and 413 false-positive detections (or 0.95 per case) being cued on two views. By increasing sensitivity through the reduction of the detection threshold only inside the matched strip regions in ipso-lateral images the scheme successfully cued masses on two views by maintaining detection sensitivity while reducing the "case-based" false-positive detection rate.

In this paper we present a novel approach for mass detection in Digital Breast Tomosynthesis (DBT) datasets. A reconstruction-independent approach, working directly on the projected views, is proposed. Wavelet filter responses on the projections are thresholded and combined to obtain candidate masses. For each candidate, we create a fuzzy contour through a multi-level thresholding process. We introduce a fuzzy set definition for the class mass contour that allows the computation of fuzzy membership values for each candidate contour. Then, an aggregation operator is presented that combines information over the complete set of projected views, resulting in 3D fuzzy particles. A final decision is made taking into account all available information. The performance of the presented algorithm was evaluated on a database of 11 one-breast-cases resulting in a sensitivity (Se) of 0.86 and a false positive rate (FPR) of 3.5 per case.

A novel technique for the detection and enhancement of microcalcifications in digital tomosynthesis mammography (DTM) is presented. In this method, the DTM projection images are used directly, instead of using a 3D reconstruction. Calcification residual images are computed for each of the projection images. Calcification detection is then performed over 3D space, based on the values of the calcification residual images at projection points for each 3D point under test. The quantum, electronic, and tissue noise variance at each pixel in each of the calcification residuals is incorporated into the detection algorithm. The 3D calcification detection algorithm finds a minimum variance estimate of calcification attenuation present in 3D space based on the signal and variance of the calcification residual images at the corresponding points in the projection images. The method effectively detects calcifications in 3D in a way that both ameliorates the difficulties of joint tissue/microcalcification tomosynthetic reconstruction (streak artifacts, etc.) and exploits the well understood image properties of microcalcifications as they appear in 2D mammograms. In this method, 3D reconstruction and calcification detection and enhancement are effectively combined to create a calcification detection specific reconstruction. Motivation and details of the technique and statistical results for DTM data are provided.

In this study, our purpose was to develop a false positive (FP) reduction method for computerized mass detection systems based on the analysis of bilateral mammograms. We first detect the mass candidates on each view by utilizing our unilateral computer-aided detection (CAD) system. For each detected object, the regional registration technique is used to define a region of interest (ROI) that is "symmetrical" to the object location on the contralateral mammogram. Spatial gray level dependence matrices (SGLD) texture features and morphological features are extracted from both the ROI containing the detected object on a mammogram and its corresponding ROI on the contralateral mammogram. Bilateral features are then generated from the extracted unilateral features and a final bilateral score is formed as a new feature to differentiate symmetric from asymmetric ROIs. By incorporating the unilateral features of the mass candidates and their bilateral scores, a bilateral classifier was trained to reduce the FPs. It was found that our bilateral CAD system achieved a case-based sensitivity of 70%, 80%, and 85% at 0.52, 0.83, and 1.05 FPs/image on the test data set. In comparison to the FP rates for the unilateral CAD system of 0.67, 1.11, and 1.69, respectively, at the corresponding sensitivities, the FP rates were reduced by 22%, 25%, and 37% with the bilateral symmetry information.

Digital bowel cleansing (DBC) is an emerging method for segmentation of fecal materials, which are tagged by an X-ray-opaque oral contrast agent in CT colonography (CTC) images, effectively removing them for digital cleansing of the colon. Existing DBC approaches tend to use simple thresholding-based methods for the removal of tagged fecal materials; however, because of the pseudo-enhancement of polyps caused by the surrounding tagged fecal materials, such methods tend erroneously to remove a part of or the entire polyps submerged in these materials. In this study, we developed a novel DBC method that preserves the soft-tissue structures submerged in or partially covered by tagged fecal materials. In our approach, submerged soft-tissue structures are characterized by their local shape signatures that are calculated based on the eigenvalues of a Hessian matrix. A structure-enhancement function is formulated for enhancing of the soft-tissue structures, and the values of the function are integrated into the speed function of a level set method to delineate the submerged soft-tissue structures while removing the tagged fecal materials. In an analysis of 10 submerged polyps, our new DBC method was shown to delineate polyps better than was possible with our previously reported cleansing method based on thresholding. Application of our computer-aided detection (CAD) scheme showed that the use of the new DBS method substantially reduced the number of false-positive detections compared with those of our previous, thresholding-based method.

Automated and accurate segmentation of the aorta in 3D+time MR image data is important for early detection of connective tissue disorders leading to aortic aneurysms and dissections. A computer-aided diagnosis method is reported that allows the objective identification of subjects with connective tissue disorders from two-phase 3D+time aortic MR images. Our automated segmentation method combines level-set and optimal border detection. The resulting aortic lumen surface was registered with an aortic model followed by calculation of modal indices of aortic shape and motion. The modal indices reflect the differences of any individual aortic shape and motion from an average aortic behavior. The indices were input to a Support Vector Machine (SVM) classifier and a discrimination model was constructed. 3D+time MR image data sets acquired from 22 normal and connective tissue disorder subjects at end-diastole (R-wave peak) and at 45% of the R-R interval were used to evaluate the performance of our method. The automated 3D segmentation result produced accurate aortic surfaces covering the aorta from the left-ventricular outflow tract to the diaphragm and yielded subvoxel accuracy with signed surface positioning errors of -0.09±1.21 voxel (-0.15±2.11 mm). The computer aided diagnosis method distinguished between normal and connective tissue disorder subjects with a classification correctness of 90.1 %.

We are developing a two-view information fusion method to improve the performance of our CAD system for mass detection. Mass candidates on each mammogram were first detected with our single-view CAD system. Potential object pairs on the two-view mammograms were then identified by using the distance between the object and the nipple. Morphological features, Hessian feature, correlation coefficients between the two paired objects and texture features were used as input to train a similarity classifier that estimated a similarity scores for each pair. Finally, a linear discriminant analysis (LDA) classifier was used to fuse the score from the single-view CAD system and the similarity score. A data set of 475 patients containing 972 mammograms with 475 biopsy-proven masses was used to train and test the CAD system. All cases contained the CC view and the MLO or LM view. We randomly divided the data set into two independent sets of 243 cases and 232 cases. The training and testing were performed using the 2-fold cross validation method. The detection performance of the CAD system was assessed by free response receiver operating characteristic (FROC) analysis. The average test FROC curve was obtained from averaging the FP rates at the same sensitivity along the two corresponding test FROC curves from the 2-fold cross validation. At the case-based sensitivities of 90%, 85% and 80% on the test set, the single-view CAD system achieved an FP rate of 2.0, 1.5, and 1.2 FPs/image, respectively. With the two-view fusion system, the FP rates were reduced to 1.7, 1.3, and 1.0 FPs/image, respectively, at the corresponding sensitivities. The improvement was found to be statistically significant (p<0.05) by the AFROC method. Our results indicate that the two-view fusion scheme can improve the performance of mass detection on mammograms.

We have analyzed the branching patterns of the breast ductal network visible in magnetic resonance (MR) autogalactograms - images of breast ducts which appear enhanced due to the presence of proteinacous or hemorrhagic material in the ducts. The enhanced portions of the ductal network were segmented separately in MRI slices acquired with a 3D GRASS sequence. A semi-automated region growing algorithm was used for segmentation. The ductal network was manually constructed from the segmented portions in each slice. The branching pattern was analyzed by calculating ramification (R-) matrices, whose elements represent the probabilities of branching at various levels of a ductal tree. The R-matrix elements have been used to classify the analyzed cases into those with and without radiological findings. The classification accuracy was estimated using the radiologists' reports as ground truth. An ROC analysis was performed to assess the classification accuracy. The classification of nine MR autogalactograms from eight women yielded an area under the ROC curve of A=0.73. This performance is comparable with our previous analysis of 25 2D x-ray galactograms from 15 women (A=0.88). The observed results support our hypothesis that a relationship exists between the topological properties of the breast ductal network and the underlying breast pathology.

Elastic registration of medical images is an active field of current research. Registration algorithms have to be validated in order to show that they fulfill the requirements of a particular clinical application. Furthermore, validation strategies compare the performance of different registration algorithms and can hence judge which algorithm is best suited for a target application. In the literature, validation strategies for rigid registration algorithms have been analyzed. For a known ground truth they assess the displacement error at a few landmarks, which is not sufficient for elastic transformations described by a huge number of parameters. Hence we consider the displacement error averaged over all pixels in the whole image or in a region-of-interest of clinical relevance. Using artificially, but realistically deformed images of the application domain, we use this quality measure to analyze an elastic registration based on transformations defined on adaptive irregular grids for the following clinical applications: Magnetic Resonance (MR) images of freely moving joints for orthopedic investigations, thoracic Computed Tomography (CT) images for the detection of pulmonary embolisms, and transmission images as used for the attenuation correction and registration of independently acquired Positron Emission Tomography (PET) and CT images. The definition of a region-of-interest allows to restrict the analysis of the registration accuracy to clinically relevant image areas. The behaviour of the displacement error as a function of the number of transformation control points and their placement can be used for identifying the best strategy for the initial placement of the control points.

Assembling partial views is an attractive means to extend the field of view of microscope images. In this paper, we propose a semi-automated solution to achieve this goal. Its intended audience is the microscopist who desires to scan a large area while acquiring a series of partial views, but who does not wish to--or cannot--planify the path of the scan. In a first stage, this freedom is dealt with by interactive manipulation of the resulting partial views, or tiles. In a second stage, the position of the tiles is refined by a fully automatic pairwise registration process. The contribution of this paper is a strategy that determines which pairs of tiles to register, among all possible pairs. The central tenet of our proposed strategy is that two tiles that happen to possess a large common area will register with higher accuracy than two tiles with a smaller overlap. Our strategy is then to minimize the number of pairwise registrations while maximizing the global amount of overlap, and while ensuring that the local registration efforts are sufficient to link all tiles together to yield a global mosaic. By stating this requirement in a graph-theoretic context, we are able to derive the optimal solution thanks to Kruskal's algorithm.

Registration of images is a crucial part of many medical imaging tasks. The problem is to find a transformation which aligns two given images. The resulting displacement fields may be for example described as a linear combination of pre-selected basis functions (parametric approach), or, as in our case, they may be computed as the solution of an associated partial differential equation (non-parametric approach). Here, the underlying functional consists of a smoothness term ensuring that the transformation is anatomically meaningful and a distance term describing the similarity between the two images. To be successful, the registration scheme has to be tuned for the problem under consideration. One way of incorporating user knowledge is the employment of weighting masks into the distance measure, and thereby enhancing or hiding dedicated image parts. In general, these masks are based on a given segmentation of both images. We present a method which generates a weighting mask for the second image, given the mask for the first image. The scheme is based on active contours and makes use of a gradient vector flow method. As an example application, we consider the registration of abdominal computer tomography (CT) images used for radiation therapy. The reference image is acquired well ahead of time and is used for setting up the radiation plan. The second image is taken just before the treatment and its processing is time-critical. We show that the proposed automatic mask generation scheme yields similar results as compared to the approach based on a pre-segmentation of both images. Hence for time-critical applications, as intra-surgery registration, we are able to significantly speed up the computation by avoiding a pre-segmentation of the second image.

Dynamic contrast enhanced (DCE) MRI mammography is currently receiving much interest in clinical research. It bears the potential to discriminate between benign and malignant lesions by analysis of the contrast uptake of the lesion. However, a registration of the individual images of a contrast-uptake series is crucial in order to avoid motion artefacts in the uptake curves, which could affect the diagnosis. It is on the other hand well known from the registration literature that a registration that uses a standard similarity measure (e.g. mean sum of squared differences, cross-correlation) may cause artefacts if contrast agent is taken up between the images to be registered. Thus we propose a registration on the basis of an application-specific similarity measure that explicitly uses features of the contrast uptake. We report initial results using this registration method.

Multi-view imaging is the primary modality for high-spatial-resolution imaging of the vasculature. The 3D vascular structure can be reconstructed if the imaging geometries are determined using known corresponding point-pairs (or k-tuples) in two or more images. Because the accuracy improves with more input corresponding point-pairs, we propose a new technique to automatically determine corresponding point-pairs in multi-view (k-view) images, from 2D vessel image centerlines. We formulate the problem, first as a multi-partite graph-matching problem. Each 2D centerline point is a vertex; each individual graph contains all vessel-points (vertices) in an image. The weight ('cost') of the edges between vertices (in different graphs) is the shortest distance between the points' respective projection-lines. Using this construction, a universe of mappings (k-tuples) is created, each k-tuple having k vertices (one from each image). A k-tuple's weight is the sum of pair-wise 'costs' of its members. Ideally, a set of such mappings is desired that preserves the ordering of points along the vessel and minimizes an appropriate global cost function, such that all vertices (in all graphs) participate in at least one mapping. We formulate this problem as a special case of the well-studied Set-Cover problem with additional constraints. Then, the equivalent linear program is solved, and randomized-rounding techniques are used to yield a feasible set of mappings. Our algorithm is efficient and yields a theoretical quality guarantee. In simulations, the correct matching is achieved in ~98% cases, even with high input error. In clinical data, apparently correct matching is achieved in >90% cases. This method should provide the basis for improving the calculated 3D vasculature from multi-view data-sets.

In carotid plaque imaging, MRI provides exquisite soft-tissue characterization, but lacks the temporal resolution for tissue strain imaging that real-time 3D ultrasound (3DUS) can provide. On the other hand, real-time 3DUS currently lacks the spatial resolution of carotid MRI. Non-rigid alignment of ultrasound and MRI data is essential for integrating complementary morphology and biomechanical information for carotid vascular assessment. We assessed non-rigid registration for fusion of 3DUS and MRI carotid data based on deformable models which are warped to maximize voxel similarity. We performed validation in vitro using isolated carotid artery imaging. These samples were subjected to soft-tissue deformations during 3DUS and were imaged in a static configuration with standard MR carotid pulse sequences. Registration of the source ultrasound sequences to the target MR volume was performed and the mean absolute distance between fiducials within the ultrasound and MR datasets was measured to determine inter-modality alignment quality. Our results indicate that registration errors on the order of 1mm are possible in vitro despite the low-resolution of current generation 3DUS transducers. Registration performance should be further improved with the use of higher frequency 3DUS prototypes and efforts are underway to test those probes for in vivo 3DUS carotid imaging.

Several 3D/2D registration algorithms for image-guided therapy have been introduced in the past years. Recently, we have proposed a method which first reconstructs a 3D image from a few intraoperative 2D X-ray images and then establishes the rigid transformation between the preoperative 3D CT or MR image and the 3D reconstructed image. The similarity measure applied in this registration method should be able to cope, among others, with the low quality of the reconstructed image. Using the recently proposed similarity measure evaluation protocol, we have evaluated the behavior of five similarity measures. The measures have been evaluated with respect to: a) preoperative imaging modalities (CT and MR); b) number of 2D images used for reconstruction; and c) number of reconstruction iterations. Increasing the number of 2D projections or reconstruction iterations improves the accuracy but slightly worsens the robustness. We have shown that almost all similarity measures have better properties if the optimal parameters are chosen. The most appropriate similarity measure for this type of registration is the asymmetric multi-feature mutual information.

In this paper, we describe a new methodology for keypoint-based affine and deformable medical image registration. This fast and computationally efficient method is automatic and does not rely on segmentation of images. The keypoint pixels used in this technique are extreme points in the scale space and are characterized by descriptor vectors which summarize the intensity gradient profile of the surrounding pixels. For each of the keypoints in the scene image, a corresponding keypoint is identified in the model image using the feature space nearest neighbor criteria. For deformable registration, B-splines are used to extrapolate a regular deformation grid for all of the pixels in the scene image based on the relative displacement vectors of the corresponding pairs. This approach results in a fast and accurate registration in the brain MRI images (an average target registration error of less than 2mm was acquired). We have also studied the affine registration problem in the liver ultrasound and brain MRI images and have acquired acceptable registrations using a mean square solution for affine parameters based on only around 30 corresponding keypoint pairs.

Registration of an individual's image data set to an anatomical atlas provides valuable information to the physician. In many cases, the individual image data sets are partial data, which may be mapped to one part or one organ of the entire atlas data. Most of the existing intensity based image registration approaches are designed to align images of the entire view. When they are applied to the registration with partial data, a manual pre-registration is usually required. This paper proposes a fully automatic approach to the registration of the incomplete image data to an anatomical atlas. The spatial transformations are modelled as any parametric functions. The proposed method is built upon a random search mechanism, which allows to find the optimal transformation randomly and globally even when the initialization is not ideal. It works more reliably than the existing methods for the partial data registration because it successfully overcomes the local optimum problem. With appropriate similarity measures, this framework is applicable to both mono-modal and multi-modal registration problems with partial data. The contribution of this work is the description of the mathematical framework of the proposed algorithm and the implementation of the related software. The medical evaluation on the MRI data and the comparison of the proposed method with different existing registration methods show the feasibility and superiority of the proposed method.

Geometric distortion is a well-recognized problem in echo planar (EP) images. One strategy for the correction of these distortions is to register an EP image to a reference image, such as a high resolution anatomical MR image in which geometric distortion is minimal. Non-rigid registration methods, which warp images locally, have been used for this purpose. While a physics-based distortion model for spin-echo (SE) EP image has been developed and used as a constraint in nonrigid registration algorithms, such a model for gradient-echo (GE) EP image has not been investigated. Here, we propose to use a physics-based model for GE EP image that incorporates a term that takes dephasing into consideration. To evaluate this technique, we generate a distortion-free EP image using an MR simulator we have developed. We then distort the image and modify its intensity values using a real field map and an analytical expression that includes dephasing. The geometric distortion computed from the field map is used as the ground truth to which the deformation fields obtained with our method is compared. We show that including the dephasing term improves the results.

Similarity metric optimization is an essential step in intensity-based rigid and nonrigid medical image registration. For clinical applications, such as image guidance of minimally invasive procedures, registration accuracy and efficiency are prime considerations. In addition, clinical utility is enhanced when registration is integrated into image analysis and visualization frameworks, such as the popular Insight Toolkit (ITK). ITK is an open source software environment increasingly used to aid the development, testing, and integration of new imaging algorithms. In this paper, we present a new ITK-based implementation of the DIRECT (Dividing Rectangles) deterministic global optimization algorithm for medical image registration. Previously, it has been shown that DIRECT improves the capture range and accuracy for rigid registration. Our ITK class also contains enhancements over the original DIRECT algorithm by improving stopping criteria, adaptively adjusting a locality parameter, and by incorporating Powell's method for local refinement. 3D-3D registration experiments with ground-truth brain volumes and clinical cardiac volumes show that combining DIRECT with Powell's method improves registration accuracy over Powell's method used alone, is less sensitive to initial misorientation errors, and, with the new stopping criteria, facilitates adequate exploration of the search space without expending expensive iterations on non-improving function evaluations. Finally, in this framework, a new parallel implementation for computing mutual information is presented, resulting in near-linear speedup with two processors.

Modern tomographic imaging devices enable the acquisition of temporal image sequences. In our project, we study cine MRI sequences of patients with myocardial infarction. Because the sequences are acquired with different temporal resolutions, a temporal interpolation is necessary to compare images at predefined phases of the cardiac cycle. This paper presents an interpolation method for temporal image sequences. We derive our interpolation scheme from the optical flow equation. The spatiotemporal velocity field between the images is determined using an optical flow based registration method. Here, an iterative algorithm is applied, using the spatial and temporal image derivatives and a spatiotemporal smoothing step. Afterwards, the calculated velocity field is used to generate an interpolated image at the desired time by averaging intensities between corresponding points. The behavior and capability of the algorithm is demonstrated by synthetic image examples. Furthermore, quantitative measures are calculated to compare this optical flow based interpolation method to linear interpolation and shape-based interpolation in 5 cine MRI data sets. Results indicate that the presented method outperforms both linear and shape-based interpolation significantly.

Respiratory motion is a significant source of error in conformal radiation therapy for the thorax and upper abdomen. Four-dimensional computed tomography (4D CT) has been proposed to reduce the uncertainty caused by internal respiratory organ motion. A 4D CT dataset is retrospectively reconstructed at various stages of a respiratory cycle. An important tool for 4D treatment planning is deformable image registration. An inverse consistent image registration is used to model lung motion from one respiratory stage to another during a breathing cycle. This diffeomorphic registration jointly estimates the forward and reverse transformations providing more accurate correspondence between two images. Registration results and modeled motions in the lung are shown for three example respiratory stages. The results demonstrate that the consistent image registration satisfactorily models the large motions in the lung, providing a useful tool for 4D planning and delivering.

Generation of digitally reconstructed radiographs (DRR) is a critical part of 2D-3D image registration that is utilized in patient position alignment for image-guided radiotherapy and radiosurgery. The DRRs are generated from a pre-operative CT scan and used as the references to match the X-ray images for determining the change of patient position. Skeletal structures are the primary image features to facilitate the registration between the DRR and X-ray images. In this paper, we present a method to enhance skeletal features of spinal regions in DRRs. The attenuation coefficient at each voxel is first calculated by applying an exponential transformation of the original attenuation coefficient in the CT scan. This is a preprocessing step that is performed prior to DRR generation. The DRR is then generated by integrating the newly calculated attenuation coefficients along the ray that connects the X-ray source and the pixel in the DRR. Finally, the DRR is further enhanced using a weighted top-hat filter. During the entire process, because there is no original CT information lost, even the small skeletal features contributed by low intensity part of CT data are preserved in the enhanced DRRs. Experiments on clinical data were conducted to compare the image quality of DRRs with and without enhancement. The results showed that the image contrast of skeletal features in the enhanced DRRs is significantly improved. This method has potential to be applied for more accurate and robust 2D-3D image registration.

In earlier work (KIM, J.S, MBEC, 2003), we demonstrated the registration method with a non-linear transformation using intensity similarity and feature similarity. Although the former approach showed good match in global shape of brain and feature-defined region, method contains user interventions for defining appropriate and sufficient number features. While manual delineating the region of interests for sufficient number of feature is a very time-consuming and can provide intra-, inter-rater variability, we proposed fully automatic hybrid registration via automatic feature defining method. Automatic feature definition was performed on the cortical surface from CLASP (KIM, J.S, Neuroimage, 2005) with using cortical surface matching algorithm (Robbins, S., MIA, 2004) and then applied to hybrid registration. The object of this work is to develop fully automated hybrid registration method which reveals enhanced performance in comparison to previous automated registration methods. In the result, our proposed scheme showed efficient performance from maintaining the strong points of hybrid registration without any user intervention.

In this paper we propose a new method for non-rigid registration of PET/CT datasets incorporating prior knowledge about the rigidity of regions within the PET volumes into the matching process. State-of-the-art medical image registration approaches usually assume that the whole image domain is associated with a homogeneous deformation property, thus bone structure and soft tissue have the same stiffness, for instance. This assumption, however, is invalid in the majority of cases. In many applications the deformation properties can be estimated automatically by a segmentation step, beforehand. The presented non-rigid registration method integrates knowledge about the tissue directly into the deformation field computation. For this reason, no additional post-processing steps, like filtering of the deformation field, are required. To integrate the tissue constraints the regularizer is replaced by a novel spatially dependent smoother. Dependent on the location within the image, the smoother is able to explicitly adjust the rigidity. Thus, different tissue classes can be treated in the registration process. To pass the stiffness coefficients to the algorithm an additional mask image is used. The registration results are illustrated on synthetic data first to give a good intuition about the effectiveness of the proposed method. Finally, we illustrate the improvement of the registration using real clinical data. It is shown that the mono-modal registration of PET images yields more reasonable results using a spatially dependent regularizer constraining the deformations of regions with high tracer concentration than using a normal curvature regularizer. Furthermore, the method is evaluated on multi-modal PET/CT registration problems.

We present a novel representation of 3D salient region features and its integration into a hybrid rigid-body registration framework. We adopt scale, translation and rotation invariance properties of those intrinsic 3D features to estimate a transform between underlying mono- or multi-modal 3D medical images. Our method combines advantageous aspects of both feature- and intensity-based approaches and consists of three steps: an automatic extraction of a set of 3D salient region features on each image, a robust estimation of correspondences and their sub-pixel accurate refinement with outliers elimination. We propose a region-growing based approach for the extraction of 3D salient region features, a solution to the problem of feature clustering and a reduction of the correspondence search space complexity. Results of the developed algorithm are presented for both mono- and multi-modal intra-patient 3D image pairs (CT, PET and SPECT) that have been acquired for change detection, tumor localization, and time based intra-person studies. The accuracy of the method is clinically evaluated by a medical expert with an approach that measures the distance between a set of selected corresponding points consisting of both anatomical and functional structures or lesion sites. This demonstrates the robustness of the proposed method to image overlap, missing information and artefacts. We conclude by discussing potential medical applications and possibilities for integration into a non-rigid registration framework.

In this paper, we present a framework that one could use to set optimized parameter values, while performing image registration using mutual information as a metric to be maximized. Our experiment details these steps for the registration of X-ray Computer Tomography (CT) images with Positron Emission Tomography (PET) images. Selection of different parameters that influence the mutual information between two images is crucial for both accuracy and speed of registration. These implementation issues need to be handled in an orderly fashion by designing experiments in their operating ranges. The conclusions from this study seem vital towards obtaining allowable parameter range for a fusion software.

Several approaches for registering a subset of imaged points to their true origins were analyzed and compared for seed based TRUS-fluoroscopy registration. The methods include the Downhill Simplex method (DS), the Powell's method (POW), the Iterative Closest Point (ICP) method, the Robust Point Matching method (RPM) and variants of RPM. Several modifications were made to the standard RPM method to improve its performance. One hundred simulations were performed for each combination of noise level, seed detection rate and spurious points and the registration accuracy was evaluated and compared. The noise level ranges from 0 to 5mm, the seed detection ratio ranges from 0.2 to 0.6, and the number of spurious points ranges from 0 to 20. An actual clinical post-implant dataset from permanent prostate brachytherapy was used for the simulation study. The experiments provided evidence that our modified RPM method is superior to other methods, especially when there are many outliers. The RPM based method produced the best results at all noise levels and seed detection rates. The DS based method performed reasonably well, especially at low noise levels without spurious points. There was no significant performance difference between the standard RPM and our modified RPM methods without spurious points. The modified RPM methods outperformed the standard RPM method with large number of spurious points. The registration error was within 2mm, even with 20 outlier points and a noise level of 3mm.

We present a novel, shape-constrained mutual information-based image registration method using a Genetic Algorithm (GA) for aligning 2D MR images of the in vivo human tongue. By restricting the computation of Mutual Information (MI) using segmented object, we are able to register images in a short time without compromising accuracy. In addition, due to the employment of GA, the robustness of this method is greatly improved such that good registration results can be achieved given large initial transformation difference. Hence, this method eliminates the failure of matching caused by partial object overlap. Our experiments clearly demonstrate the robustness and accuracy of the method.

We propose a fast and robust registration method for matching lung nodules of temporal chest CT scans. Our method is composed of four stages. First, the lungs are extracted from chest CT scans by the automatic segmentation method. Second, the gross translational mismatch is corrected by the optimal cube registration. This initial registration does not require extracting any anatomical landmarks. Third, initial alignment is step by step refined by the iterative surface registration. To evaluate the distance measure between surface boundary points, a 3D distance map is generated by the narrow-band distance propagation, which drives fast and robust convergence to the optimal location. Fourth, nodule correspondences are established by the pairs with the smallest Euclidean distances. The results of pulmonary nodule alignment of twenty patients are reported on a per-center-of mass point basis using the average Euclidean distance (AED) error between corresponding nodules of initial and follow-up scans. The average AED error of twenty patients is significantly reduced to 4.7mm from 30.0mm by our registration. Experimental results show that our registration method aligns the lung nodules much faster than the conventional ones using a distance measure. Accurate and fast result of our method would be more useful for the radiologist's evaluation of pulmonary nodules on chest CT scans.

In this paper we focus on the problem of automatically registering dermatological images, because even if different products are available, most of them share the problem of a limited field of view on the skin. A possible solution is then the composition of multiple takes of the same lesion with digital software, such as that for panorama images creation. In this work, to perform an automatic selection of matching points the Harris Corner Detector is used, and to cope with outlier couples we employed the RANSAC method. Projective mapping is then used to match the two images. Given a set of correspondence points, Singular Value Decomposition was used to compute the transform parameters. At this point the two images need to be blended together. One initial assumption is often implicitly made: the aim is to merge two rectangular images. But when merging occurs between more than two images iteratively, this assumption will fail. To cope with differently shaped images, we employed the Distance Transform and provided a weighted merging of images. Different tests were conducted with dermatological images, both with standard rectangular frame and with not typical shapes, as for example a ring due to the objective and lens selection. The successive composition of different circular images with other blending functions, such as the Hat function, doesn't correctly get rid of the border and residuals of the circular mask are still visible. By applying Distance Transform blending, the result produced is insensitive of the outer shape of the image.

Inactivation of the retinoblastoma gene in mouse embryos causes tissue infiltrations into critical sections of the placenta, which has been shown to affect fetal survivability. Our collaborators in cancer genetics are extremely interested in examining the three dimensional nature of these infiltrations given a stack of two dimensional light microscopy images. Three sets of wildtype and mutant placentas was sectioned serially and digitized using a commercial light microscopy scanner. Each individual placenta dataset consisted of approximately 1000 images totaling 700 GB in size, which were registered into a volumetric dataset using National Library of Medicine's (NIH/NLM) Insight Segmentation and Registration Toolkit (ITK). This paper describes our method for image registration to aid in volume visualization of tissue level intermixing for both wildtype and Rb^- specimens. The registration process faces many challenges arising from the large image sizes, damages during sectioning, staining gradients both within and across sections, and background noise. These issues limit the direct application of standard registration techniques due to frequent convergence to local solutions. In this work, we develop a mixture of automated and semi-automated enhancements with ground-truth validation for the mutual information-based registration algorithm. Our final volume renderings clearly show tissue intermixing differences between both wildtype and Rb^- specimens which are not obvious prior to registration.

In-vivo quantitative assessments of joint conditions and health status can help to increase understanding of the pathology of osteoarthritis, a degenerative joint disease that affects a large population each year. Magnetic resonance imaging (MRI) provides a non-invasive and accurate means to assess and monitor joint properties, and has become widely used for diagnosis and biomechanics studies. Quantitative analyses and comparisons of MR datasets require accurate alignment of anatomical structures, thus image registration becomes a necessary procedure for these applications. This research focuses on developing a registration technique for MR knee joint surfaces to allow quantitative study of joint injuries and health status. It introduces a novel idea of translating techniques originally developed for geographic data in the field of photogrammetry and remote sensing to register 3D MR data. The proposed algorithm works with surfaces that are represented by randomly distributed points with no requirement of known correspondences. The algorithm performs matching locally by identifying corresponding surface elements, and solves for the transformation parameters relating the surfaces by minimizing normal distances between them. This technique was used in three applications to: 1) register temporal MR data to verify the feasibility of the algorithm to help monitor diseases, 2) quantify patellar movement with respect to the femur based on the transformation parameters, and 3) quantify changes in contact area locations between the patellar and femoral cartilage at different knee flexion angles. The results indicate accurate registration and the proposed algorithm can be applied for in-vivo study of joint injuries with MRI.

We present an elastic registration algorithm based on local deformations modeled using cubic B-splines and controlled using 3D ChainMail. Our algorithm eliminates the appearance of folding artifacts and allows local rigidity and compressibility control independent of the image similarity metric being used. 3D ChainMail propagates large internal deformations between neighboring B-Spline control points, thereby preserving the topology of the transformed image without requiring the addition of penalty terms based on rigidity of the transformation field to the equation used to maximize image similarity. A novel application to virtual colonoscopy is presented where the algorithm is used to significantly improve cross-localization between colon locations in prone and supine CT images.

We address the problem of automated image alignment for 3D volume reconstruction from stacks of fluorescent confocal laser scanning microscope (CLSM) imagery acquired at multiple confocal depths, from a sequence of consecutive slides. We focus on automated image alignment based on centroid and area shape features by solving feature correspondence problem, also known as Procrustes problem, in highly deformable and ill-conditioned feature space. In result, we compare image alignment accuracy of a fully automated method with registration accuracy achieved by human subjects using a manual alignment method. Our work demonstrates significant benefits of automation for 3D volume reconstruction in terms of accuracy, consistency, and performance time. We also outline the limitations of fully automated and manual 3D volume reconstruction system.

An automatic 3D non-rigid body registration system based upon the genetic algorithm (GA) process is presented. The system has been successfully applied to 2D and 3D situations using both rigid-body and affine transformations. Conventional optimization techniques and gradient search strategies generally require a good initial start location. The GA approach avoids the local minima/maxima traps of conventional optimization techniques. Based on the principles of Darwinian natural selection (survival of the fittest), the genetic algorithm has two basic steps: 1. Randomly generate an initial population. 2. Repeated application of the natural selection operation until a termination measure is satisfied. The natural selection process selects individuals based on their fitness to participate in the genetic operations; and it creates new individuals by inheritance from both parents, genetic recombination (crossover) and mutation. Once the termination criteria are satisfied, the optimum is selected from the population. The algorithm was applied on 2D and 3D magnetic resonance images (MRI). It does not require any preprocessing such as threshold, smoothing, segmentation, or definition of base points or edges. To evaluate the performance of the GA registration, the results were compared with results of the Automatic Image Registration technique (AIR) and manual registration which was used as the gold standard. Results showed that our GA implementation was a robust algorithm and gives very close results to the gold standard. A pre-cropping strategy was also discussed as an efficient preprocessing step to enhance the registration accuracy.

Xalign is a tool designed to measure implant orientation after joint arthroplasty by co-registering a projection of an implant model and a digitally reconstructed radiograph of the patient's anatomy with a post operative x-ray. A mutual information based registration method is used to automate alignment. When using basic mutual information, the presence of local maxima can result in misregistration. To increase robustness of registration, our research is aimed at improving the similarity function by modifying the information measure and incorporating local spatial information. A test dataset with known groundtruth parameters was created to evaluate the performance of this measure. A synthetic radiograph was generated first from a preoperative pelvic CT scan to act as the gold standard. The voxel weights used to generate the image were then modified and new images were generated with the CT rigidly transformed. The roll, pitch and yaw angles span a range of -10/+10 degrees, while x, y and z translations range from -10mm to +10mm. These images were compared with the reference image. The proposed cost function correctly identified the correct pose in all tests and did not exhibit any local maxima which would slow or prevent locating the global maximum.

Volume registration is fundamental to multiple medical imaging algorithms. Specifically, non-rigid registration of thoracic CT scans taken at different time instances can be used to detect new nodules more reliably and assess the growth rate of existing nodules. Voxel-based registration techniques are generally sensitive to intensity variation and structural differences, which are common in CT scans due to partial volume effects and naturally occurring motion and deformations. The approach we propose in this paper is based on vessel tree extraction which is then used to infer the complete volume registration. Vessels form unique features with good localization. Using extracted vessel trees, a minimization process is used to estimate the motion vectors at vessels. Accurate motion vectors are obtained at vessel junctions whereas vessel segments support only normal component estimation. The obtained motion vectors are then interpolated to produce a dense motion field using thin plate splines. The proposed approach is evaluated on both real and synthetically deformed volumes. The obtained results are compared to several standard registration techniques. It is shown that by using vessel structure, the proposed approach results in improved performance.

Joint entropy, mutual information, and normalized mutual information are widely used image similarity measures in multimodality image registration and other problems that involve comparing images with arbitrary intensity relationships. While these image similarity measures have been successfully used in various applications, their mathematical properties have not been studied thoroughly. This paper analyzes several properties of practical interest of the three image similarity measures. It is shown that mutual information, despite its popularity, and joint entropy have a few undesirable properties. On the other hand, normalized mutual information does not suffer from these problems. The properties are proven mathematically, which renders the conclusions independent of image type, noise, and artifacts. The conclusions are in line with the results of previous experimental studies, in which normalized mutual information outperformed other information theoretic image similarity measures.

The visualization and comparison of local deformation from 3D image sequences is of critical importance in understanding the etiology of Ischemic cardiac disease. In this paper we describe a framework to combine our previous fast spherical harmonic surface alignment algorithm with a new local special surface reconstruction method to reconstruct the surface of LV with Ischaemic cardiac disease. Our new surface computational model allows people to extract the valuable ischemic tissues behavior from the dynamic shape. We have demonstrated our approaches by the experiments on cardiac MRI. A brief description of motivation is put forth, as well as an overview of the approaches and some initial results are described.

We implemented an iterative nonrigid registration algorithm to accurately combine functional (PET) and anatomical (MRI) images in 3D. Our method relies on a Finite Element Method (FEM) and a set of fiducial skin markers (FSM) placed on breast surface. The method is applicable if the stress conditions in the imaged breast are virtually the same in PET and MRI. In the first phase, the displacement vectors of the corresponding FSM observed in MRI and PET are determined, then FEM is used to distribute FSM displacements linearly over the entire breast volume. Our FEM model relies on the analogy between each of the orthogonal components of displacement field, and the temperature distribution field in a steady state heat transfer (SSHT) in solids. The problem can thus be solved via standard heat-conduction FEM software, with arbitrary conductivity of surface elements set much higher than that of volume elements. After determining the displacements at all mesh nodes, moving (MRI) breast volume is registered to target (PET) breast volume using an image-warping algorithm. In the second iteration, to correct for any residual surface and volume misregistration, a refinement process is applied to the moving image, which was already grossly aligned with the target image in 3D using FSM. To perform this process we determine a number of corresponding points on each moving and target image surfaces using a nearest-point approach. Then, after estimating the displacement vectors between the corresponding points on the surfaces we apply our SSHT model again. We tested our model on twelve patients with suspicious breast lesions. By using lesions visible in both PET and MRI, we established that the target registration error is below two PET voxels. The surface registration error is comparable to the spatial resolution of PET.

In this paper we develop a novel image processing technique to register two dimensional temporal mammograms for effective diagnosis and therapy. Our registration framework is founded upon triangular B-spline finite element method (TBFEM). In contrast to tensor-product B-splines, which is widely used in medical imaging, triangular B-splines are much more powerful, associated with many desirable advantages for image registration, such as flexible triangular domain, local control, space-varying smoothness, and sharp feature modeling. Empowered by the rigorous theory of triangular B-splines, our method can explicitly model the transformation between temporal mammogram pairs over irregular region of interest(ROI), using a collection of triangular B-splines. In addition, it is also capable of describing C⁰ continuous deformation at the interfaces between different elastic tissues, while the overall displacement field is smooth. Our registration process consists of two steps: 1) The template image is first nonlinearly deformed using TBFEM model, subject to pre-segmented feature constraints; 2) The deformed template image is further perturbed by applying pseudo image forces, aiming to reducing intensity-based discrepancies. The proposed registration framework has been tested extensively on practical clinical data, and the experimental results demonstrates that the registration accuracy is improved comparing to using conventional FEMs. Besides, the modeling of local C⁰ continuities of the displacement field helps to further increase the registration quality considerably.

The development of multi-modality image analysis has gained increasing popularity over recent years. Multi-modality image databases are being developed to benefit patient clinical care, research and education. The incorporation of histopathology in these multi-modality datasets is complicated by the large differences in image quality, content and spatial association. We have developed a novel system, the large-scale image microtome array (LIMA), to bridge the gap between non-structurally destructive and destructive imaging such that reliable registration and incorporation of three-dimensional (3D) histopathology can be achieved. We have developed registration algorithms to align the micro-CT, LIMA and histopathology data to a common coordinate system. Using this multi-modality image dataset we have developed a classification algorithm to identify on a pixel basis, the tissue types present. The output from the classification processing is a 3D color coded map of tissue distributions. The resulting complete dataset provides an abundance of valuable information relating to the tissue sample including density, anatomical structure, color, texture and cellular information in three dimensions. In this study we have chosen to use normal and diseased lung tissue, however the flexibility of the image acquisition and subsequent processing algorithms makes it applicable to any soft organ tissue.

A hybrid, dual modality single photon emission computed tomography (SPECT) and x-ray computed mammotomography (CmT) scanner for dedicated breast and axillary imaging is under development. CmT imaging provides high resolution anatomical images, whereas SPECT provides functional images albeit with coarser resolution. As is being seen clinically in whole body imaging, integration of the images is expected to enhance (visually) and improve (with attenuation correction of SPECT) information provided by either modality for the detection, characterization and potentially staging of breast cancer. The registration of these images considers variations in object positions between the different modalities and imaging parameters (pixel size, conditions of acquisition, scan limitations). Automatic methods can be used which find the geometric transformations of the different imaging modalities involved. Here we demonstrate the initial stages of iterative 2-dimensional registration and fusion of SPECT with parallel beam geometry and CmT with offset cone-beam acquisition geometry for mammotomography with images acquired and reconstructed independently on each system. Two registration algorithms are considered: the first is an intrinsic correlation, Mutual Information (MI) method based on intrinsic image content; the second is a rigid body transform method, Iterative Closest Point (ICP) method based on identification of fiducial markers visible to both emission (SPECT) and transmission (CmT) imaging modalities. Experiments include use of a geometric resolution/frequency phantom imaged under different conditions, and two different anthropomorphic breast phantom sizes (325 and 935mL). Initial results with the geometric phantom demonstrate that MI can be misled by highly symmetric features, and ICP using control points is more accurate to within fractions of a voxel. Initial breast phantom studies indicate that object size and SPECT resolution limitations may contribute to registration errors.

The objectives of this investigation are to improve quality of subtraction MR breast images and improve accuracy of time-signal intensity curves (TSIC) related to local contrast-agent concentration in dynamic MR mammography. The patients, with up to nine fiducial skin markers (FSMs) taped to each breast, were prone with both breasts suspended into a single well that housed the receiver coil. After a preliminary scan, paramagnetic contrast agent gadopentate digmeglumine (Gd) was delivered intravenously, followed by physiological saline. The field of view was centered over the breasts. We used a gradient recalled echo (GRE) technique for pre-Gd baseline, and five more measurements at 60s intervals. Centroids were determined for corresponding FSMs visible on pre-Gd and any post-Gd images. This was followed by segmentation of breast surfaces in all dynamic-series images, and meshing of all post-Gd breast images. Tetrahedral volume and triangular surface elements were used to construct a finite element method (FEM) model. We used ANSYS^TM software and an analogy between orthogonal components of the displacement field and the temperature differences in steady-state heat transfer (SSHT) in solids. The floating images were warped to a fixed image using an appropriate shape function for interpolation from mesh nodes to voxels. To reduce any residual misregistration, we performed surface matching between the previously warped floating image and the target image. Our method of motion correction via nonrigid coregistration yielded excellent differential-image series that clearly revealed lesions not visible in unregistered differential-image series. Further, it produced clinically useful maximum intensity projection (MIP) 3D images.

Diagnostic thoracic procedures using computed tomography (CT) often include comparisons of scans acquired with different slice thicknesses. In this manuscript, we investigated the potential for alignment of different CT scans from the same patient using skeletal knowledge of the thoracic region. Skeletal matching was selected because it is expected to be less susceptible to differences associated with patient breath hold, positioning and cardiac motion. Our method utilized the positioning of the ribs relative to the vertebra for matching. It also included matching the scapula when visible in the scans. Rib positioning was described by the angles formed between the vertebra centroid and combinations of pairs of rib centroids visible on each CT slice; this was used as the primary matching mechanism. Scapula morphology was described using a feature based on the local maxima of the distance transform. Since the scapula is not visible in all slices of a full scan, its description was limited to only defining the potential range of slices. A cost function incorporating the difference of features from rib positioning and scapula morphology between two slices was derived and used to match slices. The method was evaluated on an independent set of 10 pairs of full and partial CT scans. Assessment was based on whether or not slices containing known nodules between each pair of scans were overlapping after the alignment procedure. Results showed that the proposed metric correctly aligned 9 out of 10 scans. The preliminary results are encouraging for using this method as a first step towards temporal analysis of lung nodules.

We recently studied bone loss in crewmembers making 4 to 6 months flights on the International Space Station. We employed Quantitative Computed Tomography (QCT) technology (Lang et. al., J Bone Miner Res. 2004; v. 19, p. 1006), which made measurements of both cortical and trabecular bone loss that could not be obtained by using 2-dimensional dual x-ray absorptiometry (DXA) imaging technology. To further investigate the bone loss after spaceflight, we have developed image registration technologies to align serial scans so that bone changes can be directly visualized in a subregional level, which can provide more detailed information for understanding bone physiology during long-term spaceflight. To achieve effective and robust registration when large bone changes exist, we have developed technical adaptations to standard registration methods. Our automated image registration is mutual-information based. We have applied an automatically adaptive binning method in calculating the mutual information. After the pre- and post-flight scans are geometrically aligned, the interior bone changes can be clearly visualized. Image registration can also be applied to Finite Element Modeling (FEM) to compare bone strength change, where consistent loading conditions must be applied to serial scans.

We propose a methodology for making optimal registration decisions during 3D volume reconstruction in terms of (a) anticipated accuracy of aligned images, (b) uncertainty of obtained results during the registration process, (c) algorithmic repeatability of alignment procedure, and (d) computational requirements. We researched and developed a web-enabled, web services based, data-driven, registration decision support system. The registration decisions include (1) image spatial size (image sub-area or entire image), (2) transformation model (e.g., rigid, affine or elastic), (3) invariant registration feature (intensity, morphology or a sequential combination of the two), (4) automation level (manual, semi-automated, or fully-automated), (5) evaluations of registration results (multiple metrics and methods for establishing ground truth), and (6) assessment of resources (computational resources and human expertise, geographically local or distributed). Our goal is to provide mechanisms for evaluating the tradeoffs of each registration decision in terms of the aforementioned impacts. First, we present a medical registration methodology for making registration decisions that lead to registration results with well-understood accuracy, uncertainty, consistency and computational complexity characteristics. Second, we have built software tools that enable geographically distributed researchers to optimize their data-driven registration decisions by using web services and supercomputing resources. The support developed for registration decisions about 3D volume reconstruction is available to the general community with the access to the NCSA supercomputing resources. We illustrate performance by considering 3D volume reconstruction of blood vessels in histological sections of uveal melanoma from serial fluorescent labeled paraffin sections labeled with antibodies to CD34 and laminin. The specimens are studied by fluorescence confocal laser scanning microscopy (CLSM) images.

We have been developing a computer-aided diagnosis (CAD) system for automatically recognizing cervical cancer cells from Papanicolaou smear. Considering that pathological changes of cervix can be indicated by the abnormity of the nucleus of intermediate cell, the key task of this system is to find the intermediate cells and segment the nucleus precisely. This paper presents a novel approach for automatic segmentation of microscopic cervical cell images using multispectral imaging techniques. In order to capture images at different wavelengths, a Liquid Crystal Tunable Filter (LCTF) device is used to provide wavelength selection from 400nm to 720nm with an increment of 10nm. Considering the spectral variances of background, nucleus and cytoplasm, background is extracted firstly from the microscopic images by calculating pixel intensity variance at 470nm, 530nm, 570nm, 580nm and 650nm. Then superficial cells are extracted apart from intermediate cells easily at 530nm 650nm because of the different pixel intensity distribution of the two kinds of cells at these two wavelengths. To segment the nucleus from intermediate cells, we adopt two procedures. Firstly, the nuclei are roughly segmented apart by using an iterative maximum deviation between-cluster algorithm. Secondly, a novel rigorous algorithm based on active contour model is adopted to achieve more exact nuclei segmentation. Using the method proposed in this paper, we did experiments on over 300 cervical smears, and the results show that this method is more robust and precise.

The application of computer based image analysis to the diagnosis of retinal disease is rapidly becoming a reality due to the broad-based acceptance of electronic imaging devices throughout the medical community and through the collection and accumulation of large patient histories in picture archiving and communications systems. Advances in the imaging of ocular anatomy and pathology can now provide data to diagnose and quantify specific diseases such as diabetic retinopathy (DR). Visual disability and blindness have a profound socioeconomic impact upon the diabetic population and DR is the leading cause of new blindness in working-age adults in the industrialized world. To reduce the impact of diabetes on vision loss, robust automation is required to achieve productive computer-based screening of large at-risk populations at lower cost. Through this research we are developing automation methods for locating and characterizing important structures in the human retina such as the vascular arcades, optic nerve, macula, and lesions. In this paper we present results for the automatic detection of the optic nerve using digital red-free fundus photography. Our method relies on the accurate segmentation of the vasculature of the retina along with spatial probability distributions describing the luminance across the retina and the density, average thickness, and average orientation of the vasculature in relation to the position of the optic nerve. With these features and other prior knowledge, we predict the location of the optic nerve in the retina using a two-class, Bayesian classifier. We report 81% detection performance on a broad range of red-free fundus images representing a population of over 345 patients with 19 different pathologies associated with DR.

To achieve sufficient accuracy and robustness, 2D/3D registration methods between DSA and MRA of the cerebral artery require an automatic extraction method that can isolate wanted segments from the cerebral artery tree. Here, we described an automatic segmentation method that divides the cerebral artery tree in time-of-flight magnetic resonance angiography (TOF-MRA) into each artery. This method requires a 3D dataset of the cerebral artery tree obtained by TOF-MRA. The processes of this method are: 1) every branch in the cerebral artery tree is labeled with a unique index number, 2) the 3D center of the Circle of Willis is determined using 2D and 3D templates, and 3) the labeled branches are classified with reference to the 3D territory map of cerebral arteries centered on the Circle of Willis. This method classifies all branches into internal carotid arteries (ICA), basilar artery (BA), middle cerebral artery (MCA), a1 segment of anterior cerebral artery (ACA(A1)), other segments of the anterior cerebral artery (ACA), posterior communication artery (PcomA), and posterior cerebral artery (PCA). In the eleven cases examined, the numbers of correctly segmented pixels in each branch were counted and the percentages based on the total number of pixels of the artery were calculated. Manually classified arteries of each case were used as references. Mean percentages were: ACA, 87.6%; R-ACA(A1), 44.9%; L-ACA(A1), 30.4%; R-MC, 82.4%; L-MC, 79.0%; R-PcomA, 0.5%; L-PcomA, 0.0%; R-PCA, 77.2%; L-PCA, 80.0%; R-ICA, 78.6%; L-ICA, 93.05; BA, 77.1%; and total arteries, 78.9%.

A scaling index based segmentation (SIBS) method is proposed in order to improve visualization and interpretation of data obtained by electron tomography. Based on the interpretation of the scaling index as a measure for dimensionality, the pixels/voxels of an image/volume are subdivided into different categories according to the kind of structure they belong to. Using the weighted scaling index method proposed by Räth¹ in conjunction with morphological operators, the approach was adapted to the field of electron microscopy, especially to three-dimensional application as needed by electron tomography. The method turns out to be quite effective for linear structures and membranes. Theory, implementation, parameter settings and results obtained with different kinds of data are presented and discussed.

Manual segmentation of medical images is unpractical because it is time consuming, not reproducible, and prone to human error. It is also very difficult to take into account the 3D nature of the images. Thus, semi- or fully-automatic methods are of great interest. Current segmentation algorithms based on an Expectation- Maximization (EM) procedure present some limitations. The algorithm by Ashburner et al., 2005, does not allow multichannel inputs, e.g. two MR images of different contrast, and does not use spatial constraints between adjacent voxels, e.g. Markov random field (MRF) constraints. The solution of Van Leemput et al., 1999, employs a simplified model (mixture coefficients are not estimated and only one Gaussian is used by tissue class, with three for the image background). We have thus implemented an algorithm that combines the features of these two approaches: multichannel inputs, intensity bias correction, multi-Gaussian histogram model, and Markov random field (MRF) constraints. Our proposed method classifies tissues in three iterative main stages by way of a Generalized-EM (GEM) algorithm: (1) estimation of the Gaussian parameters modeling the histogram of the images, (2) correction of image intensity non-uniformity, and (3) modification of prior classification knowledge by MRF techniques. The goal of the GEM algorithm is to maximize the log-likelihood across the classes and voxels. Our segmentation algorithm was validated on synthetic data (with the Dice metric criterion) and real data (by a neurosurgeon) and compared to the original algorithms by Ashburner et al. and Van Leemput et al. Our combined approach leads to more robust and accurate segmentation.

The recent development of real-time 3D ultrasound enables intracardiac beating heart procedures, but the distorted appearance of surgical instruments is a major challenge to surgeons. In addition, tissue and instruments have similar gray levels in US images and the interface between instruments and tissue is poorly defined. We present an algorithm that automatically estimates instrument location in intracardiac procedures. Expert-segmented images are used to initialize the statistical distributions of blood, tissue and instruments. Voxels are labeled of voxels through an iterative expectation-maximization algorithm using information from the neighboring voxels through a smoothing kernel. Once the three classes of voxels are separated, additional neighboring information is used to give spatial information based on the shape of instruments in order to correct for misclassifications. We analyze the major axis of segmented data through their principal components and refine the results by a watershed transform, which corrects the results at the contact between instrument and tissue. We present results on 3D in-vitro data from a tank trial, and 3D in-vivo data from a cardiac intervention on a porcine beating heart. The comparison of algorithm results to expert-annotated images shows the correct segmentation and position of the instrument shaft.

In Radio Frequency Ablation (RFA) procedures, hepatic tumor tissue is heated to a temperature where necrosis is insured. Unfortunately, recent results suggest that heating tumor tissue to necrosis is complicated because nearby major blood vessels provide a cooling effect. Therefore, it is fundamentally important for physicians to perform a careful analysis of the spatial relationship of diseased tissue to larger liver blood vessels. The liver contains many of these large vessels, which affect the RFA ablation shape and size. There are many sophisticated vasculature detection and segmentation techniques reported in the literature that identify continuous vessels as the diameter changes size and it transgresses through many bifurcation levels. However, the larger blood vessels near the treatment area are the only vessels required for proper RFA treatment plan formulation and analysis. With physician guidance and interaction, our system can segment those vessels which are most likely to affect the RFA ablations. We have found that our system provides the physician with therapeutic, geometric and spatial information necessary to accurately plan treatment of tumors near large blood vessels. The segmented liver vessels near the treatment region are also necessary for computing isolevel heating profiles used to evaluate different proposed treatment configurations.

Conventional k-Nearest-Neighbor (kNN) classification, which has been successfully applied to classify brain tissue, requires laborious training on manually labeled subjects. In this work, the performance of kNN-based segmentation of gray matter (GM), white matter (WM) and cerebrospinal fluid (CSF) using manual training is compared with a new method, in which training is automated using an atlas. From 12 subjects, standard T2 and PD scans and a high-resolution, high-contrast scan (Siemens T1-weighted HASTE sequence with reverse contrast) were used as feature sets. For the conventional kNN method, manual segmentations were used for training, and classifications were evaluated in a leave-one-out study. The performance as a function of the number of samples per tissue, and k was studied. For fully automated training, scans were registered to a probabilistic brain atlas. Initial training samples were randomly selected per tissue based on a threshold on the tissue probability. These initials were processed to keep the most reliable samples. Performance of the method for varying the threshold on the tissue probability method was studied. By measuring the percentage overlap (SI), classification results of both methods were validated. For conventional kNN classification, varying the number of training samples did not result in significant differences, while increasing k gave significantly better results. In the method using automated training, there is an overestimation of GM at the expense of CSF at higher thresholds on the tissue probability maps. The difference between the conventional method (k=45) and the observers was not significantly larger than inter-observer variability for all tissue types. The automated method performed slightly worse and performed equal to the observers for WM, and less for CSF and GM. From these results it can be concluded that conventional kNN classification may replace manual segmentation, and that atlas-based kNN segmentation has strong potential for fully automated segmentation, without the need of laborious manual training.

We present a fully-automated and robust microarray image analysis system for handling multi-resolution images (down to 3-micron with sizes up to 80 MBs per channel). The system is developed to provide rapid and accurate data extraction for our recently developed microarray analysis and quality control tool (SNP Chart). Currently available commercial microarray image analysis applications are inefficient, due to the considerable user interaction typically required. Four-channel DNA microarray technology is a robust and accurate tool for determining genotypes of multiple genetic markers in individuals. It plays an important role in the state of the art trend where traditional medical treatments are to be replaced by personalized genetic medicine, i.e. individualized therapy based on the patient's genetic heritage. However, fast, robust, and precise image processing tools are required for the prospective practical use of microarray-based genetic testing for predicting disease susceptibilities and drug effects in clinical practice, which require a turn-around timeline compatible with clinical decision-making. In this paper we have developed a fully-automated image analysis platform for the rapid investigation of hundreds of genetic variations across multiple genes. Validation tests indicate very high accuracy levels for genotyping results. Our method achieves a significant reduction in analysis time, from several hours to just a few minutes, and is completely automated requiring no manual interaction or guidance.

During any image segmentation process, two distinct tasks are performed - recognition and delineation. Recognition consists of the searching phase which roughly identifies a particular object of interest among other neighboring structures present in the image. Delineation consists of precisely defining the spatial extent of the object region. Well designed interactive segmentation methods, such as live wire (LW) and snakes, exploit the synergy between the user knowledge (for recognition) and the underlying computer processing done automatically (for delineation). We present in this paper two new methods, referred to as iterative live wire and live snake, for interactive 3D segmentation of medical images. In both methods, the segmentation initiated by the LW or snake method is propagated under user control to subsequent slices by projecting the anchor points. In iterative LW (ILW), the LW segments are iteratively updated in the new slice by selecting the mid points of previous LW segments as new anchor points. In live snake (LS), the snake method is first applied in the new slice for the projected anchor points and ended with an application of ILW. The methods have been evaluated on 30 3D MRI data sets of the breast. The results indicate that, on average, far fewer number of user interventions during segmentation and anchor point specification are needed by using the new methods than by using snakes propagation or live wire. The ILW segmentations are slightly more accurate, with statistical significance (P<0.01), than LS segmentations, and the former are more efficient than the latter (P<0.03), both being more efficient than pure live wire and snake methods.

Functional assessment of cardiac ventricular function requires time consuming manual interaction. Some automated methods have been presented that predominantly used cardiac magnet resonance images. Here, an automatic shape tracking approach is followed to estimate left ventricular blood volume from multi-slice computed tomography image series acquired with retrospective ECG-gating. A deformable surface model method was chosen that utilized both shape and local appearance priors to determine the endocardial surface and to follow its motion through the cardiac cycle. Functional parameters like the ejection fraction could be calculated from the estimated shape deformation. A clinical validation was performed in a porcine model with 60 examinations on eight subjects. The functional parameters showed a good correlation with those determined by clinical experts using a commercially available semi-automatic short axes delineation tool. The correlation coefficient for the ejection fraction (EF) was 0.89. One quarter of these acquisitions were done with a low dose protocol. All of these degraded images could be processed well. Their correlation slightly decreases when compared to the normal dose cases (EF: 0.87 versus 0.88).

This paper introduces image processing methods to automatically detect the 3D volume-of-interest (VOI) and 2D region-of-interest (ROI) for deep gray matter organs (thalamus, globus pallidus, putamen, and caudate nucleus) of patients with suspected iron deposition from MR dual echo images. Prior to the VOI and ROI detection, cerebrospinal fluid (CSF) region is segmented by a clustering algorithm. For the segmentation, we automatically determine the cluster centers with the mean shift algorithm that can quickly identify the modes of a distribution. After the identification of the modes, we employ the K-Harmonic means clustering algorithm to segment the volumetric MR data into CSF and non-CSF. Having the CSF mask and observing that the frontal lobe of the lateral ventricle has more consistent shape accross age and pathological abnormalities, we propose a shape-directed landmark detection algorithm to detect the VOI in a speedy manner. The proposed landmark detection algorithm utilizes a novel shape model of the front lobe of the lateral ventricle for the slices where thalamus, globus pallidus, putamen, and caudate nucleus are expected to appear. After this step, for each slice in the VOI, we use horizontal and vertical projections of the CSF map to detect the approximate locations of the relevant organs to define the ROI. We demonstrate the robustness of the proposed VOI and ROI localization algorithms to pathologies, including severe amounts of iron accumulation as well as white matter lesions, and anatomical variations. The proposed algorithms achieved very high detection accuracy, 100% in the VOI detection , over a large set of a challenging MR dataset.

Automated methods to delineate brain structures of interest are required to analyze large amounts of imaging data like that being collected in several on going multi-center studies. We have previously reported on using artificial neural networks (ANN) to define subcortical brain structures such as the thalamus (0.825), caudate (0.745), and putamen (0.755). One of the inputs into the ANN is the apriori probability of a structure existing at a given location. In this previous work, the apriori probability information was generated in Talairach space using a piecewise linear registration. In this work we have increased the dimensionality of this registration using Thirion's demons registration algorithm. The input vector consisted of apriori probability, spherical coordinates, and an iris of surrounding signal intensity values. The output of the neural network determined if the voxel was defined as one of the N regions used for training. Training was performed using a standard back propagation algorithm. The ANN was trained on a set of 15 images for 750,000,000 iterations. The resulting ANN weights were then applied to 6 test images not part of the training set. Relative overlap calculated for each structure was 0.875 for the thalamus, 0.845 for the caudate, and 0.814 for the putamen. With the modifications on the neural net algorithm and the use of multi-dimensional registration, we found substantial improvement in the automated segmentation method. The resulting segmented structures are as reliable as manual raters and the output of the neural network can be used without additional rater intervention.

An efficient way to improve the robustness of the segmentation of medical images with deformable models is to use a priori shape knowledge during the adaptation process. In this work, we investigate how the modeling of the shape variability in shape-constrained deformable models influences both the robustness and the accuracy of the segmentation of cardiac multi-slice CT images. Experiments are performed for a complex heart model, which comprises 7 anatomical parts, namely the four chambers, the myocardium, and trunks of the aorta and the pulmonary artery. In particular, we compare a common shape variability modeling technique based on principal component analysis (PCA) with a more simple approach, which consists of assigning an individual affine transformation to each anatomical subregion of the heart model. We conclude that the piecewise affine modeling leads to the smallest segmentation error, while simultaneously offering the largest flexibility without the need for training data covering the range of possible shape variability, as required by PCA.

Automatic segmentation of skin lesions in clinical images is a very challenging task; it is necessary for visual analysis of the edges, shape and colors of the lesions to support the melanoma diagnosis, but, at the same time, it is cumbersome since lesions (both naevi and melanomas) do not have regular shape, uniform color, or univocal structure. Most of the approaches adopt unsupervised color clustering. This works compares the most spread color clustering algorithms, namely median cut, k-means, fuzzy-c means and mean shift applied to a method for automatic border extraction, providing an evaluation of the upper bound in accuracy that can be reached with these approaches. Different tests have been performed to examine the influence of the choice of the parameter settings with respect to the performances of the algorithms. Then a new supervised learning phase is proposed to select the best number of clusters and to segment the lesion automatically. Examples have been carried out in a large database of medical images, manually segmented by dermatologists. From these experiments mean shift was resulted the best technique, in term of sensitivity and specificity. Finally, a qualitative evaluation of the goodness of segmentation has been validated by the human experts too, confirming the results of the quantitative comparison.

To facilitate high level analysis of medical image data in research and clinical environments, a wrapper for the ITK toolkit is developed to allow ITK algorithms to be called in MATLAB. ITK is a powerful open-source toolkit implementing state-of-the-art algorithms in medical image processing and analysis. However, although ITK is rapidly gaining popularity, its user base is mostly restricted to technically savvy developers with expert knowledge of C++ and advanced programming concepts. MATLAB, on the other hand, is well-known for its easy-to-use, powerful prototyping capabilities that significantly improve productivity. Unfortunately, the 3D image processing capabilities of MATLAB are very limited and slow to execute. With the help of the wrapper we introduce in this paper, biomedical computing researchers familiar with MATLAB can harness the power of ITK while avoiding learning C++ and dealing with low-level programming issues. We strongly believe this functionality will be of considerable interest to the medical image computing community. In this paper we provide details about the design and usage of this interface in medical image filtering, segmentation, and registration.

This article describes a method for automatic segmentation of the abdomen into three anatomical regions: subcutaneous, retroperitoneal and visceral. For the last two regions the amount of adipose tissue (fat) is quantified. According to recent medical research, the distinction between retroperitoneal and visceral fat is important for studying metabolic syndrome, which is closely related to diabetes. However previous work has neglected to address this point, treating the two types of fat together. We use T1-weighted three-dimensional magnetic resonance data of the abdomen of obese minipigs. The pigs were manually dissected right after the scan, to produce the "ground truth" segmentation. We perform automatic segmentation on a representative slice, which on humans has been shown to correlate with the amount of adipose tissue in the abdomen. The process of automatic fat estimation consists of three steps. First, the subcutaneous fat is removed with a modified active contour approach. The energy formulation of the active contour exploits the homogeneous nature of the subcutaneous fat and the smoothness of the boundary. Subsequently the retroperitoneal fat located around the abdominal cavity is separated from the visceral fat. For this, we formulate a cost function on a contour, based on intensities, edges, distance to center and smoothness, so as to exploit the properties of the retroperitoneal fat. We then globally optimize this function using dynamic programming. Finally, the fat content of the retroperitoneal and visceral regions is quantified based on a fuzzy c-means clustering of the intensities within the segmented regions. The segmentation proved satisfactory by visual inspection, and closely correlated with the manual dissection data. The correlation was 0.89 for the retroperitoneal fat, and 0.74 for the visceral fat.

Alzheimer's disease (AD) is associated with the degeneration of cerebral cortex, which results in focal volume change or thinning in the cerebral cortex in magnetic resonance imaging (MRI). Therefore, the measurement of the cortical thickness is important for detection of the atrophy related to AD. Our purpose was to develop a computerized method for automated measurement of the cortical thickness for three-dimensional (3-D) MRI. The cortical thickness was measured with normal vectors from white matter surface to cortical gray matter surface on a voxel-by-voxel basis. First, a head region was segmented by use of an automatic thresholding technique, and then the head region was separated into the cranium region and brain region by means of a multiple gray level thresholding with monitoring the ratio of the first maximum volume to the second one. Next, a fine white matter region was determined based on a level set method as a seed region of the rough white matter region extracted from the brain region. Finally, the cortical thickness was measured by extending normal vectors from the white matter surface to gray matter surface (brain surface) on a voxel-by-voxel basis. We applied the computerized method to high-resolution 3-D T1-weighted images of the whole brains from 7 clinically diagnosed AD patients and 8 healthy subjects. The average cortical thicknesses in the upper slices for AD patients were thinner than those for non-AD subjects, whereas the average cortical thicknesses in the lower slices for most AD patients were slightly thinner. Our preliminary results suggest that the MRI-based computerized measurement of gray matter atrophy is promising for detecting AD.

In this paper, we present a two-stage method for the segmentation of breast mass lesions on digitized mammograms. A radial gradient index (RGI) based segmentation method is first used to estimate a initial contour close to the lesion boundary location in a computationally efficient manner. Then a region-based active contour algorithm, which minimizes an energy fucntion based on the homogeneities inside and outside of the envolving coutour, is applied to refine the contour closer to the lesion boundary. The minimization algorithm solves, by the level set method, the Euler-Lagrange equation that describes the contour evolution. By using a digitized screening film dababase with 96 biopy-proven, malignant lesions, we quantitatively compare this two-stage segmentation algorithm with a RGI-based method and a conventional region-growing algorithm by measuring the area similarity. At an overlap threshold of 0.30, the new method correctly segments 95% of the lesions while the prior methods delineate only 83% of the lesions. Our assessment demonstrates that the two-stage segmentation algorithm yields closer agreement with manually contoured lesion boundaries.

In this paper we describe a method for assisting radiological technologists in their routine work to automatically determine the imaging plane in lumbar MRI. The method is first to recognize the spinal cord and the intervertebral disk (ID) from the lumbar vertebra 3-plane localizer image, and then the imaging plane is automatically determined according to the recognition results. To determine the imaging plane, the spinal cord and the ID are automatically recognized from the lumbar vertebra 3-plane localizer image with a series of image processing techniques. The proposed method consists of three major steps. First, after removing the air and fat regions from the 3-plane localizer image by use of histogram analysis, the rachis region is specified with Sobel edge detection filter. Second, the spinal cord and the ID were respectively extracted from the specified rachis region making use of global thresholding and the line detection filter. Finally, the imaging plane is determined by finding the straight line between the spinal cord and the ID with the Hough transform. Image data of 10 healthy volunteers were used for investigation. To validate the usefulness of our proposed method, manual determination of the imaging plane was also conducted by five experienced radiological technologists. Our experimental results showed that the concordance rate between the manual setting and automatic determination reached to 90%. Moreover, a remarkable reduction in execution time for imaging-plane determination was also achieved.

Diffusion tensor (DT) MRI provides the directional information of water molecular diffusion, which can be utilized to estimate the connectivity of white matter tract pathways in the human brain. Several white matter tractography methods have been developed to reconstruct the white matter fiber tracts using DT-MRI. With conventional methods (e.g., streamline techniques), however, it would be very difficult to trace the white matter tracts passing through the fiber crossing and branching regions due to the ambiguous directional information with the partial volume effect. The purpose of this study was to develop a new white matter tractography method which permits fiber tract branching and passing through crossing regions. Our tractography method is based on a three-dimensional (3D) directional diffusion function (DDF), which was defined by three eigenvalues and their corresponding eigenvectors of DT in each voxel. The DDF-based tractography (DDFT) consists of the segmentation of white matter tract region and fiber tracking process. The white matter tract regions were segmented by thresholding the 3D directional diffusion field, which was generated by the DDF. In fiber tracking, the DDFT method estimated the local tract direction based on overlap of the DDFs instead of the principal eigenvector, which has been used in conventional methods, and reconstructed tract branching by means of a one-to-many relation model. To investigate the feasibility and usefulness of the DDFT method, we applied it to DT-MRI data of five normal subjects and seven patients with a brain tumor. With the DDFT method, the detailed anatomy of white matter tracts was depicted more appropriately than the conventional methods.

Dynamic Single Photon Emission Computed Tomography (SPECT) has the potential to quantitatively estimate physiological parameters by fitting compartment models to the tracer kinetics. The generalized linear least square method (GLLS) is an efficient method to estimate unbiased kinetic parameters and parametric images. However, due to the low sensitivity of SPECT, noisy data can cause voxel-wise parameter estimation by GLLS to fail. Fuzzy C-Mean (FCM) clustering and modified FCM, which also utilizes information from the immediate neighboring voxels, are proposed to improve the voxel-wise parameter estimation of GLLS. Monte Carlo simulations were performed to generate dynamic SPECT data with different noise levels and processed by general and modified FCM clustering. Parametric images were estimated by Logan and Yokoi graphical analysis and GLLS. The influx rate (K₁), volume of distribution (V_d) were estimated for the cerebellum, thalamus and frontal cortex. Our results show that (1) FCM reduces the bias and improves the reliability of parameter estimates for noisy data, (2) GLLS provides estimates of micro parameters (K₁-k₄) as well as macro parameters, such as volume of distribution (Vd) and binding potential (BP₁ & BP₂) and (3) FCM clustering incorporating neighboring voxel information does not improve the parameter estimates, but improves noise in the parametric images. These findings indicated that it is desirable for pre-segmentation with traditional FCM clustering to generate voxel-wise parametric images with GLLS from dynamic SPECT data.

Recently ground glass opacities (GGOs) have become noteworthy in lung cancer diagnosis. It is crucial to define the boundary a GGO accurately and consistently, since the growth rate is the most manifest evidence of its malignancy. The indefinite and irregular boundary of a GGO makes deformable models adequate for its segmentation. Among deformable models a level set method has the ability to handle topological changes. For the exact estimation of GGO's volume change, the pulmonary airways inside GGO should be excluded in its volume estimation, which necessitate the segmentation into more regions than two of the object and the background. Hence, we adopted a multi-phase deformable model of two level set functions and modified its energy functional into an asymmetric form. The main two modifications are the elimination of one region in four regions of the conventional 4-phase deformable model and the prevention of the outer region from spreading out of the initialization. The proposed model segments the input image into three regions of the inner and outer regions, and the background. The GGO tissues are segmented as the inner region and the outer region plays the role of blockade for the inner region not to leak out to adjacent anatomical structures of similar Hounsfield Unit (HU) values. Our experimental results confirmed the feasibility of the proposed method as a pre-processing step for three dimensional (3-D) volume measurement of the GGO.

Osteo-Arthritis (OA) is a very common age-related cause of pain and reduced range of motion. A central effect of OA is wear-down of the articular cartilage that otherwise ensures smooth joint motion. Quantification of the cartilage breakdown is central in monitoring disease progression and therefore cartilage segmentation is required. Recent advances allow automatic cartilage segmentation with high accuracy in most cases. However, the automatic methods still fail in some problematic cases. For clinical studies, even if a few failing cases will be averaged out in the overall results, this reduces the mean accuracy and precision and thereby necessitates larger/longer studies. Since the severe OA cases are often most problematic for the automatic methods, there is even a risk that the quantification will introduce a bias in the results. Therefore, interactive inspection and correction of these problematic cases is desirable. For diagnosis on individuals, this is even more crucial since the diagnosis will otherwise simply fail. We introduce and evaluate a semi-automatic cartilage segmentation method combining an automatic pre-segmentation with an interactive step that allows inspection and correction. The automatic step consists of voxel classification based on supervised learning. The interactive step combines a watershed transformation of the original scan with the posterior probability map from the classification step at sub-voxel precision. We evaluate the method for the task of segmenting the tibial cartilage sheet from low-field magnetic resonance imaging (MRI) of knees. The evaluation shows that the combined method allows accurate and highly reproducible correction of the segmentation of even the worst cases in approximately ten minutes of interaction.

We present a fast and robust approach for automatic centerline extraction of tubular structures. The underlying idea is to cut traditional snakes into a set of shorter, independent segments - so-called snakelets. Following the same variational principles, each snakelet acts locally and extracts a subpart of the overall structure. After a parallel optimization step, outliers are detected and the remaining segments then form an implicit centerline. No manual initialization of the snakelets is necessary, which represents one advantage of the method. Moreover, computational complexity does not directly depend on dataset size, but on the number of snake segments necessary to cover the structure of interest, resulting in short computation times. Lastly, the approach is robust even for very complex datasets such as the small intestine. Our approach was tested on several medical datasets (CT datasets of colon, small bowel, and blood vessels) and yielded smooth, connected centerlines with few or no branches. The computation time needed is less than a minute using standard computing hardware.