KEYWORDS: 3D modeling, 3D image processing, Shape analysis, Image segmentation, Statistical analysis, Statistical modeling, Principal component analysis, Magnetism, Magnetic resonance imaging, 3D metrology
Conventional analysis of cardiac ventricular magnetic resonance images is performed using short axis images and does not guarantee completeness and consistency of the ventricle coverage. In this paper, a four-dimensional (4D, 3D+time) left and right ventricle statistical shape model was generated from the combination of the long axis and short axis images. Iterative mutual intensity registration and interpolation were used to merge the long axis and short axis images into isotropic 4D images and simultaneously correct existing breathing artifact. Distance-based shape interpolation and approximation were used to generate complete ventricle shapes from the long axis and short axis manual segmentations. Landmarks were automatically generated and propagated to 4D data samples using rigid alignment, distance-based merging, and B-spline transform. Principal component analysis (PCA) was used in model creation and analysis. The two strongest modes of the shape model captured the most important shape feature of Tetralogy of Fallot (TOF) patients, right ventricle enlargement. Classification of cardiac images into classes of normal and TOF subjects performed on 3D and 4D models showed 100% classification correctness rates for both normal and TOF subjects using k-Nearest Neighbor (k=1 or 3) classifier and the two strongest shape modes.
We previously reported on 2D and 3D Active Appearance Models (AAM) for automated segmentation of cardiac MR. AAMs are shown useful for such segmentations because they exploit prior knowledge about cardiac shape and image appearance, yet segmentation of object borders might not be the only benefit of AAMs. An AAM represents objects as a linear combination of shape and texture variations applied to a mean object via Principal Component Analysis (PCA) to form an integrated model. This model captures enough shape, texture, and motion variations to accurately synthesize reconstructions of target objects from a finite set of parameters. Because of this, we hypothesize that AAM coefficients may be used for the classification of disease abnormalities.
PCA is useful for reducing the dimensionality of vectors, however it does not produce vectors optimal for the separation of classes needed for disease classification. Discriminate analysis such as Linear Discriminate Analysis (LDA) and Kernel Discriminate Analysis (KDA) are dimension reducing techniques with the added benefit of supervised learning for the purpose of classification. Once AAM segmentation is complete, disease probabilities are computed from model coefficients via discriminate analysis. Preliminary results on model coefficients show tendency of disease separation for certain disease classes.
Principal Component Analysis of sets of temporal shape sequences renders eigenvariations of shape/motion, including typical normal and pathological endocardial contraction patterns. A previously developed Active Appearance Model for time sequences (AAMM) was employed to derive AAMM shape coefficients (ASCs) and we hypothesized these would allow classification of wall motion abnormalities (WMA). A set of stress echocardiograms (single-beat 4-chamber and 2-chamber sequences with expert-verified endocardial contours) of 129 infarct patients was split randomly into training (n=65) and testing (n=64) sets. AAMMs were generated from the training set and for all sequences ASCs were extracted and statistically related to regional/global Visual Wall Motion Scoring (VWMS) and clinical infarct severity and volumetric parameters. Linear regression showed clear correlations between ASCs and VWMS. Infarct severity measures correlated poorly to both ASCs and VWMS. Discriminant analysis showed good prediction from low #ASCs of both segmental (85% correctness) and global WMA (90% correctness). Volumetric parameters correlated poorly to regional VWMS. Conclusions: 1)ASCs show promising accuracy for automated WMA classification. 2)VWMS and endocardial border motion are closely related; with accurate automated border detection, automated WMA classification should be feasible. 3)ASC shape analysis allows contour set evaluation by direct comparison to clinical parameters.
A novel 3-D Active Appearance Model (3-D AAM) is applied to fully automated endocardial contour detection in 2-D + time (2DT) 4-chamber ultrasound sequences, without knowledge of cardiac phase (ED/ES frames). 2DT appearance of the heart is modeled in 3-D by converting the stack of 2-D time slices into a 3-D voxel space. In a training set, an expert defines corresponding endocardial contour points for one complete cardiac cycle (ED to ED). 2DT shape is represented as a 3-D surface. Image appearance is modeled as a vector of voxel intensities in a volume-patch spanned by the 3-D surface. Principal Component Analysis extracts eigenvariations of 3-D shape and appearance, capturing typical cardiac motion patterns. 3-D AAM segments the image volume by minimizing 3-D model-to-target intensity differences, adjusting eigenvariation coefficients and 3-D pose using gradient descent minimization. This provides time-continuous border localization for one beat in both time and space. The method was used on 3-beat sequences from 129 patients split randomly into a training (65) and a test set (64). An independent expert manually drew all endocardial contours. 3-D AAM converged well in 89% of test cases. Average absolute temporal error was 37.0 msec, spatial error 3.35 mm, comparable to human inter-observer variabilities.
KEYWORDS: 3D modeling, Image segmentation, Data modeling, 3D image processing, Magnetic resonance imaging, Principal component analysis, Shape analysis, 3D acquisition, Data acquisition, Statistical modeling
Active Appearance Models (AAMs) are useful for the segmentation of cardiac MR images since they exploit prior knowledge about the cardiac shape and image appearance. However, traditional AAMs only process 2D images, not taking into account the 3D data inherent to MR. This paper presents a novel, true 3D Active Appearance Model that models the intrinsic 3D shape and image appearance of the left ventricle in cardiac MR data. In 3D-AAM, shape and appearance of the Left Ventricle (LV) is modeled from a set of expert drawn contours. The contours are then resampled to a manually defined set of landmark points, and subsequently aligned. Appearance variations in both shape and texture are captured using Principal Component Analysis (PCA) on the training set. Segmentation is achieved by minimizing the model appearance-to-target differences by adjusting the model eigen-coefficients using a gradient descent approach. The clinical potential of the 3D-AAM is demonstrated in short-axis cardiac magnetic resonance (MR) images. The method's performance was assessed by comparison with manually-identified independent standards in 56 clinical MR sequences. The method showed good agreement with the independent standards using quantitative indices such as border positioning errors, endo- and epicardial volumes, and left ventricular mass. The 3D AAM method shows high promise for successful segmentation of three-dimensional images in MR.
KEYWORDS: 3D modeling, Intravascular ultrasound, Heart, Data modeling, Angiography, 3D image processing, Data fusion, Arteries, Visualization, Image segmentation
Conventional reconstructions from intravascular ultrasound (IVUS) stack the frames as acquired during the pullback of the catheter to form a straight three-dimensional volume, thus neglecting the vessel curvature and merging images from different heart phases. We are developing a comprehensive system for fusion of the IVUS data with the pullback path as determined from x-ray angiography, to create a geometrically accurate 4-D (3-D plus time) model of the coronary vasculature as basis for computational hemodynamics. The overall goal of our work is to correlate shear stress with plaque thickness. The IVUS data are obtained in a single pullback using an automated pullback device; the frames are afterwards assigned to their respective heart phases based upon the ECG signal. A set of 3-D models is reconstructed by fusion of IVUS and angiographic data corresponding to the same ECG-gated heart phase; methods of computational fluid dynamics (CFD) are applied to obtain important hemodynamic data. Combining these models yields the final 4-D reconstruction. Visualization is performed using the platform-independent VRML standard for a user-friendly manipulation of the scene. An extension for virtual angioscopy allows an easy assessment of the vessel features within their local context. Validation was successfully performed both in-vitro and in-vivo.
Active Appearance Models (AAM) are suitable for segmenting 2D images, but for image sequences time-continuous results are desired. Active Appearance-Motion Models (AAMM) model shape and appearance of the heart over the full cardiac cycle. Single-beat sequences are phase-normalized into stacks of 16 2D images. In a training set, corresponding shape points on the endocard are defined for each image based on expert drawn contours. Shape (2D) and intensity vectors are derived similar to AAM. Intensities are normalized non-linearly to handle ultrasound-specific problems. For all time frames, shape vectors are simply concatenated, as well as and intensity vectors. Principal Component Analysis extracts appearance eigenvariations over the cycle, capturing typical motion patterns. AAMMs perform segmentation on complete sequences by minimizing model-to-target differences, adjusting AAMM eigenvariation coefficients using gradient descent minimization. This results in time-continuous segmentation. The method was trained and tested on echocardiographic 4-chamber sequences of 129 unselected patients split randomly into a training set (n=65) and a test set (n=64). In all sequences, an independent expert manually drew endocardial contours. On the test set, fully automated AAMM performed well in 97% of cases (average distance 3.3 mm, 9.3 pixels, comparable to human inter- and intraobserver variabilities).
Active Appearance Models (AAMs) are useful for segmentation of static cardiac MR images since they exploit prior knowledge about the cardiac shape and image appearance. However, applying 2D AAMs to full cardiac cycle segmentation would require multiple models for different phases of the cardiac cycle because traditional AAMs account only for the variations within image classes and not temporal classes. This paper presents a novel 2D+time Active Appearance Motion Model (AAMM) that represents the dynamics of the cardiac cycle in combination with shape and image appearance of the heart, ensuring a time-continuous segmentation of a complete cardiac MR sequence. In AAMM, single-beat sequences are phase-normalized into sets of 2D images and the shape points and gray intensities between frames are concatenated into a shape vector and intensity vector. Appearance variations over time are captured using Principal Component Analysis on both vectors in the training set. Time-continuous segmentation is achieved by minimizing the model appearance-to-target differences by adjusting the model eigen-coefficients using gradient descent approach. In matching tests, the model shows to be robust in initial position and approximates the true segmentation very well. Large-scale clinical validation in patients is ongoing.
We are presenting a comprehensive system for fusion of intravascular ultrasound (IVUS) data and x-ray angiography, aiming to create a geometrically accurate 3-D or 4-D (3-D plus time) model of the coronary vasculature. For hemodynamic analyses, methods of computational fluid dynamics (CFD) are applied to the reconstructed data, resulting in quantitative estimates of the wall shear stress. Visualization is performed using the Virtual Reality Modeling Language (VRML). Lumen and adventitia borders are modeled as surfaces using indexed face sets; quantitative results are encoded as color per vertex. The endoscopic mode (virtual angioscopy) allows an interactive fly-through animation with variable speed along with arbitrary positioning within the vessel. Since this functionality exceeds those of the standard VRML animation nodes, an external prototype library containing VRML and JavaScript definitions has been developed that provides a 3-D graphical user interface to navigate within the endoscopic mode. The control panel is available on demand, but does neither obstruct any vessel features when not needed, nor does it limit the viewport for the scene. Preliminary results showed a good feasibility of the overall procedure, and a high reliability of the fusion and CFD methods as well as the visualization with the virtual endoscopy VRML library.
KEYWORDS: Intravascular ultrasound, Angiography, Image segmentation, Visualization, Heart, Data fusion, Image fusion, 3D image processing, Tissues, In vivo imaging
In the rapidly evolving field of intravascular ultrasound (IVUS) for tissue characterization and visualization, the assessment of vessel morphology still lacks a geometrically correct 3D reconstruction. The IVUS frames are usually stacked up to form a straight vessel, neglecting curvature and the axial twisting of the catheter during the pullback. This paper presents a comprehensive system for geometrically correct reconstruction of IVUS images by fusion with biplane angiography, thus combining the advantages of both modalities. Vessel cross-section and tissue characteristics are obtained form IVUS, while the 3D locations are derived by geometrical reconstruction from the angiographic projections. ECG-based timing ensures a proper match of the image data with the respective heart phase. The fusion is performed for each heart phase individually, thus yielding the 4-D data as a set of 3-D reconstructions.
Active Appearance Models (AAM), which have been recently introduced by Cootes et al., describe the shape of objects and gray level appearance from a set of example images. An AAM is created from user-placed contours defining the shape of objects of interest in each training image. The information about shape changes observed in the training set is used to model the shape variation. Principle component analysis (PCA) is utilized to model gray level variation observed in the training set. The resulting model describes objects as a linear combination of eigen vectors both in shape and gray levels applied to the mean image. The main purpose of this work is to investigate the clinical potential of AAMs for segmentation of cardiovascular MR images acquired in routine clinical practice. An AAM was constructed using 102 end- diastolic short-axis cardiac MR images at the papillary muscle level from normals and patients with varying pathologies. The resulting AAM is a compact representation consisting of a mean image and a limited number of coefficients of eigen vectors, representing 97% of shape and gray level variation observed in the training set. The segmentation performance is tested in 60 end-diastolic short-axis cardiac MR images from different patients.
KEYWORDS: Intravascular ultrasound, Angiography, Visualization, 3D image processing, Image segmentation, Data modeling, Arteries, Image visualization, 3D modeling, Control systems
Medical visualization is a rapidly developing field with many application areas spanning from visualization of anatomy to surgery planning, to understanding of disease processes. With increasing computer speed, medical visualization is becoming more real-time. In this paper, we present a novel application of real-time three-dimensional visualization of coronary arteries during catheter interventions that combines image information from two complementary sources: biplane x-ray contrast angiography and intravascular ultrasound (IVUS). After identification of the three-dimensional characteristics of the intravascular ultrasound pullback sequence, vessel geometry and vessel wall images are combined into a single visualization using semi-automated analysis of a corresponding pair of biplane angiography images. Visualization data are represented using the Virtual Reality Modeling Language (VRML), the code for which is automatically generated by our angiography/IVUS image processing and analysis software system. Selection of the VRML approach facilitates real-time 3-D visualization with an ability of over-the-network image processing and dissemination of results. The visualization specifics are easily modifiable in near real time to consider the immediate requirements of the end-user, the cardiologist who performs the coronary intervention.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.