This article concerns the integration of multimodal volumetric brain images. Integration consists of two steps: matching or registration, where the images are brought into spatial agreement, and fusion or simultaneous display where the registered multimodal image information is presented in an integrated fashion. Approaches to register multiple images are divided into extrinsic methods based on artificial markers, and intrinsic matching methods based solely on the patient related image data. The various methods are compared by a number of characteristics, which leads to a clear preference for one class of intrinsic methods, viz. voxel based matching. Furthermore, two- and three-dimensional techniques to display multimodality image information are outlined.

Analysis of magnetic resonance images (MRI) of the brain permits the identification and measurement of brain compartments. These compartments include normal subdivisions of brain tissue, such as gray matter, white matter and specific structures, and also include pathologic lesions associated with stroke or viral infection. A fuzzy system has been developed to analyze images of animal and human brain, segmenting the images into physiologically meaningful regions for display and measurement. This image segmentation system consists of two stages which include a fuzzy rule-based system and fuzzy c-means algorithm (FCM). The first stage of this system is a fuzzy rule-based system which classifies most pixels in MR images into several known classes and one `unclassified' group, which fails to fit the predetermined rules. In the second stage, this system uses the result of the first stage as initial estimates for the properties of the compartments and applies FCM to classify all the previously unclassified pixels. The initial prototypes are estimated by using the averages of the previously classified pixels. The combined processes constitute a fast, accurate and robust image segmentation system. This method can be applied to many clinical image segmentation problems. While the rule-based portion of the system allows specialized knowledge about the images to be incorporated, the FCM allows the resolution of ambiguities that result from noise and artifacts in the image data. The volumes and locations of the compartments can easily be measured and reported quantitatively once they are identified. It is easy to adapt this approach to new imaging problems, by introducing a new set of fuzzy rules and adjusting the number of expected compartments. However, for the purpose of building a practical fully automatic system, a rule learning mechanism may be necessary to improve the efficiency of modification of the fuzzy rules.

Video is the primary media used for cardiology diagnosis. Video data is currently stored on either cinefilm or videotape and provides no means of integration with other information such as patient's history, x-ray images, and test results. Maintaining and searching this data is also difficult. A multimedia database system would significantly enhance the diagnosis process by facilitating these integration, maintenance, and searching processes. One of the challenges in designing such a database is indexing the sequential video streams. The required storage space and the retrieval time to one specific video frame are two crucial issues in the design of a video database system. This paper presents two novel and fast image processing techniques for use in the selection of video frames. These selected frames can then be stored in the database. The first type of analysis is based on the analysis of spatial transitions and the second is based on the spectral components in the discrete cosine domain. The proposed techniques extract the length of the total edges in every frame of the cineangiogram. The extracted parameter is directly related to the amount of radio opaque dye injected in the coronary arteries. Those frames in the cineangiogram with a significant level of dye are then identified and indexed for evaluation. The extracted parameter index is used to select these frames for physician review. The same techniques are applied to the centered window of every frame to extract the cardiac cycles in the cineangiogram. This information can be used to index the stored cineangiogram for fast retrieval. It can also be used to synchronize two cineangiograms playing at the same time for clinical comparison and evaluation. Both techniques have been applied to twenty-one sets of cineangiograms. The results are very consistent with physician observation. Since both techniques produce comparable result, the selection between them is dependent on the format of the video. The second technique is effective when the compression standard JPEG or MPEG is used to encode the cineangiogram while the first one can be used for uncompressed or lossless compressed cineangiograms.

This paper describes an approach to mammogram analysis which detects suspicious regions in mammograms by comparing newly acquired mammograms with previous screenings of the same patient. Utilization of multiple screenings is motivated by the desire to reduce false positives and to increase sensitivity to true positives. Because of intractability of mammogram registration at the pixel level we propose a strategy for comparing mammograms regionally while taking into account differences normally expected between two images. In the proposed method mammograms are compared in three steps. First, mammograms are put into correspondence by using internal control points. Next, the images are partitioned into statistically homogeneous regions. Finally, the regions are compared by using intensity and texture measures and particular differences are labeled suspicious. This work concentrates on the issues of mammogram correspondence and region comparison.

Breast cancer is the leading cause of death among women. Breast cancer can be detected earlier by mammography than any other non-invasive examination. About 30% to 50% of breast cancers demonstrate tiny granulelike deposits of calcium called microcalcifications. It is difficult to distinguish between benign and malignant cases based on an examination of calcification regions, especially in hard-to-diagnose cases. We investigate the potential of using energy and entropy features computed from wavelet packets for their correlation with malignancy. Two types of Daubechies discrete filters were used as prototype wavelets. The energy and entropy features were computed for 128 benign and 63 malignant cases and analyzed using a multivariate cluster analysis and a univariate statistical analysis to reduce the feature set to a `five best set of features.' The efficacy of the reduced feature set to discriminate between the malignant and benign categories was evaluated using different multilayer perceptron architectures. The multilayer perceptron was trained using the backpropagation algorithm for various training and test set sizes. For each case 40 partitions of the data set were used to set up the training and test sets. The performance of the features was evaluated by computing the best area under the relative operating characteristic (ROC) curve and the average area under the ROC curve. The performance of the features computed from the wavelet packets was compared to a second set of features consisting of the wavelet packet features, image structure features and cluster features. The classification results are encouraging and indicate the potential of using features derived from wavelet packets in discriminating microcalcification regions into benign and malignant categories.

Cardiac B-mode ultrasonic tissue characterization is designed to use the fluctuations in acoustic signals from the myocardium to differentiate normal from morbid tissues due to their characteristic texture attenuation. In our paper, we suggest a new way that is based on frequency-domain texture analysis to quantify tissue characterization. The results of our experiments indicate that three of the power spectrum features PSE, PSM and LT can describe the characterization of myocardiac tissue very well either in normal or in diseased condition. We select these three features together with another feature PSER derived by combining two PSE values to construct a multilayered neural network based on BP algorithm which acts as a classifier to diagnose automatically input image samples of various diseases after being trained with known samples. The performance of the classifier is very satisfactory. In short, our system opens a new way for the quantitative diagnostic detection of diverse diseases including coronary heart disease myocardial infarction (CHD-MI) and dilated cardiomyopathy (DC).

When forming a diagnosis or developing a treatment plan, physicians frequently rely on images from multiple modalities of acquisition. Each of the sets of images contain different information vital to a proper diagnosis or treatment. Typically physicians are required to spatially integrate information from different sources (such as CT, MR, and SPECT) in order to reconstruct an informationally complete 3D model of the patient. This spatial integration of multimodality information is an extremely complex cognitive task, and in the end there is no tangible model available to verify either the diagnosis or treatment plan. A system allowing the registration and visualization of multiple modality 3D data is a tool physicians can use to improve both the quality of and their confidence in diagnosis and treatment planning. At the University of North Carolina at Chapel Hill we have developed a tool called FUSION to register and visualize multiple modality 3D data. Although our initial investigation began with, and still includes, automatic registration methods, we have found that an interactive method using 3D interactive visualizations is the most successful general approach for our registration needs. FUSION uses real-time volume and surface rendering techniques implemented on a SGI reality engine. With this tool, the user is provided with manipulation controls (to rotate, translate, and scale one or both data sets), contrast controls (to visualize various portions of each study's information), and visualization tools to see one or both of the studies with gray scale or various colors. These visualization methods include both static and dynamic display methods.

Intersubject registration of brain images involves localizing a set of easily detectable common features, matching these features, and finding a transformation that establishes a one-to-one correspondence of two image volumes from different subjects. Several different features and different methods for finding transformations have been described in the literature. A drawback common to the existing methods is that no, or few, features in the brain cortex are used. This paper studies the importance of incorporating such landmarks in brain registration. We apply a previously reported method to localize cortical convolutions in magnetic resonance images. We then derive a set of cortical landmarks from the traces representing the localized convolutions. Adding these cortical landmarks to a set of primarily subcortical landmarks proposed in an existing method, we compute matching transformations using a three dimensional thin-plate spline. Experiments comparing our method with a linear scaling method as well as an existing method that uses the previously proposed landmarks show that our method with the addition of cortical landmarks dramatically improves the registration quality in the cortex region.

There is an increasing demand for accurate registration of CT and MR images for diagnosis and for surgical planning. One useful technique registers images using externally attached fiducial markers. Previous studies have shown that point-based registration accuracy depends on fiducial localization accuracy, that is, the accuracy with which the position of a marker centroid can be estimated. This paper investigates the dependence of localization accuracy on marker size. The investigation is based on computer simulations. Three shapes are evaluated-- a sphere, a cylinder, and a cube, each with similar dimensions. The results indicate that (1) the accuracy depends strongly on the ratio of marker size to voxel size and (2) the dependence is almost identical among the three shapes tested. The simulation results are shown to agree with results from clinical images.

In this paper we present a hybrid registration technique that uses a weighted combination of several feature shapes, e.g. points and surfaces. The technique is a modification of Besl & McKay's iterative closest point (ICP) algorithm (IEEE Trans. Pattern Anal. Mach. Intell., 14:239-256, 1992). We use the technique to register x-ray computed tomography (CT) and T2-weighted magnetic resonance (MR) volume head images acquired from eleven patients that underwent craniotomies in a neurosurgical clinical trial. Each patient had five external markers attached to transcutaneous posts screwed into the outer table of the skull. We define registration error as the distance between corresponding marker positions after registration and transformation. The CT and MR images are registered using points (marker positions) only, surfaces only, and a weighted combination of points and surfaces. The CT surfaces are derived from contours corresponding to the inner surface of the skull. The MR surfaces are derived from contours corresponding to the cerebrospinal fluid (CSF)-dura interface. Registration using points and surfaces is more accurate than registration using only points or surfaces.

Ultrafast computed tomography is currently being investigated for the screening of early asymptomatic heart disease by evidence of the amount of calcium in the coronary arteries. The scanning technique for this test assumes that perfectly contiguous slices can be made by ECG gating and breath holds. Only subjective interpretation of the sequence of images is used to assess the adequacy of axial spatial sampling. This report investigates the autocorrelation matrix of thresholded binary images of heart cross sections in 10 subjects with and without ECG gating problems. In subject with gating problem the average correlation of adjacent slices was statistically reduced compared to subjects without gating problems. The incidence of extremely low or high adjacent slice correlations was greater in the poorly gated group. These preliminary results show that image based information may be used to assess the axial sampling adequacy of UFCT coronary calcification scans.

Radiation therapy is a treatment modality which seeks to deliver radiation energy to a localized site within a patient, in order to destroy a malignant tumor. The nature of radio-therapy results in dual, conflicting treatment goals: (1) the ability to deliver sufficient energy to a site so as to destroy the growth and, (2) sufficient localization of the energy to minimize the damage of surrounding, healthy tissue. One of the most important aspects of radiation dose treatment planning is the accurate localization of tumor masses. In order for a course of radiation therapy to be successful, the treatment volume must encompass the entire malignant process. Accordingly, the treatment volume must include the primary tumor of interest, as well as the direct and indirect course of the cancer's metastasis. Clinical results have demonstrated that a patient's tolerance to a given dose of radiation decreases as the volume exposed is increased. Therefore, improvements in tumor localization will provide the maximum amount of tissue sparing to the patient while encompassing the necessary target volume. An improved methodology is presented for the localization of tumors. This approach focuses on the integration of MRI and CT imaging data towards the generation of a mathematically optimal, tumor boundary. The solution to this problem is formulated within a framework integrating concepts from the fields of deformable modeling, fuzzy logic, and data fusion. Fuzzy edges derived from CT and MR are combined to form an integrated edge map, which subsequently guides the `growth' of a deformable tumor model. The fusion algorithm yields tumor contours which may be employed directly in the radiation therapy treatment planning process. Results are presented for the case of a phantom data set, with a simulated-implanted tumor, as well as for an actual patient.

The application of image matching to the problem of localizing structural anatomy in images of the human brain forms the specific aim of our work. The interpretation of such images is a difficult task for human observers because of the many ways in which the identity of a given structure can be obscured. Our approach is based on the assumption that a common topology underlies the anatomy of normal individuals. To the degree that this assumption holds, the localization problem can be solved by determining the mapping from the anatomy of a given individual to some reverential atlas of cerebral anatomy. Previous such approaches have in many cases relied on a physical interpretation of this mapping. In this paper, we examine a more general Bayesian formulation of the image matching problem and demonstrate the approach on two dimensional magnetic resonance images.

We have developed segmentation and simultaneous display techniques to facilitate the visualization of the three-dimensional spatial relationships between organ structures and organ vasculature. We concentrate on the visualization of the liver based on spiral computed tomography images. Surface-based 3-D rendering and maximal intensity projection algorithms are used for data visualization. To extract the liver in the serial of images accurately and efficiently, we have developed a user-friendly interactive program with a deformable-model segmentation. Surface rendering techniques are used to visualize the extracted structures, adjacent contours are aligned and fitted with a Bezier surface to yield a smooth surface. Visualization of the vascular structures, portal and hepatic veins, is achieved by applying a MIP technique to the extracted liver volume. To integrate the extracted structures they are surface-rendered and their MIP images are aligned and a color table is designed for simultaneous display of the combined liver/tumor and vasculature images. By combining the 3-D surface rendering and MIP techniques, portal veins, hepatic veins, and hepatic tumor can be inspected simultaneously and their spatial relationships can be more easily perceived. The proposed technique will be useful for visualization of both hepatic neoplasm and vasculature in surgical planning for tumor resection or living-donor liver transplantation.

In order to provide a better quantitative and morphologic description of complex vascular lesions, we propose an approach of 3D reconstruction of the vessel internal wall, based on data fusion from two different imaging sources: two x ray digital angiography projections and a stack of endovascular echography slices. After extraction of echographic and angiographic information to be fused, a geometric model leads to the determination of the unknown parameters which allow the alignment of all data in a common reference frame. Both types of data are then directly included in a probabilistic reconstruction process based on Markov random fields. The Markovian model consists of cost functions reflecting x ray and ultrasonic data consistency and regularization elements to control the anatomic reality of the reconstruction. The optimal solution according to the definition criteria is obtained by minimizing the model energy with an algorithm based on simulated annealing. Preliminary results have been obtained with data acquired on a dog aorta. The accuracy of reconstruction by data fusion is significantly improved compared with results obtained with separate reconstruction from angiographic or echographic data. Taking into account all information available about the problem, the method avoids uncertainties and ambiguities of a reconstruction based only on one modality, and the probabilistic fusion solves the possible contradictions between both acquisitions.

Multispectral classification uses registered 3-D image volumes from more than one imaging modality or from different sequences within a modality to classify tissues within those volumes. The complementary information contained within the different image volumes may allow for the separation of tissue class types in multidimensional feature space when the same tissue classes would be indistinct using just one image volume. When segmentation is complete, attributes of these classes may be determined (e.g., volumes), or the classes may be visualized as objects in 3-D. There are two main types of classification algorithms: supervised and unsupervised. Unsupervised classifiers offer the promise of totally automated classification of tissue types and calculation of tissue volumes and other tissue properties in medical images. This would have two benefits: (1) elimination of the time-consuming process of manual segmentation by medical experts, and (2) ensuring reproducible results. While accurate performance by unsupervised classifiers is, in general, still impossible, an intermediate step is the development of tools to allow users to obtain useful results in a relatively short period of time. This paper describes such a tool which allows users to quickly and easily experiment with various choices of unsupervised classification algorithms and their input parameters and evaluate the results.

Conventional vessel tracking and segmentation techniques identify the positions and two- dimensional structure of arteries in each frame of the angiographic sequence, but cannot distinguish the artery and background contributions to the intensity. We report a new technique for motion-compensated estimation of artery and background structures in coronary angiograms. The image within a region of interest is modeled as consisting of a sum of two independently moving layers, one of which contains the artery and one consisting of only background structures. The density of each of these layers is solved under two assumptions: (1) within each layer, the density varies from frame to frame only by rigid translation, and (2) the sum of the densities of the two layers equals the actual image density. This technique can be used to enhance image sequences by subtracting the component of the background whose temporal variation is entirely due to rigid translation. The feasibility of this technique is demonstrated on synthetic and clinical image sequences.

Since both detection and classification in an unsupervised stochastic model-based segmentation technique require the results from estimation, model parameter estimation becomes a core part of these types of techniques. Based on FNM model, ML likelihood equations and their solutions have been derived. After investigating the Bayesian probability in these solutions and comparing it with probability membership of EM algorithm, this paper proves that both CM and EM algorithms which are used in this segmentation technique do produce asymptotically unbiased ML estimates of FNM parameters, through the iterative process, when the stopping criteria are satisfied. The evolution of probability membership in the iterative process finally leads to the pixel labeling. The corresponding Cramer-Rao bounds of the variances of these estimates are also derived. The results by applying this performance analysis method to the simulated images with signal-to-noise (SNR) as a parameter, are obtained and reported in this paper. These results show that for the images with the moderate quality, the parameter estimates are quite accurate. When SNR >= 14.5 db, all estimates are within the one- standard deviation interval.

The precision and accuracy of area estimates from magnetic resonance (MR) brain images and using manual and automated segmentation methods are determined. Areas of the human hippocampus were measured to compare a new automatic method of segmentation with regions of interest drawn by an expert. MR images of nine normal subjects and nine schizophrenic patients were acquired with a 1.5-T unit (Siemens Medical Systems, Inc., Iselin, New Jersey). From each individual MPRAGE 3D volume image a single comparable 2-D slice (matrix equals 256 X 256) was chosen which corresponds to the same coronal slice of the hippocampus. The hippocampus was first manually segmented, then segmented using high dimensional transformations of a digital brain atlas to individual brain MR images. The repeatability of a trained rater was assessed by comparing two measurements from each individual subject. Variability was also compared within and between subject groups of schizophrenics and normal subjects. Finally, the precision and accuracy of automated segmentation of hippocampal areas were determined by comparing automated measurements to manual segmentation measurements made by the trained rater on MR and brain slice images. The results demonstrate the high repeatability of area measurement from MR images of the human hippocampus. Automated segmentation using high dimensional transformations from a digital brain atlas provides repeatability superior to that of manual segmentation. Furthermore, the validity of automated measurements was demonstrated by a high correlation with manual segmentation measurements made by a trained rater. Quantitative morphometry of brain substructures (e.g. hippocampus) is feasible by use of a high dimensional transformation of a digital brain atlas to an individual MR image. This method automates the search for neuromorphological correlates of schizophrenia by a new mathematically robust method with unprecedented sensitivity to small local and regional differences.

Segmentation of medical images poses a critical problem in image analysis. Segmenting a scene into different regions in the absence of sufficient apriori information is a challenging problem. A multiresolution image representation approach is presented here which makes use of a fuzzy neural network to segment a reconstructed image from wavelet decomposition into regions of interest. The multiresolution wavelets provide a basis for analyzing the information content of the image with global as well as local perspectives. The higher resolution levels contain information pertaining to the finer details while the lower resolutions capture the global features. A neuro-fuzzy algorithm facilitates the segmentation of the wavelet reconstructed image into different regions based on image intensity. The proposed algorithm has been applied to images of different kinds and has yielded promising results. The concept of using multiresolution wavelets and a neuro-fuzzy classification scheme has the added advantage of flexibility in the level of segmentation achieved.

A central problem in computer vision is to detect, delineate (segment) and recognize objects in an image. One reason why this is difficult is that very little information specific to given types of objects is used during segmentation. Making use of information about an object's shape -- in particular, about how it grows -- should facilitate and improve the segmentation of that object. We introduce a 2D discrete growth model for shapes on a Cartesian grid, based on notions related to biological growth. By growth is meant an accretionary process occurring at the boundary of the shape. We introduce a new type of deterministic growth model based on the notion of time delay. Associating a delay with each direction defines a time delay kernel (TDK); such kernels produce classes of convex octagons, and sequences of TDKs can give rise to arbitrary convex polygons. We show that growth in a stochastic environment of facilitators and inhibitors, which decrease or increase the time delays respectively, is a plausible analog for biological growth processes. As an example, we present results which suggest that simple periodic growth processes in an environment describe the gross morphology of multiple sclerosis lesions at the scale afforded by magnetic resonance images.

The application of mathematical morphology operations to medical grey level images is achieved by its fuzzy extension. We concentrate on angiographic images and the segmentation into linear vessel structures. A sequence of circular structuring elements serves both to match the course of arteries and to determine the diameters. In contrast to the traditional approach of globally applying morphological operations we defined a local operator. As a probe a small circular structuring element is put into a region of interest, i.e. an artery. It is capable of enlarging or shrinking itself dynamically and incrementally evaluating the subsumed area. While trying to maximize the diameter it moves and centers its focus. A sequence of such elements approximates parts of an artery. Branchpoints of the arterial tree are considered as further entry points. A fuzzy evaluation considers the distribution of intensity values found within the subsumed area. So a definition of a fuzzy set is derived which is representative for the depicted material. Fuzzy measures such as sharpness, fuzzy entropy and fuzzy equivalence are used to control the region growing. Using a modified Bresenham algorithm the fuzzy operator is implemented efficiently. An optimal approximation is achieved by utilizing the partial volume effect: here a horizontal interpretation is proposed considering the cumulative effect of a pixel instead of the vertical one used when interpolating CT-slices. Simply duplicating all pixel leads to a finer resolution of the adaptation process. We also include a fuzzy theoretic section to clarify the theoretic basis on which we interpret image pixel as fuzzy vectors and which allows us to perform a fuzzification of the image space or its associated coordinate system.

This paper describes efforts to study the motion of the left ventricle (LV) over the entire heart cycle directly from 4D MRI-data. A set of robust spatiotemporal operators with both a spatial and a temporal degree of freedom is constructed to provide a complete spatiotemporal description at a certain fixed location. This information is used to track features that correspond to anatomical structures of the LV. In order to measure the change of physical quantities such as the curvedness of the endocardial wall, we need its position in consecutive frames. In the case that a flow field is known we can use spatiotemporally oriented filters. Using a generalization of the optic flow constraint equation, which allows for a divergence term in the flow field and therefore is particularly suited for MRI-data, we calculated the flow field under the assumption of normal flow. Examples are included.

We propose a segmentation approach which integrates region growing and edge detection in a regularization framework. Our method is a modified active contour model and uses region statistics in addition to gradient information. We formulate the active contour model using a Bayesian approach. We have implemented this integrated approach and characterized its performance on synthetic images and on 36 short-axis cardiac ultrasound images. The resulting boundaries are compared to true boundaries in the case of the synthetic images and to manually outlined boundaries in the case of ultrasound images. The results are also compared with those obtained using the balloon force to expand the active contour model. We found that our integrated algorithm detects boundaries more accurately than the active contour method using a balloon force. Furthermore, the integrated algorithm is less sensitive to the placement of the initial contour inside the LV cavity than the active contour algorithm using a balloon force.

A nonlinear unsharp masking algorithm designed specifically to improve visualization of tube and line placements in chest x-ray imaging is presented in this paper. This algorithm was incorporated as an on-demand button in the design of a PACS display console. To address the concerns of radiologists about the artifacts accompanying image processing operations, we utilized a window-shade display scheme that shows both processed and unprocessed images simultaneously in the same display window at full resolution. We have found that the new algorithm enhances more details in low contrast areas without more over-shootings at high contrast edges. This algorithm is beneficial to tube/line placement examinations since the tips of tubes and lines are typically located in the areas of high brightness and low contrast. Preliminary results indicate that our modified unsharp masking algorithm as an option in a PACS workstation leads to better visualizations of tube-placement position for MICU physicians.

In this paper we propose an image enhancement algorithm that converts raw data from a storage phosphor scanner into a diagnostically optimized image. The spatial segmentation algorithm is applied to extract the body part of a clinical image, and histogram analysis is then used to develop the tonescale for the region of interest. This approach reduces the sensitivity to nonanatomical regions and allows full use of the display dynamic range for image viewing. The algorithms have been tested on clinical images and the results were reviewed by radiologists. Currently, these algorithms are being successfully used in the clinical environment.

In medical imaging applications, it is often necessary to enhance the bone-background contrast to improve the visibility of anatomical landmarks. Because of the low contrast nature of these images, many classical image enhancement techniques have met with limited success. In this paper, the image enhancement problem is treated as a natural extension to the image segmentation. We make specific use of the energy dependency of the x-ray attenuation characteristics to establish the basis for classification. The inherent ambiguity or vagueness in the bone-background classification is handled nicely with the fuzzy logic approach. The membership grade is generated with a generalized adaptive median filter to achieve the noise suppression and edge preservation. The final image is obtained by a non-linear gray scale mapping. Phantom and clinical studies have demonstrated the effectiveness of this approach. Some limitations of the approach are also discussed in the paper.

We have developed a procedure to image the major veins of the brain using three dimensional magnetic resonance imaging sequences. This procedure is based on the selective enhancement of blood vessels following intravenous administration of contrast media such as gadolinium. A simple substraction is performed between the post-contrast and pre-contrast MR images after correcting for overall intensity differences. The brain is segmented from the pre-contrast image volume. The resulting segmentation mask is dilated and used to extract the vessels of brain from the subtraction volume. The veins are reconstructed in three dimensions for visualization. The resulting tissue selective magnetic resonance venography is comparable to projection angiography for the larger veins. This procedure has been incorporated into a three dimensional stereotactic neurosurgical planning system.

The GI tract examination with CT and MRI is currently performed by slice-based inspection despite the volumetric nature of the problem. With this paper we propose to unravel the GI tract digitally to facilitate, quantify and automate the process. To demonstrate the feasibility, an algorithm was developed to unravel and measure convoluted curvilinear structures. In the algorithm, a local K-L transform was applied along the main axis of the structure to estimate the local orientation as determined by the principal eigenvector. An adult dog bowel piece was spirally scanned in vitro and reconstructed. A digital GI tract phantom was designed and numerically synthesized. In both cases, the structures were successfully followed using our unraveling algorithm, the lengths of the structures measured with less than 5% error, and cross-sectional features of the GI tract phantom were defined, computed and plotted.

By presenting a coordinated sequence of 3D projections of ultrasound information, the impression of a 3D object rotating in space is obtained. Thus, orientation and spacial distribution of 3D objects can be easily recognized and analyzed. By means of this reconstruction technique, transparency and 3D character are emphasized during the perception. Diagnostic interpretation far beyond the possibilities of a simple static projection is achieved. Using this technique in oral-maxillo-facial surgery we found improvement in diagnostic possibilities and diagnostic safety in the following applications: tumor diagnostics in oncology, improved localization in surgery, scanning of the temporomandibular joint, volume calculation of organs, volume calculation of arteriosclerotic plaques, localization and treatment diagnostics in lithotripsy. The possibilities in the different applications are presented in selected cases. Our experiences show that 3D sonography is now at the point where clinical use in different applications is possible. Further clinical trials will determine their benefits in daily clinical practice.

This paper discusses an approach to measuring the location of the human eye cornea using stereo pair images. A grid pattern is projected onto the cornea via an optical system and two digital cameras capture different views of the reflection simultaneously. The two images are then matched and rectified, and the surface can be inferred from the disparity of known points in the grid pattern. The design of an experimental prototype system is described and some typical results are presented.

A method of a three-dimensional surface reconstruction of the retina in the area of the papilla is presented. The surface reconstruction is based on a sequence of discrete gray-level images of the retina recorded by a scanning laser ophthalmoscope (SLO). The underlying assumption of the developed surface reconstruction algorithm is that the depth information is also encoded in the brightness values of the single pixels beside the ordinary spatial 2D information. The brightness of an image position depends also on the degree of reflection of a confocal laser beam. Only these surface structures produce a high response of the focused laser light, which are located directly in the focus plane of the confocal laser beam. The occurring disparities between the single images of a sequence are considered to be approximately linear and are corrected by applying the cepstrum technique. The depth information is estimated out of the volumetric representation of the image sequence by searching the maximal value of the brightness within a computed depth profile at every image position. In the resulting range images disturbances which occur during the recording cause wrong local estimations of the depth information. These local disturbances are corrected by applying especially developed surface improvement processes. The work is completed by investigating several different approaches to reduce the noise and disturbances of SLO image data.

Three-dimensional image acquisition, display, and analysis of dental structures was performed and validated using spiral computed tomography (SCT) with metal artifact suppression. Isolated extracted teeth, a dry mandible, cadaver mandible, and cadaver head were scanned and reconstructed using a spiral CT scanner (Siemens Somatom PLUS-S) with 1 mm detector collimation, 1-mm table feed, and 0.1 - 1 mm reconstruction interval using specially developed software. Algorithms for metal artifact reduction including extended attenuation range and interpolation of missing projections were applied. Volumetric rendering of voxel sum images was performed to synthesize images comparable to conventional intraoral dental radiographs. Direct comparison of voxel-based synthetic and digitized film images was made. Several isolated, extracted teeth were sectioned with a diamond saw and submitted for histomorphometric analysis to aid in direct comparison with CT slice images obtained by multiplanar reconstruction. Metal artifact reduction was successful in markedly reducing the streaks and star patterns that usually accompany metallic restorations and intraoral appliances. Individual teeth were comparable to CT slice images. Voxel sum images were comparable to dental radiographs; however, for the SCT images, the spatial resolution was higher within the plane of section than it was orthogonal to the plane of section. Serial examinations were obtained by SCT, registered by surface matching, and interval change measured by 3D subtraction. Simulated lesions and restorations were introduced and quantitatively evaluated pre- and post-interventionally to assess imaging method performance.

Coronary arteriography is a technique used for evaluating the state of the coronary arteries. Matching of coronary arteries from multiple views is necessary for obtaining a 3-D description of the arterial tree. Overlapping vessels and artifacts due to digital subtraction of the angiogram background make the matching process quite difficult. The simplex method applied for linear programming and a relaxation technique for pre-processing the data are applied to skeletons from two views in order to obtain a matching of branches between views. The elements of the centerline along the branch are modeled as a Markov random field and a matching of each element in the two views is obtained by minimizing the energy of the matching contour. The element matching is treated as an estimation problem such that the a- posteriori probability is maximized. Results are provided for the 3-D reconstruction using these algorithms for automatic correspondence, and compared to those obtained by manual correspondence specification. This work was performed using a pig-cast realistic phantom. The results are encouraging.

Confocal images of double labeled DNA replication sites in mouse 3T3 cells were analyzed using a set of image processing algorithms to investigate the spatial organization of replication sites at different times. Since confocal microscopy images have about a 1:10 ratio of horizontal (X-Y) versus vertical (Z) scale, extending traditional 2D image processing techniques to 3D rarely gives satisfactory results. By using 2D image segmentation on the individual slices to obtain replication site contours and combining this high-level 2D data using a modified 3D connected component labeling algorithm with weak connectivity in the Z direction, we were able to reconstruct the network of 3D replication sites. Instead of using the equivalent of traditional 2D 8-connectivity in 3D, we used a metric which evaluated the amount of overlap between successive contours of a replication site to decide whether the two contours belong to the same replication site. This method is applicable in analyzing the images of replication sites as they are bead like structures. Once the data structures describing the individual sites are obtained, several pattern recognition techniques are applied to elucidate 3D higher order assembly of replication sites. In one method, the amount of site overlap between the two channels was calculated in an attempt to quantify the closeness of sites at different times. Another approach consisted of grouping the replication sites using a pre-set distance between sites required for a set of sites to be included in the same group.

In positron emission tomography (PET) images have to be reconstructed from noisy projection data. The noise on the PET data can be modeled by a Poisson distribution. The development of statistical (iterative) reconstruction techniques addresses the problem of noise. In this paper we present the results of introducing the simulated annealing technique as a statistical reconstruction algorithm for PET. We have successfully implemented a reconstruction algorithm based upon simulated annealing, with paying particular attention to the fine-tuning of various parameters (cooling schedule, granularity, stopping rule, ...). In addition, we have developed a cost function more appropriate to the noise statistics (e.g. Poisson) and the reconstruction method (e.g. ML). The comparison with other reconstruction methods using computer phantom studies proves the potential power of the simulated annealing technique for the reconstruction of PET-images.

The ill-posed problem of reconstructing an animation from a set of motion-corrupted MR images is investigated. A temporal sampling model has been developed which maps each phase-encoded row of a motion-corrupted MR image to particular frames of an animation. This mapping partially fills the Fourier space of the image frames, and the remaining Fourier space is reconstructed using the projection onto convex sets (POCS) algorithm. We have tested this procedure by reconstructing an animation of a beating heart from a set of simulated motion-corrupted heart images. There is evidence to suggest that this technique may reduce the amount of data required for reconstructing an animation.

The multi-grid expectation maximization algorithm (MGEM), an extension of the maximum likelihood (ML) algorithm, has been applied to the problem of reconstruction in positron emission tomography (PET). The MGEM algorithm implemented the expectation maximization (EM) algorithm on different size image grids. The algorithm is based on the idea that the low frequency image components can be recovered faster than the high frequency components. The algorithm begins with a coarse grid, where the low frequency components are recovered. After the low frequency components have been recovered, the solution is projected onto the next finer grid. On the next finder grid, the high frequency components from the previous grid become the low frequency components. The algorithm continues iterating and switching levels until the finest grid has been reached. This method provides faster convergence than the single grid EM algorithm. An important issue concerning the MGEM algorithm is when to stop iterating at a particular grid and project to the next grid, or stop at the finest grid. The convergence rate of the MGEM algorithm was used as a grid level transition criterion. A grid level transition criterion which used the co-occurrence matrix statistics of the reconstructed image is presented. The spatial distribution and dependence among gray levels in a local area of the reconstructed image can be studied by gray level dependent co-occurrence statistics. The second order histogram represents the probability of occurrence of a pair of gray levels separated by a given displacement vector. The features computed from the second-order histogram were entropy, contrast, angular second moment, inverse difference moment, correlation, mean, and deviation. The difference histogram is derived from the second order histogram and represents the probability of occurrence of a difference in gray levels of two pixels separated by a displacement vector. The features extracted from the difference histogram were entropy, contrast, and angular second moment. The behavior of each of the statistics was compared to the root mean squared (RMS) error, to evaluate the potential for use as a transition criterion. An appropriate window was used to average the variance of each statistic. The indication of switching to the next finer grid level, or of stopping at the finest grid level was provided by a decline in the variance of a statistic. These statistics were computed on an image reconstructed from simulated PET data. Each of the statistics was evaluated versus the number of iterations at each grid level. The transitions for each level occurred when the variance of one of the statistics decreased sharply. For comparison, the single grid EM algorithm was allowed to run for the same amount of CPU time as the MGEM algorithm. The RMS error indicates that the MGEM method, using the co- occurrence statistics as a transition criterion, produced a better reconstruction than the single grid EM algorithm.

The depth of invasion and three-dimensional characteristics of skin lesions are important parameters in the early diagnosis and prognosis of malignant melanoma. `Nevoscopy' (Nevus - pigmented skin lesion), an optical trans-illumination based modality, has previously been proposed for the non-invasive assessment of skin tumors. The skin area surrounding a lesion is illuminated with a visible light source in contact with the skin, avoiding all surface illumination. The Nevoscope with its mirrors at different angles and orientations provides multiple 2D views of the trans-illuminated lesions. Nevoscopy together with image analysis has the potential to quantify all the parameters used in melanoma diagnosis. This paper describes the development of reconstruction algorithms for application to Nevoscopy. Initially, x-ray CT algorithms were applied to Nevoscopy data under heuristic assumptions of linear source-detector relationships. In reality, the reconstruction from scattered optical radiation is a nonlinear inverse problem which is typically solved by progressively solving linear inverse problems. The problem defaults to determination of influence of increased localized absorption on a given source-detector measurement. Photon transport theory and its surrogate diffusion theory are employed to compute this local contribution or `weight.' A three-point diffusion theory model, based on diploe solutions to the diffusion equation, is proposed as a deterministic method to compute the voxel weights for a discrete representation of a homogeneous medium. Simulations of 2D Nevoscope-like imaging geometries indicate that the weights computed by this method approximate those computed by rigorous statistical Monte Carlo statistical procedure. Reconstructions of embedded absorber in homogeneous media using the three-point diffusion weights are comparable to those obtained using Monte Carlo weights. The proposed three point diffusion model was extended to the existing image geometry and applied to reconstruct volumes from the projections of phantom and skin images. Results are presented. Direct extension of the three point diffusion model to the existing Nevoscope geometry has brought into light some of the deficiencies of both the current geometry of the system and the model.

The Bayesian approach allows one to combine measurement data with prior knowledge about models of reality to draw inferences about the validity of those models. The posterior probability quantifies the degree of certainty one has about those models. We propose a method to explore the reliability, or uncertainty, of specific features of a Bayesian solution. If on draws an analogy between the negative logarithm of the posterior and a physical potential, the gradient of this potential can be interpreted as a force that acts on the model. As model parameters are perturbed from their maximum a posteriori (MAP) values, the strength of the restoring force that drives them back to the MAP solution is directly related to the uncertainty in those parameter estimates. The correlations between the uncertainties of parameter estimates can be elucidated.

In this work we have explored the problem of automating the construction of a customized femoral prosthesis. We meditate why automated construction of such prostheses can be interesting, both in terms of a rational design process, quality assurance, low level optimization and explicit inclusion of design criteria based on the experience developed through evaluation of manual designs. It is demonstrated how design criteria have successfully been given an explicit mathematical description and included in a semi-automatic environment for the extraction of the prosthesis geometry based on a set of digital CT-slices of femur. The mathematical models are integrated in an edge detection framework based on active contours and dynamic programming. A comparison between a manual and the proposed automated design process is given.

Spiral x-ray computed tomography (SXCT) volumetric imaging was applied to in situ goodness of fit evaluation for lower extremity (LE) prostheses with and without axial loading. SXCT data was obtained (Siemens Somatom PLUS-S) with and without the prosthesis in place. An algorithm was developed to map and measure the residuum bony and soft tissue structure and their relationship to the rigid prosthesis socket. A transform was applied along the main axis of the structure to estimate the local soft tissue thickness relative to bone and map it from a Cartesian coordinate voxel array into cylindrical and spherical (Lambert projection) maps. Interval changes in the soft tissue envelope relative to the underlying skeleton were measured by comparing maps obtained from serial examinations. The test-retest repeatability and validity of SXCT methods was assessed using cadaver parts, phantom test objects, and human volunteers. The soft tissue envelope of lower limb residua were successfully determined, and the precision (repeatability) of SXCT was consistently better than 90%. Soft tissue SXCT mapping of a lower limb residuum is feasible with the prosthesis in situ and provides comprehensive information on the geometry and tissue characteristics for static evaluation of prosthesis fit.

The purpose is to develop a method of estimation of the field of vectors velocity to evaluate motion of the heart in view of an assistance with the medical diagnosis. This method is based on an application of neural network to optical flow estimation in a sequence of echocardiographic images. The network is discrete, deterministic and locally connected. It contains redundant neurons for representing w_(i,j)x and w_(i,j)y the components of the velocity vector at point (i,j). We formulate the motion estimation problem as one of minimizing an error function of three terms. The first term is a least-squares penalty expression which coerce the flow field into satisfying the flow equation; the second is a smoothness constraint used for obtaining a smooth optical flow and the third term is a line process to weaken the smoothness constraint and to detect motion discontinuities. The minimization of this function is obtained with the help from a neural network. The model consist of 2*L*M Hopfield sub-networks for an image of size L*M. Each sub-network is totally interconnected. Results provide a good coherence between the homogeneous area cartography of motion and textured regions present in the image.

Analogic CNN algorithms were developed and introduced to detect the major pathological signs of the breast cancer in x-ray mammogram images. The reported algorithms can detect the microcalcifications and the spiculi around a given tumor kernel.

A computer aided diagnostic system has been developed to predict the status of a breast implant (intact/ruptured) based on mammographic findings. Mammograms were obtained from 112 patients who presented for surgical removal of breast implants. Findings were recorded by radiologists for each patient. Of these 112 cases, 77 were ruptured while 35 were intact at the time of surgery. An artificial neural network (ANN) was trained to output the implant status when given the mammographic findings as inputs. The ANN was a backpropagation network with nine inputs, one hidden layer with 4 nodes, and one output node (implant status). The network was trained using the round-robin technique and evaluated using ROC analysis. The network performed well with an ROC area of 0.84. This was better than the radiologists's performance with sensitivity of 0.67 and specificity of 0.72. At a sensitivity of 0.67 (to match the radiologists), the network had a specificity of 0.89. At a specificity of 0.72 (to match the radiologists), the network has a sensitivity of 0.78. An ANN has been developed which demonstrates encouraging diagnostic performance for predicting the status of breast implants from mammographic findings.

We have developed interval-based Bayesian belief networks and used them in a medical image recognition system to identify brain anatomy in a set of multi-modality images. Head images from the multi-modalities of x-ray CT, proton density MR, and T2-weighted MR are used in the proposed medical image recognition system. To incorporate the capability of handling uncertainty in the medical image recognition system, we use interval-based probability to represent incomplete information and uncertain evidence and the interval-based Bayesian belief networks to provide the framework for evidence accumulation and spatial reasoning. In evidence accumulation, the regional information extracted from the CT and MR images is converted to evidence in the form of probability intervals and organized in a hierarchical structure. The evidence is integrated by the belief networks. In spatial reasoning, the knowledge of spatial relationships among objects is organized in hierarchical structure with pairwise spatial relationships. The interval-based Bayesian belief networks and a prediction box are combined to hypothesize the potential location and likelihood of the target object. In the proposed medical image system, the use of the interval-based Bayesian belief networks provides flexibility for representing and handling uncertain evidence and reduces the distraction in defining point probabilities for the domain knowledge. It also allows the proposed recognition system to perform reasoning efficiently and identify brain anatomy effectively.

Accurate and reproducible volume calculations are essential for diagnosis and treatment evaluation for many medical situations. Current techniques employ planimetric methods that are very time consuming to obtain reliable results. The reproducibility and accuracy of these methods depend on the user and the complexity of the volume being measured. We have reported on an algorithm for volume calculation that uses the Eigenimage filter to segment a desired feature from surrounding, interfering features. The pixel intensities of the resulting image have information pertaining to partial volume averaging effects in each voxel preserved thus providing an accurate volume calculation. Also, the amount of time required is significantly reduced, as compared to planimetric methods, and the reproducibility is less user dependent and is independent of the volume shape. In simulations and phantom studies the error in accuracy and reproducibility of this method were less than 2%. The purpose of this study was to determine the reproducibility of the method for volume calculations of the human brain. Ten volunteers were imaged and the volume of white matter, gray matter, and CSF were estimated. The time required to calculate the volume for all three tissues was approximately one minute per slice. The inter- and intra-observer reproducibility errors were less than 5% on average for all volumes calculated. These results were determined to be dependent on the proper selection of the ROIs used to define the tissue signature vectors and the non-uniformity of the MRI system.

Multimodality images obtained from different medical imaging systems such as magnetic resonance (MR), computed tomography (CT), ultrasound (US), positron emission tomography (PET), single photon emission computed tomography (SPECT) provide largely complementary characteristic or diagnostic information. Therefore, it is an important research objective to `fuse' or combine this complementary data into a composite form which would provide synergistic information about the objects under examination. An important first step in the use of complementary fused images is 3D image registration, where multi-modality images are brought into spatial alignment so that the point-to-point correspondence between image data sets is known. Current research in the field of multimodality image registration has resulted in the development and implementation of several different registration algorithms, each with its own set of requirements and parameters. Our research has focused on the development of a general paradigm for measuring, evaluating and comparing the performance of different registration algorithms. Rather than evaluating the results of one algorithm under a specific set of conditions, we suggest a general approach to validation using simulation experiments, where the exact spatial relationship between data sets is known, along with phantom data, to characterize the behavior of an algorithm via a set of quantitative image measurements. This behavior may then be related to the algorithm's performance with real patient data, where the exact spatial relationship between multimodality images is unknown. Current results indicate that our approach is general enough to apply to several different registration algorithms. Our methods are useful for understanding the different sources of registration error and for comparing the results between different algorithms.

In digital subtraction angiography, the use of an image intensifier as a detector introduces geometrical distortions in the images. For stereotactic applications, such as the irradiation of cerebral arteriovenous malformations, these distortions have necessarily to be corrected, and the accuracy of this correction has to be examined. As the distortions depend on many parameters that vary during an examination (such as magnetic field and spatial position of the acquisition chain), the correction accuracy must be defined as a function of the acquisition protocol. We have developed a correction method based on the calibration of geometrical distortions using an image of a grid phantom. An experimental study of the influence of acquisition parameters over the distortion has been performed. A protocol has been defined which ensures a correction accuracy of 0.1 millimeter. Finally, we have studied the accuracy obtained in the 3D location of a target as a function of the accuracy of the distortion correction. The final precision allows the use of our method for digital x-ray stereotactic applications.

A method utilizing digital dual-energy substraction x-ray radiography for measuring calcium densities localized to the cortical and cancellous regions of bone cross sections is described. The method is proposed as a technique for studying calcium loss in femurs of rats used in experiments modeling the temporal of osteoporosis. The densitometry data obtained for a rat leg consists of six x-ray projection images acquired at 30 degree(s) angular intervals about the bone axis by an intensifying screen/CCD camera imaging system. Images of bone cross sections are reconstructed by application of a maximum entropy algorithm constrained by the six projection images. The observed density data are further discriminated into cortical, cancellous and external regions on the basis of reference levels found on image density histograms.

A fully automated system is being developed for the portal verification of tangential breast fields in radiation therapy of breast cancer. The automated verification system involves image acquisition, image feature extraction, feature correlation between reference and portal images, and quantitative evaluation of patient setup. In this study, the portal images are acquired using a matrix liquid ion-chamber electronic portal imaging device (EPID), and have a matrix size of 256 X 256 pixels with 12-bit gray levels. A hierarchical region processing technique is developed to extract poor contrast features in the portal image generated by megavoltage photon beams at different levels sequentially. The treatment field is initially extracted from the portal image. The skin line is then extracted from the treatment field. Finally, the lung/soft tissue separation is extracted from the breast region. A Chamfer matching filter is used to correlate features in the portal image with those in the reference image. The resulting parameters for rotation, translation and scaling are used for the setup evaluation of the treatment field.

Arterial hypertension is a high prevalence disease in Western countries and it is associated with increased risk for cardiovascular accidents. Retinal vessel changes are common findings in patients suffering from long-standing hypertensive disease. Morphological evaluations of the fundus oculi represent a fundamental tool for the clinical approach to the patient with hypertension. A qualitative analysis of the retinal lesions is usually performed and this implies severe limitations both in the classification of the different degrees of the pathology and in the follow-up of the disease. A diagnostic system based on a quantitative analysis of the retinal changes could overcome these problems. Our computerized approach was intended for this scope. The paper concentrates on the results and the implications of a computerized approach to the automatic extraction of numerical indexes describing morphological details of the fundus oculi. A previously developed image processing and recognition system, documented elsewhere and briefly described here, was successfully tested in pre-clinical experiments and applied in the evaluation of normal as well as of pathological fundus. The software system was developed to extract indexes such as caliber and path of vessels, local tortuosity of arteries and arterioles, positions and angles of crossings between two vessels. The reliability of the results, justified by their low variability, makes feasible the standardization of quantitative parameters to be used both in the diagnosis and in the prognosis of hypertension, and also allows prospective studies based upon them.

In this report, we present an adaptive-sized neural network model for the detection of microcalcifications. The neural network has capabilities of automatically adjusting the network size depending on the training set, of rejecting unknown inputs, and of fast learning. When the adaptive-sized neural network is used, the user can find the optimal network size without trial and error. In addition, the reliability of the network performance is high because of the rejection of unlearned inputs. The inputs for the neural network used in this study were 11 X 11 pixel sub-images that were extracted from digitized mammograms. The experiments in 83.3% sensitivity, 84.3% specificity, and 22.4% rejection rate. The weight patterns after learning process and the dependency of the network performance on the order of presenting training examples were also studied.

The objective of many existing computer-aided diagnosis (CADx) schemes for lung nodule detection is to reduce the number of false-positives (i.e., increase specificity) while maintaining a high level of sensitivity. Our examination of the false-positives obtained with the previously developed CADx program show that many round objects, such as rib crossings, end-on vessels, and aggregates of vessels, were mistakenly classified as nodules. Among the problems of decreasing the number of false-positives, the differentiation between nodules and end-on vessels is one of the most challenging tasks performed by computers. To eliminate the false-positives, two methods are proposed. One method is to extract the known features (i.e., contrast and size) based on a conventional digital image processing technique. The other method uses an artificial neural network (ANN) which is specifically trained to classify nodules and end-on vessels. Performances of the two approaches are evaluated using the receiver operating characteristics (ROC) method and the area under the ROC curve (A_z). Based on our test database, the FFNN and the algorithmic approaches showed preliminary ROC performances with A_z values equal to 0.90 and 0.94, respectively.

This study aimed to develop an artificial neural network for computer-aided diagnosis in mammography, using an optimally minimized number of inputs from a standardized lexicon for mammographic features. A three-layer backpropagation neural network merged seven inputs (six radiographic findings extracted by radiologists plus age) to predict biopsy outcome as its output. Each input feature was ranked by importance, as determined by the reduction of Az when that feature was excluded and the network retrained. Once ranked, the input features were discarded in order from least to most important until performance was significantly degraded, resulting in an optimized subset of features. The neural network trained on all seven input features performed with an Az of 0.90 +/- 0.02, compared to experienced radiologists' Az of 0.88 +/- 0.02. The difference in Az was not statistically significant (p equals 0.29). The network continued to perform well given as few as three inputs: mass margin, age, and calcification description.

A two-dimensional convolution neural network (CNN) with wavelet kernels (WK) has been developed for image pattern recognition. The structure of the CNN is a simplified version of the neocognitron. We used only a two-level structure and eliminated all complex-cell layers. Nets between two adjacent layers in the feature selection level of the CNN are selectively interconnected across groups. In this part of the CNN signals processing, each group in the receiving layer receives signals from a group of weights (i.e., kernels). For the forward signal propagation, the product obtained from the kernel convoluting the front layer is collected onto the corresponding matrix element of the receiving layer. In this paper, the convolution kernels of the CNN (CNN/WK) are wavelet based and are trained by a supervised manner. In the development of the CNN/WK, we forced each updated convolution kernel to be orthonormal. Therefore, features (transformed coefficients) selected on the transform domain are linearly independent. Hence, the fully connected layers in the classification level of the CNN can perform more effectively. The applications of the CNN for disease pattern recognition have been very successful. When isolated patterns were further processed by internal filtering and classification layers were built into the neural network structure, the disease patterns were more easily recognized. Although, we did not receive substantial improvement of the ROC performance using the CNN/WK, this method may assist us in the analysis of the trained kernels and eventually lead to the optimization of feature extraction in a course of disease pattern recognition.

This paper presents a novel approach for segmentation of suspicious mass regions in digitized mammograms using a new adaptive density-weighted contrast enhancement (DWCE) filter in conjunction with Laplacian-Gaussian (LG) edge detection. The new algorithm processes a mammogram in two stages. In the first stage the entire mammogram is filtered globally using a DWCE adaptive filter which enhances the local contrast of the image based on its local mean pixel values. The enhanced image is then segmented with an LG edge detector into isolated objects. In the second stage of processing, the DWCE adaptive filter and the edge detector are applied locally to each of the segmented object regions detected in the first stage. The number of objects is then reduced based on morphological features. ROIs are selected from the remaining object set based on the centroid locations of the individual objects. The selected ROIs are then input to either a linear discriminant analysis (LDA) classifier or a convolution neural network (CNN) to further differentiate true-positives and false-positives. In this study ROIs obtained from a set of 84 images were used to train the LDA and CNN classifiers. The DWCE algorithm was then,used to extract ROIs from a set of 84 test images. The trained LDA and CNN classifiers were subsequently applied to the extracted ROIs, and the dependence of the detection system's accuracy on the feature extraction and classification techniques was analyzed.

This paper reports on the preliminary results of an on-going prospective evaluation of an `intelligent' mammography workstation. This workstation can provide to radiologists a `second opinion' as to the location of suspicious lesions on mammograms. The workstation consists of a high speed computer, film digitizer, image archive, and both hard and soft copy output. Running on the workstation are automated computerized schemes for the detection of breast masses and clustered microcalcifications. In the current study, all screening mammograms are digitized on the workstation and then analyzed by the computerized schemes. The preliminary results for the first 37 days (573 patients) have been analyzed. Although follow-up to establish truth has not been done for all patients, the two schemes detected the lesion in 10 of the 14 patients who had a `suspicious' lesion present mammographically. Three of the lesions missed by the computer were found to be benign either at biopsy or after further work-up, and the fourth one is scheduled for further work-up. For two patients, a cluster of microcalcifications was detected by the computerized scheme that was initially missed by the radiologist. The false positive rate was 1.2 false masses and 0.87 false clusters per image. Over 70% of the false positive masses were caused by nodular densities and approximately 50% of the false cluster included obviously benign calcifications. The results from this ongoing study will be used to plan a full-scale clinical study.

We are investigating the use of texture features to distinguish abnormal from normal tissue for computer-aided diagnosis algorithms. In this study, the improvement in the accuracy of classifying masses from normal breast tissue on digitized mammograms by using local texture features was evaluated. One hundred and sixty-eight mammograms were randomly selected from patient files and digitized with a laser scanner at a pixel size of 100 micrometers . Four different regions of interest (ROIs), each of 256 X 256 pixels, were selected manually from each of the digitized mammograms. One of the four ROIs contained a biopsy-proven mass and the other three contained normal parenchyma including dense, mixed dense/fatty, and fatty tissues. The mass ROIs were randomly and equally divided into a training and a test group along with corresponding normal ROIs from the same film. The wavelet transform was used to decompose the ROIs into several scales. Global multiresolution texture features were calculated from the spatial gray level dependence matrices of the low-pass wavelet coefficients up to a certain scale and then at variable distances between the pixel pairs. Texture features and their differences in suspicious object sub-region and its peripheral sub-regions of the ROIs were also calculated to form a local texture feature space. Stepwise linear discriminant analysis was used to select effective features from the combined global-local feature space to maximize the separation of mass and normal tissue for all ROIs. Receiver operating characteristic (ROC) analysis was used to evaluate the classification accuracy and its improvement using features from global and local feature spaces. Using the global multiresolution feature space alone, it was found that the texture features at large pixel distances were important for the classification task. Using local features only, the classification accuracy was comparable to that of the global features. With the combined global and local feature spaces, the average area, A_z, under the ROC curve reached 0.91 and 0.90 for the training and test groups, respectively. The improvement was statistically significant. The results demonstrate that a linear discriminant classifier using the combination of global multiresolution texture features and the local texture features can effectively classify masses from normal tissue on mammograms.

We are developing computer-aided diagnosis algorithms to assist radiologists in detection and classification of microcalcifications on mammograms. A digitized mammogram was processed with a difference-image technique and signal segmentation methods to identify suspicious signals. False-positive detections were reduced by using morphological features as well as a convolution neural network. A regional clustering technique was applied to the remaining signals to identify clinically significant clustered microcalcifications. For the development of a malignant/benign classifier, the microcalcifications were extracted from the digital images by computerized segmentation techniques. A number of visibility descriptors and shape descriptors were developed to describe the features of the microcalcifications. Linear discriminant analysis and receiver operating characteristic (ROC) methodology were used to classify the benign and malignant microcalcifications. For detection of microcalcifications, the computer reached a true-positive (TP) rate of 100% at 0.1 false-positive (FP) clusters per image for obvious microcalcifications, a TP rate of 93% at 1 FP clusters per image for average subtle microcalcifications, and a TP rate of 87% at 1.5 FP clusters per image for very subtle microcalcifications. For classification of microcalcifications, preliminary results indicated that an area under the ROC curve (A_z) of 0.91 and 0.89 could be achieved during training, and an A_z of 0.82 and 0.87 during jackknife testing for obvious and subtle clusters, respectively. When all cases were combined, the A_z was 0.87 and 0.84, respectively, for training and jackknife testing.

To aid radiologists in the detection of lung cancer, we are developing a computer-aided diagnosis (CAD) system that locates areas suspicious for nodules in digital chest radiographs. The system involves a difference-image approach and various feature-extraction techniques. To improve the sensitivity of the overall scheme, we investigated an approach based on the wavelet transform. We found that the wavelet transform is effective in detecting subtle nodules that were not detected by the difference-image technique. By combining the two schemes based on the difference-image and the wavelet transform, the sensitivity was increased from 81% to 86% while maintaining approximately the same level of specificity.

A three-dimensional ultrasound imaging system was developed for studying tongue configurations during speech and swallowing. A sequence of two-dimensional ultrasound B- mode images was acquired by moving the ultrasound transducer under the subject's chin. A six-degree-of-freedom electromagnetic position sensor was used in order to determine the spatial position and orientation of the ultrasound transducer during the scanning. Registration of image slices was achieved by using a time code generator to synchronize ultrasound images with the spatial information. Techniques were developed for 3D reconstruction of the tongue surface from multi-planar ultrasound scans using both commercial software and NIH- developed programs for PC and Macintosh computers. The system demonstrated its potential to quickly acquire and reconstruct 3D tongue images, and to assist speech pathologists and radiologists in speech and swallowing disorder diagnosis.

This study investigates the differences in both image texture and numerical performance among various filtering methods used in filtered back-projection reconstructions, including Butterworth, variable conductance diffusion (VCD), and a combination approach of the two. A (chi) ² Butterworth method is proposed to be used for 2-D prefiltering on the projection data and a 3-D VCD method is then applied post-reconstruction. The combination approach has the smooth boundaries a VCD method cannot normally obtain while minimizing the severe ringings normally incurred by a Butterworth with low cut-off. Use of (chi) ² criterion in Butterworth provides a reasonably good starting point for VCD, which then only needs a few iterations to reach a desirable degree of smoothness. The use of Butterworth filtering before VCD also results in less noisy estimates of the parameters which control VCD. Thus, the combination of global smoothing with the Butterworth, and local smoothing with VCD is much faster than VCD alone and preserves the desirable properties of each.

Observed medical ultrasound radio-frequency (rf) images are blurred and noisy representations of the true tissue reflectance. The blurring of the specular reflections that occur at boundaries between regions of different tissue types is particularly troublesome, and it is difficult to remove with standard restoration techniques such as Wiener filtering. In this paper we develop an iterative restoration/segmentation algorithm for 2-D ultrasound rf images, that is based on a new statistical image model incorporating spatial smoothness constraints and a physically accurate model for edge reflections. The algorithm alternates steps of restoration and segmentation, calculated using simulated annealing, and includes estimation of reflectance parameters. The smoothness constraints serve to regularize the overall procedure, and the edge reflection model allows the algorithm to use the specular reflections to help locate region boundaries. The resulting restorations of some simulated and real rf-images are significantly better than those produced by Wiener filtering.

The purpose of this paper is to develop a robust and accurate method that automatically segments phalangeal and epiphyseal bones from digital pediatric hand radiographs exhibiting various stages of growth. The algorithm uses an object-oriented approach comprising several stages beginning with the most general objects to be segmented, such as the outline of the hand from background, and proceeding in a succession of stages to the most specific object, such as a specific phalangeal bone from a digit of the hand. Each stage carries custom operators unique to the needs of that specific stage which will aid in more accurate results. The method is further aided by a knowledge base where all model contours and other information such as age, race, and sex, are stored. Shape models, 1-D wrist profiles, as well as an interpretation tree are used to map model and data contour segments. Shape analysis is performed using an arc-length orientation transform. The method is tested on close to 340 phalangeal and epiphyseal objects to be segmented from 17 cases of pediatric hand images obtained from our clinical PACS. Patient age ranges from 2 - 16 years. A pediatric radiologist preliminarily assessed the results of the object contours and were found to be accurate to within 95% for cases with non-fused bones and to within 85% for cases with fused bones. With accurate and robust results, the method can be applied toward areas such as the determination of bone age, the development of a normal hand atlas, and the characterization of many congenital and acquired growth diseases. Furthermore, this method's architecture can be applied to other image segmentation problems.

Many image registration methods use the surface of the head or brain to estimate rotation and translation parameters. The surface is usually characterized by a set of edge or contour points extracted from cross-sectional images. Automatic extraction of contour points is complicated by discontinuity of edges in the back of eyes and ears and sometimes by an inadequate field of view. We have developed an automated method for contour extraction that connects discontinuities using a multi-scale pyramid. Steps of the method are: (1) contour points are found by an edge-tracking algorithm; (2) a multi-scale pyramid of contour points is constructed; (3) contour points of reduced images are found; (4) from the continuous contour found at the lowest scale, contour points at a higher scale are found; (5) step 4 is repeated until contour points at the highest resolution (original image) are found. The method runs fast and has successfully been used for MRI and CT image registration. We illustrate the method and its performance using MRI and CT images of the human brain.

We are developing automated-detection schemes for the masses and clustered microcalcifications on laser-digitized mammograms (0.1 mm, 10-bit resolution, 2000 X 2510) by using a conventional workstation. The purpose of this paper is to provide an overview of our recent schemes and to evaluate the current performance of the schemes. The fully automated computer system consists of several parts such as the extraction of breast region, detection of masses, detection of clustered microcalcifications, classification of the candidates, and the display of the detected results. Our schemes tested with more than 200 cases of Japanese women achieved an about 95% (86%) true-positive rate with 0.61 (0.55) false-positive masses (clusters) per image. It was found that the automated method has the potential to aid physicians in screening mammograms for breast tumors. Initial results for the mammograms digitized with the pixel sizes of 25, 50, and 100 micrometers are also discussed, in which a genetic algorithm (GA) technique was applied to the detection filter for the microcalcifications. It was indicated from the experiment with a breast phantom that 100- micrometers pixel size is not enough for the computer detection of microcalcifications, and it seems that at least 50-micrometers pixel size is required.

The specific problem of compensating for the presence of speckle in ultrasound B-mode imaging systems is addressed. The basic principles underlying current approaches to speckle reduction are first briefly reviewed. For real-time imaging systems, speckle reduction techniques which operate in the context of a single image (or `look') of the object are of more practical importance. A large number of such techniques have been developed which are based on some form of noise model of speckle, and are hence statistical in nature. All such approaches reduce the presence of speckle at the cost of some other factor of image quality, usually resolution. This paper examines an alternative approach to the single image problem, based on an alternative speckle definition, which does not suffer the usual resolution trade-off. The basis of the approach is to reformulate the single image problem in terms of a structure uniqueness problem which underpins a speckle model based on zeros of the analytic continuation of the rf A-line signal into the complex time and frequency planes.

The clinical use of image processing requires both medical knowledge and expertise in image processing techniques. We have designed a knowledge-based interactive quantification support system (IQSS) to help the medical user in the use and evaluation of medical image processing, and in the development of specific protocols. As the user proceeds according to a heuristic and intuitive approach, our system is meant to work according to a similar behavior. At the basis of the reasoning of our monitoring system, there are the semantic features of an image and of image processing. These semantic features describe their intrinsic properties, and are not symbolic description of the image content. Their obtention requires modeling of medical image and of image processing procedures. Semantic interpretation function gives rules to obtain the values of the semantic features extracted from these models. Then, commonsense compatibility rules yield to compatibility criteria which are based on a partial order (a subsumption relationship) on image and image processing, enabling a comparison to be made between data available to be processed and appropriate image processing procedures. This knowledge-based approach makes IQSS modular, flexible and consequently well adapted to aid in the development and in the utilization of image processing methods for multidimensional and multimodality medical image quantification.

Image processing techniques using wavelet signal analysis techniques have shown promise in mammography. Wavelet algorithms are compared with traditional image enhancement techniques of unsharp masking and median filtering. Computer simulated phantom images were generated containing lesions mimicking masses and microcalcifications. The degree of image enhancement was evaluated by comparing processed and original signal-to-noise (SNR) ratios in such phantom images. Results obtained in this study suggest that image processing algorithms based on the wavelet transform are likely to enhance the visibility of low-contrast features in mammograms.

In this paper we propose a novel method for boundary detection of vector fields. The derivation is based on the assumption that the observed vector fields are realizations of a spatially quasi-stationary process and that the vector observations at each pixel site are generated by a parametric probability distribution function. The form of the function is known but its parameters are not. To detect and estimate the locations of the boundaries, we use Bayesian theory and adopt the maximum a posteriori probability (MAP) criterion. It is shown that the criterion is a penalized maximum likelihood, which is composed of two terms. One is a data term that monotonically decreases as the number of hypothesized boundaries increases. The other term is a penalty that penalizes for the complexity of the model used to describe the data. The MAP solution is the one that minimizes the criterion. Simulation results are provided that show the performance of the proposed method. The results on synthesized magnetic resonance (MR) images demonstrate that this technique yields highly accurate estimates of the number of boundaries and their locations even for low contrast-to-noise ratios.

Extracting 3-D position of actin fibers is an important task in bio-medical research, since it can lead to a quantitative description of cell structure, motion and development. In this paper, we present a new approach to automatically extract these fibers from a 3-D stack of cell images obtained by three-dimensional fluorescence microscopy. Our algorithm incorporates two stages. First, the computer vision technique of depth-from-focusing is used to construct the 2-D best focused pseudo-image and the depth map from the original images. Second, we track actin fibers using the 2-D pseudo-image and the depth map based on a dense local filtering strategy and continuity constraints. A group of specially designed local line feature filters and continuity constraints are used to guide the tracking. Also we applied a backtracking strategy to refine the fiber searching when ambiguities occur. Finally, the systematic method of searching for all fibers in an image is presented.

The paper presents a method for automatic localization and feature extraction of white blood cells (WBCs) with color images to develop an efficient automated WBC counting system based on image analysis and recognition. Nucleus blobs extraction consists of five steps: (1) nucleus pixel labeling; (2) filtration of nucleus pixel template; (3) segmentation and extraction of nucleus blobs by region growing; (4) removal of uninterested blobs; and (5) marking of external and internal blob border, and holes pixels. The detection of nucleus pixels is based on the intensity of the G image plane and the balance between G and B intensity. Localized nucleus segments are grouped into a cell nucleus by a hierarchic merging procedure in accordance with their area, shapes and conditions of their spatial occurrence. Cytoplasm segmentation based on the pixel intensity and color parameters is found to be unreliable. We overcome this problem by using an edge improving technique. WBC templates are then calculated and additional cell feature sets are constructed for the recognition. Cell feature sets include description of principal geometric and color properties for each type of WBCs. Finally we evaluate the recognition accuracy of the developed algorithm that is proved to be highly reliable and fast.

Prefrontal cortex volumetry by brain magnetic resonance (MR) is required to estimate changes postulated to occur in certain psychiatric and neurologic disorders. A semiautomated method with quantitative characterization of its performance is sought to reliably distinguish small prefrontal cortex volume changes within individuals and between groups. Stereological methods were tested by a blinded comparison of measurements applied to 3D MR scans obtained using an MPRAGE protocol. Fixed grid stereologic methods were used to estimate prefrontal cortex volumes on a graphic workstation, after the images are scaled from 16 to 8 bits using a histogram method. In addition images were resliced into coronal sections perpendicular to the bicommissural plane. Prefrontal cortex volumes were defined as all sections of the frontal lobe anterior to the anterior commissure. Ventricular volumes were excluded. Stereological measurement yielded high repeatability and precision, and was time efficient for the raters. The coefficient of error was <EQ 0.03. The overall 3-way inter- rater ICC equals 0.95; intra-rater ICCs equals 0.95 - 0.98. The use of specific internal landmarks to define prefrontal cortex boundaries on 3D images was critical to obtaining accurate measurements. MR prefrontal cortex volumetry by stereology can yield accurate and repeatable measurements. Small frontal lobe volume reductions in patients with brain disorders such as depression and schizophrenia can be efficiently assessed using this method.

Accurate image segmentation is one of the key problems in computer vision. In domains such as radiation treatment planning, dosimetrists must manually trace the outlines of a few critical structures on large numbers of images. Considerable similarity can be seen in the shape of these regions, both between adjacent slices in a particular patient and across the spectrum of patients. Consequently we should be able to model this similarity and use it to assist in the process of segmentation. Previous work has demonstrated that a constraint-based 2D radial model can capture generic shape information for certain shape classes, and can reduce user interaction by a factor of three over purely manual segmentation. Additional simulation studies have shown that a probabilistic version of the model has the potential to further reduce user interaction. This paper describes an implementation of both models in a general-purpose imaging and graphics framework and compares the usefulness of the models on several shape classes.

This paper describes our new medical imaging processing system using fuzzy chips, which is capable of generating an image of contour of tissue, such as liver, carotid artery, and hepatic duct. Fuzzy inference method was used in extracting contour image of such structures out of the original image from MRI. Our previous system was built on a personal computer equipped with a fuzzy chip, while our new system reported in this paper has a graphic workstation equipped with a specially made fuzzy inference board having multiple fuzzy chips. This change has achieved great enhancement in processing speed. The fuzzy inference board was connected via VME bus to a graphic workstation, which controls flow of data to and from the fuzzy inference board, preprocessing of original medical image data, and final stage of determining contour of interested structures.

Since both detection and classification in an unsupervised stochastic model-based segmentation technique require the results from estimation, model parameter estimation becomes a core part of this type of technique. Based on FNM model, ML likelihood equations and their solutions have been derived. After investigating the Bayesian probability in these solutions and comparing it with probability membership of EM algorithm, this paper proved that both CM and EM algorithms which are used in this segmentation technique do produce asymptotically unbiased ML estimates of FNM parameters, through the iterative process, when the stopping criteria are satisfied. The evolution of probability membership in the iterative process finally leads to the pixel labeling. The corresponding Cramer-Rao bounds of the variances of these estimates are also derived. The results by applying this performance analysis method to the simulated images with signal-to-noise (SNR) as parameter are obtained and reported in this paper. These results show that for the images with the moderate quality, the parameter estimates are quite accurate. When SNR >= 14.5 db, all estimates are within the one- standard deviation interval.

The detection after surgery of residual tumor from magnetic resonance (MR) images is difficult due to the low contrast level of the images. Gadolinium-enhanced MR imaging has been found valuable in detecting residual enhancing tumor when performed within 72 hours after surgery. The patient is scanned by the MR scanner with and without infusion of gadolinium, a contrast agent. Usually, the estimation of post-operative tumor volume is done by visual comparison of the T1 MR images obtained with and without gadolinium infusion. The T1 MR images, in most cases, without contrast demonstrates areas of hyper intensities (high brightness levels), consistent with hemorrhage. These hyper intense areas often make it difficult to detect residual tumor in post contrast images. This is due to the presence of both acute hemorrhage and gadolinium enhancement which have high brightness levels in T1 MR images. Even in MR images taken within 72 hours after surgery, detection of tumor enhancement in areas of increased T1 signal produced by blood products or by postoperative changes can be difficult when performed by the naked eye. Due to these problems, the quantification of residual tumor becomes a subjective issue among neuro-radiologists. Thus to reduce errors produced by the human factor, an automated procedure to detect residual tumor is required. We have developed a technique to differentiate tumor enhancement from postoperative changes and blood products on MR imaging. The technique involves fusion of pre- and post-gadolinium MR images performed in the immediate postoperative period. Computerized slice based substraction is then done on the corresponding fused images of the two sets. The subtraction process results in a composite slice, which is examined for differences between pre- and post-gadolinium studies. The presented technique was tested on 14 cases in which MR images were obtained from brain tumor patients within 72 hours after surgery. The subtraction technique easily distinguished residual enhancing tumor from postoperative surgical changes and was simple to perform. The technique proposed and developed has given good results and will be used in clinical trial and diagnosis. Future potentials of the technique are discussed and illustrative cases presented.

A segmentation algorithm devoted to extracting heart and vessel walls on ultrasound image sequences was developed. This algorithm is based on a variety of active contours called Snake-Splines. It addresses the specific problems of ultrasonography (texture, dropouts) while using all the information available from temporal redundancy in a scheme of sequence processing for motion analysis. Active contours are 2D deformable models defined as energy- minimizing curves, including internal and external energy terms. The internal energy refers to smoothness constraints characteristics of the curve (controlled continuity splines), whereas the external one refers to forces computed from the gray-level image. A contour B-spline parametrization which intrinsically provides smoothness properties helps formulate the numerical problem with the only consideration of external forces governing the spline curve deformation. The algorithm is tested either on apical and parasternal transthoracic views, or transoesophageal or again intravascular ones, directly digitized at the video output or after recording on video-tapes. However, the methodology can be applied to all structures. The results show the necessity to deal with adaptive iterative process as well as problems associated with numerical stability, and the methodology is currently being extended to tracking structures through image sequences.

In this paper, we apply geodesic paths (the shortest path which is an important object in the task of 3D shape analysis) to the description of natural textures. We first introduce an approach called propagation algorithm to estimate the shortest paths on discrete surfaces and then derive a set of texture features (LR, PLR, SDR, DIF and 1c) from them. The results of the experiments indicate that these features can reflect texture surface patterns in various aspects and are suitable for texture description.

Neural networks are applied to the Rayleigh discrimination task. Network performance is compared to results obtained previously using human viewers, and to the best machine approximation to the ideal observer found in an earlier investigation. We find that simple preprocessing of the input image, in this case by projection, greatly improves network convergence and only results obtained on projections are presented here. It is shown that back propagation neural networks significantly outperform a standard nonadaptive linear machine also operating on the projections. In addition, this back propagation neural network performs competitively to a nonadaptive machine that uses the complete two-dimensional information, even though some relevant information is destroyed in the projection process. Finally, improved performance on this Rayleigh task is found for nonlinear (over linear) neural network decision strategies.

A convolution neural network (CNN) and a backpropagation neural network (BPN) were used for classification of regions of interest (ROIs) on mammograms as either mass or normal tissue. Input images to the CNN were obtained from the ROIs using averaging and subsampling. Input features to the BPN were obtained from spatial gray level dependence (SGLD) matrices at multiple resolutions. A data set consisting of 168 ROIs containing biopsy- proven masses and 504 ROIs containing normal breast tissue was used for training and testing of the neural networks. Receiver operating characteristic (ROC) methodology was used to evaluate the classification accuracy. The area under the test ROC curve reached 0.83 for the CNN, 0.88 for the BPN, and 0.89 for a combination of the CNN and BPN outputs. Our results indicate that (1) CNN performance may be improved by using additional texture information; and (2) the overall performance may be improved by combining CNN and BPN classifiers.

Artificial neural networks have been applied to the differentiation of masses from false- positive detections in digital mammograms. A database of 110 pairs of digital mammograms containing a total of 102 masses (54 malignant, 48 benign) was utilized in this study. Three- hundred-two false positive regions were selected from these images to be used in the training of the artificial neural network. Over 90 features were calculated for both the true masses and the false positives. Features that showed the most separation, in a one-dimensional analysis, between true positives and false positives were selected for artificial neural network input. A three level feed-forward neural network was used with one input layer, one hidden layer and an output layer. By varying the structure and learning rate of the neural network an optimal structure was found. The performance of the ANN was evaluated by means of receiver operating characteristic (ROC) analysis and free-response receiver operating characteristic (FROC) analysis. Results from a round robin evaluation yielded an Az of 0.97 in the task of differentiating between masses and false-positive detections. In the future, multi-dimensional feature analysis will be performed to obtain the optimal performance using a combination of rule-based decision making along with artificial neural networks.

Mammographic stellate lesions and architectural distortions are usually associated with malignancy, which makes them important signs in the screening for breast cancer. This paper proposes a novel method for the automatic detection of these lesions with the use of a Hough spectrum based geometric texture analysis. The Hough spectrum technique is developed from the traditional Hough transform, which is effective in the description of geometric structures. The Hough spectrum inherits such an ability and extends it into texture analysis for the description of those texture features geometric in nature, such as the spicula of a stellate lesion. When applied in mammographic image processing, an ROI is first transformed into its Hough spectrum, and then thresholding is conducted with its threshold level based on the statistical properties of the spectrum. Those ROIs with strong signals of spiculation are then identified as regions of potential lesions. Our preliminary studies, using only the Hough spectrum based technique without further feature analyses to reduce false positive detection, have yielded sensitivities of 81% for spiculated masses and 67% for architectural distortions at false positive rates of 0.97 and 2.2 per image, respectively. This manifests clearly the potential of the developed technique in the detection of such types of mammographic lesions.