The optical structure sampled by the human observer is insufficient to determine the structure of a scene. The equivalence class of scenes that lead to the same optical structure can be worked out precisely for specific 'cues' (shading, texture, ...). If the 'observed scene' is a member of the correct class, the observation must be considered 'veridical,' even if the observed scene differs from the actual one. In many cases it is impossible to indicate the equivalence classes, and it therefore must remain undecided whether observations that deviate from physical reality should be denoted 'veridical' or not. For observations on the basis of images, such as straight photographs of physical scenes, the equivalence classes are unknown, but are certainly large. This case is especially important in the design of computer interfaces where a scene is being presented to the user as an image. We find that observations differ appreciably according to the precise task. The observer uses the freedom resulting from the iconic underdetermination to choose some idiosyncratic perspective by directing the 'mind's eye.' This can be demonstrated with simple means. Stretchings of depth by a few hundred percent and changes in viewing direction (not rotations, but shears) of tens of degrees are quite common.

It has been found that the L* function defined in the CIELAB color space is not suitable to predict the human visual perception of modulated patterns at high spatial frequencies. For example, in multilevel halftoning (multitoning), when output levels are equally spaced in L*, it has been observed that the visibility of the resulting multitone patterns is not uniform across different parts of the tone scale. This leads to the hypothesis that the CIE L* function may not be a good metric to evaluate the perceived lightness differences at high-spatial frequencies as it was derived based on the perception of large area uniform patches. To investigate the relationship between suprathreshold lightness difference perception with regard to spatial frequency and amplitude modulation, we designed a psychophysical experiment, which was conducted using a lightness difference matching paradigm. The stimuli used in the experiment were horizontal square-wave gratings. The behavior of lightness difference perception under varying spatial frequencies and modulation amplitudes across the entire L* scale was studied. Consistent results were acquired that show a significant frequency-dependent effect where the effective lightness difference for high- frequency patterns is reduced for low L* values. The magnitude of this effect was found to be highly related to the spatial frequency of the modulation. Based on these results, we derived an effective lightness function that is dependent on spatial frequency. The effective lightness function can be applied to the selection of the output levels for multitoning.

We describe results of experiments studying the tradeoff between resolution and bit depth. Images in the experiments were printed on a high resolution imagesetter, eliminating most, if not all, device effects. They were first converted from PostScript to antialiased rasters at one of a set of resolutions, then converted from 8 bit to n bits for some value of n less than 8. Before printing, they were converted back to 8 bits and scaled up to printer resolution, then halftoned for printing. We were measuring human response to a system that had a bandwidth bottleneck somewhere upstream of the printer, and sophisticated resampling and halftoning at the printer itself. The images, typical of those used for critical evaluation of hard copy, contained text, analytical test targets, synthetic graphics and pictorial images. We found bit depths and resolutions beyond which no further improvement was observed, typically somewhat higher limits than previously believed. We also compared methods of font hinting for antialiased text, and found that font hinting improves text only at one bit per pixel, degrading it at higher bit depths.

The design of reliable visual quality metrics is complicated by our limited knowledge of the human visual system and the resulting variety of pertinent vision models. We have begun to analyze and compare a number of implementation choices for some components found in most of today's visual quality metrics that are based on a model of human vision and present the first results here.

In this paper we evaluate two objective quality measures, the root-mean-square-error and a model based on the human visual system (HVS), on their ability to predict the perceived image quality for variations in bit-rate, processing method, and scene content. In theory quality metrics should be able to predict the perceived image quality independent of these variations. However, one can imagine that in practice this is not trivial to meet. But also subjects might have difficulties in making comparisons across processing methods or across scenes. In order to test whether subjects use separate quality scales for each identifiable scene and processing method or whether they use a single quality scale, we set up experiments in which the influence of bit-rate, processing method, and scene content was measured. In all experiments subjects were instructed to judge the quality difference between two simultaneously presented images.

In order to evaluate the visual quality of images, most methods compare a degraded image to a perfect reference. We propose an original univariant (i.e. without reference) method based on the use of artificial neural networks. The principle is first to use a neural network to learn the quality of images taken from a pool of known examples, then use it to assess the quality of unknown images. The considered defects are compression artefacts, ringing or local singularities. To simplify, only images with defects that are not mixed with each other were first used. The method follows four steps. Observers are first required to mark degraded images to establish a pool of examples. Then, a characterization of the defect is extracted mathematically from the image. Then, the neural network learns how to establish a relation between the mathematical characterization of the defect and the visual mark. Finally, it can be used to assess the visual quality of an unknown image from the mathematical characterization of its defects. Two illustrative examples are presented: the assessment of the quality of JPEG compressed images and the detection of local defects.

Data compression of digitized images usually leads to pictures with two or more perceptually distinguishable coding artifacts. In this paper, percentage scaling is presented as a new method for analyzing the perceived quality of multiply impaired images. By this method, subjects assess, per coding level, the proportion each artifact contributes to the overall impairment of the perceived quality of the coded image. These proportions are expressed in percentages such that the total sum is always 100%. As an illustration, percentage scaling has been employed to evaluate the perceived quality of JPEG-coded images comprising three types of coding artifacts: blockiness, ringing and blur. In one experiment, eight subjects expressed in percentages how much they believed these artifacts contributed to the overall impairment. In addition, overall impairment as well as the perceived strengths of blockiness, ringing and blur was assessed on 11-point numerical category scales. The percentages were found to vary with the coding level. Blockiness appeared to be the most dominant artifact at high compression levels, whereas ringing turned out to be the most annoying artifact at low compression levels. The results support the claim that perceptually distinguishable coding artifacts can be represented by a set of orthogonal vectors in a multidimensional Euclidean space.

We present algorithms for predicting the quality of color reproductions of natural scenes. The algorithms are based on a three-point concept for image quality: (1) images are carries of visual information about the outside world and the objects located in it, (2) images are used by the visual and cognitive systems to reconstruct and interpret the outside world, and (3) the quality of an image is the degree of success with which this image can be used by the visual and cognitive systems.

Four perceptually based metrics were compared for use in studies of video bandwidth compression in a military context. They include the National Imagery Interpretability Rating Scale (NIIRS), a task satisfaction confidence scale (TSC), an information conveyance scale (IC), and an artifact rating (AR) scale. The metrics were used in two compression studies using both MPEG-2 and frame decimation. The studies used video imagery collected with an unmanned aerial vehicle (UAV). The metrics were compared in terms of sensitivity and reliability. Results were also compared to those from other studies employing the ITU-R BT. 500-9 double-stimulus continuous quality-scale (DSCQS) in terms of both findings and metric performance. DSCQS is the metric commonly used in studies of bandwidth compression for commercial applications. It was concluded that for studies of video quality for military applications, NIIRS was the preferred perceptual metric. It was shown to be applicable to video, was comparable to DSCQS in reliability and sensitivity, and is a metric most familiar to the intelligence community.

Many approaches have been proposed recently for estimating the perceived fidelity of digitally compressed and reconstructed video clips. Several metrics rely on the accurate estimation of detection thresholds, basically assuming that perceived quality depends significantly on the error threshold. To test that assumption, we designed an experiment to measure the actual detection threshold of some MPEG-2 artifacts in typical video sequences. In each of the test clips, we briefly replaced a region with a test stimulus created using an MPEG-2 encoder. The location, time, and strength of the artifact were varied between test videos. At the end of each clip, each subject provided a detection response and, for the detected artifacts, location and annoyance information. From the data, we determined the detection threshold for each artifact. Using the thresholds, we computed a perceptually weighted error metric. In this paper, we describe the experiment in more detail, summarize the experimental results including the threshold calculations, and compare the weighted error measure to the output of a commercial fidelity metric. Finally, we discuss our conclusions on the validity of predicting quality from threshold measurements.

In consumer photography, image appeal may be defined by the interest that a picture generates when viewed by third-party observers. In this paper, the results of a ground truth experiment on human estimation of image appeal are reported, where 11 participants were asked to rank pictures in 30 groups, based on their relative appeal within their group and comment on the factors that influenced their decisions. Based on their responses, a list of both positive and negative influences was compiled and influences were grouped in general categories that include people, composition, subject, and objective metrics. The results of our experiment indicate that image appeal is related to image quality only with respect to the influences in the category of objective metrics, while the majority of influences belong to the categories of people, composition, and subject. The influences in these categories are scene dependent and fundamentally different in nature from traditional image quality metrics. Thus, when evaluating image appeal a new set of metrics needs to be developed. Individual influences and their relative merits are discussed.

With the advent of digital television transmission, measurements to insure quality of service have expanded to include models of the human vision system. This paper reviews international and North American broadcast-video standardization activities. A description of the scopes and related work in such organizations as the Society of Motion Picture and Television Engineers, the Institute of Electrical and Electronics Engineers and the International Telecommunication Union provides prospective for the developing field of objective television picture quality measurement methods.

Subjective assessment methods have been used reliably for many years to evaluate video quality. They continue to provide the most reliable assessments compared to objective methods. Some issues that arise with subjective assessment include the cost of conducting the evaluations and the fact that these methods cannot easily be used to monitor video quality in real time. Furthermore, traditional, analog objective methods, while still necessary, are not sufficient to measure the quality of digitally compressed video systems. Thus, there is a need to develop new objective methods utilizing the characteristics of the human visual system. While several new objective methods have been developed, there is to date no internationally standardized method. The Video Quality Experts Group (VQEG) was formed in October 1997 to address video quality issues. The group is composed of experts from various backgrounds and affiliations, including participants from several internationally recognized organizations working in the field of video quality assessment. The majority of participants are active in the International Telecommunications Union (ITU) and VQEG combines the expertise and resources found in several ITU Study Groups to work towards a common goal. The first task undertaken by VQEG was to provide a validation of objective video quality measurement methods leading to Recommendations in both the Telecommunications (ITU-T) and Radiocommunication (ITU-R) sectors of the ITU. To this end, VQEG designed and executed a test program to compare subjective video quality evaluations to the predictions of a number of proposed objective measurement methods for video quality in the bit rate range of 768 kb/s to 50 Mb/s. The results of this test show that there is no objective measurement system that is currently able to replace subjective testing. Depending on the metric used for evaluation, the performance of eight or nine models was found to be statistically equivalent, leading to the conclusion that no single model outperforms the others in all cases. The greatest achievement of this first validation effort is the unique data set assembled to help future development of objective models.

A robust model of the human visual system (HVS) would have a major practical impact on the difficult technological problems of transmitting and storing digital images. Although most HVS models exhibit similarities, they may have significant differences in predicting performance. Different HVS models are rarely compared using the same set of psychophysical measurements, so their relative efficacy is unclear. The Modelfest organization was formed to solve this problem and accelerate the development of robust new models of human vision. Members of Modelfest have gathered psychophysical threshold data on the year one stimuli described at last year's SPIE meeting. Modelfest is an exciting new approach to modeling involving the sharing of resources, learning from each other's modeling successes and providing a method to cross-validate proposed HVS models. The purpose of this presentation is to invite the Electronic Imaging community to participate in this effort and inform them of the developing database, which is available to all researchers interested in modeling human vision. In future years, the database will be extended to other domains such as visual masking, and temporal processing. This Modelfest progress report summarizes the stimulus definitions and data collection methods used, but focuses on the results of the phase one data collection effort. Each of the authors has provided at least one dataset from their respective laboratories. These data and data collected subsequent to the submission of this paper are posted on the WWW for further analysis and future modeling efforts.

In Year One of the Modelfest project, several laboratories collaborated to collect threshold data of human observers on 45 pattern stimuli. In this preliminary study, we used a principal component analysis (PCA) and a confirmatory factor analysis on the variations among observers to explore the underlying visual mechanisms for detecting Modelfest Stimuli. This analysis is based on the assumption that there are channels in common among observers that are represented with variations in sensitivity level only. We found three principal components. Assuming that each principal component represents a single mechanism, we compute the sensitivity profile of each mechanism as the sum of test stimuli weighted by the factor loadings on each component. The first mechanism is a spot detector. The second mechanism is dominated by a horizontal periodic pattern around 4 c/deg and the third may be characterized as a narrow bar detector.

Development of video quality metrics has taken support from experimental vision data mainly at two levels of abstraction. On the one hand are the carefully controlled tests of human visual response to well-defined, controlled visual stimuli, such as the ModelFest study. On the other hand are experiments in which viewers rate the global quality of 'natural' video sequences exhibiting impairments of loosely-controlled composition and amplitude, as in the Video Quality Experts Group study. The IEEE Broadcast Technology Society Subcommittee on Video Compression Measurements has initiated an intermediate level approach to video quality assessment aimed toward developing a scale of video impairment and unit of measure by which to describe video distortion in both perceptual and engineering terms. The proposed IEEE study will attempt to define a scale of video impairment in terms of multiple measurements of the just-noticeable-difference (JND) of compression-induced video impairments. A paired comparison psychophysical method will be used to define a psychometric function of the visual sensitivity to compression-induced video impairments of various amplitudes. In this effort, quality assessment is related directly to visual perception of video impairments rather than to the more 'atomic' visual stimuli as used in many human vision experiments. Yet the experimenter's control over the stimuli is greater than that used in much of contemporary video quality testing.

In picture quality assessment, the amount of distortion perceived by a human observer differs from one region to another according to its particular local content. This subjective perception can be explained/predicted by considering some simple psychovisual properties (masking) of the Human Visual System (HVS). We have implemented a HVS model based on a pyramid decomposition for extracting the spatial frequencies, associated with a multi-resolution motion representation. Then the visibility of the decoded errors is computed by exploiting the Kelly's contrast sensitivity spatio-velocity model. The resulting data is called a 'Quality-map.' Special attention has been paid to temporal/moving effects since, in the case of video sequences, motion strongly influences the subjective quality assessment. The quality of the motion information is thus preponderant. In the second part, two possible uses of these psychovisual properties for improving MPEG video encoding performances are depicted: (1) The pre-processing of the pictures to remove non-visible information using a motion adapted filtering. This process is efficient in term of bits saved and degradation is not significant especially on consumer electronic TV sets. (2) A perceptual quantizer based on a local adaptation scheme in order to obtain Quality-maps as uniform as possible (homogeneous perceived distortion), at constant bit-rate. Further improvements have been considered, especially when the viewer is tracking a moving object in the scene.

The image sequence discrimination model we use models optical blurring and retinal light adaptation. Two parallel channels, sustained and transient, with different masking rules based on contrast gain control, are used. Performance of the model was studied for two tasks representative of a video communication system with versions of monochrome H.263 compressed images. In the first study, five image sequences constituted pairs of non-compressed and compressed images to be discriminated with a 2-alternative-forced-choice method together with a staircase procedure. The thresholds for each subject were calculated. Analysis of variance showed that the differences between the pictures were significant. The model threshold was close to the average of the subjects for each picture, and the model thus predicted these results quite well. In the second study, the effect of transmission errors on the Internet, i.e. packet losses, was tested with the method of constant stimuli. Both reference and comparison image was distorted. The task of the subjects was to judge whether the presented video quality was worse than the initially seen reference video. Two different quality levels of the compressed sequences were simulated. The differences in the thresholds among the different video scenes were to some extent predicted by the model. Category scales indicate that detection of distortions and overall quality judgements are based on different psychological processes.

Models of human vision can be efficiently exploited to improve the visual quality of image compression schemes. Most of the psycho-visual experiments documented in literature use very basic test patterns to analyze specific low-level vision effects. More complex patterns of real world images and strong constrains of an image compression scheme result in a very rough mapping of this psycho-visual data to the real system. So it is no longer guaranteed that the predicted visibility of the distortions is correct. In this paper we present an experiment strongly linked to wavelet-based image compression. It evaluates the effect of contrast and entropy masking based on complex images of natural scenery. The test images were reconstructed from a wavelet-decomposition where the wavelet coefficients were quantized at specific locations. The visibility of these distortions is influenced in a first order by the contrast masking effect and in a second order by the complexity of the affected image sub-region. With the collected test data some models are compared in terms of prediction performance.

This paper describes the results of a psychophysical experiment to quantify sensitivities to supra-threshold distortion caused by wavelet coefficient quantization. Natural images were selected for their energy distributions, subject matter, and edge/detail content. A single subband was quantized and the quantization step size to produce a minimum noticeable distortion (denoted the MNDSS) was recorded; these step sizes are defined to produce a visual distortion unit. The test results indicate that the MNDSS are strongly correlated with band energy, and that at a single frequency, the VDUs for the horizontal and vertical orientations occur when the squared error in the image is constant. Because the quantizers are not operating in the granular region, however, this does not imply that equal step sizes in the two orientations at the same frequency produce equivalent visual distortion, in contrast with common sub-threshold quantization assumptions. Furthermore, contrast sensitivity curves based on these results indicate an increased sensitivity to the orientation with higher energy at each frequency; neither orientation is favored when there is no inequality in energy. The results provide evidence of both orientation-dependent and content-dependent masking in supra-threshold quantization.

Today, the use of objective quality metrics is well-known for the optimization of digital image processing systems. The work presented in this paper is about an algorithmic construction of an image quality criterion. This criterion takes into account the human visual system HVS properties in order to ensure the correspondence between objective measures given by the criterion and subjective notes given by a group of observers, and is decomposed functionally into three principal blocks. The first one corresponds to a perceptual image representation: a set of 17 frequential channels models the radial and angular selectivity of the HVS. The second block concerns the construction of the adaptation function of perception thresholds due to masking effects. Thanks to psychophysical experiments, the visibility thresholds of impairments are first measured in each individual channel, then in the presence of masking signals from other channels. The aim of this paper is to present these results and the masking model which takes into account both masking effects within channels and between channels. Finally in the third block, both frequential and spatial pooling are performed.

While problems of image coding and audio coding have frequently been assumed to have similarities, specific sets of relationships have remained vague. One area where there should be a meaningful comparison is with central masking noise estimates, which define the codec's quantizer step size. In the past few years, progress has been made on this problem in the auditory domain (Allen and Neely, J. Acoust. Soc. Am., 102, 1997, 3628-46; Allen, 1999, Wiley Encyclopedia of Electrical and Electronics Engineering, Vol. 17, p. 422-437, Ed. Webster, J.G., John Wiley & Sons, Inc, NY). It is possible that some useful insights might now be obtained by comparing the auditory and visual cases. In the auditory case it has been shown, directly from psychophysical data, that below about 5 sones (a measure of loudness, a unit of psychological intensity), the loudness JND is proportional to the square root of the loudness (Delta) pounds sterling(pounds sterling) varies direct as (root)pounds sterling(I). This is true for both wideband noise and tones, having a frequency of 250 Hz or greater. Allen and Neely interpret this to mean that the internal noise is Poisson, as would be expected from neural point process noise. It follows directly that the Weber fraction (the relative loudness JND), decreases as one over the square root of the loudness, namely (Delta) pounds sterling/pounds sterling varies direct as 1/(root)pounds sterling. Above pounds sterling equals 5 sones, the relative loudness JND (Delta) pounds sterling/pounds sterling approximately equals 0.03 (i.e., Ekman law). It would be very interesting to know if this same relationship holds for the visual case between brightness (Beta) (I) and the brightness JND (Delta) (Beta) (I). This might be tested by measuring both the brightness JND and the brightness as a function of intensity, and transforming the intensity JND into a brightness JND, namely (Delta) (Beta) (I) equals (Beta) (I + (Delta) I) - (Beta) (I) approximately equals (Delta) Id(Beta) /dI. If the Poisson nature of the loudness relation (below 5 sones) is a general result of central neural noise, as is anticipated, then one would expected that it would also hold in vision, namely that (Delta) (Beta) ((Beta) ) varies direct as (root)(Beta) (I). It is well documented that the exponent in the S.S. Stevens' power law is the same for loudness and brightness (Stevens, 1961) (i.e., both brightness (Beta) (I) and loudness pounds sterling(I) are proportional to I^0.3). Furthermore, the brightness JND data are more like Riesz's near miss data than recent 2AFC studies of JND measures.

Sensitivity to AV (auditory-visual) asynchrony was measured using brief tonal signals accompanied by a moving disk on a monitor. Stimuli were presented which either had synchronous audio and video or a positive or negative delay between audio and video. In a detection task, video delays of 29 ms and audio delays of 85 ms were just detectable. In a second experiment subjects had to judge whether the audio and the video were synchronous, whether audio preceded video, or the reverse. Results show a considerable variability in the range over which stimuli were judged to be synchronous. Typically, stimuli with audio delays in the range of -75 to 175 ms were judged synchronous. The distribution of synchronous responses had a mean of around 40 ms. The different results of the first and second experiment indicate that tolerance to AV delays is considerably larger than the just noticeable AV delay. In a next set of experiments several subsequent AV events were presented with variable amounts of jitter in the AV timing. Results of an experimental condition with a constant amount of overall AV delay indicate that the jitter was highly detectable (thresholds as low as 40 ms) even if an overall delay of up to 100 ms was presented. An additional experiment with a variable number of subsequent AV events shows that detecting an asynchrony in one AV event is more difficult than detecting jitter in multiple AV events.

This paper studies two different 'Diamond Wall' experiments. It demonstrates that 'Early Vision' explanations are possible, without reliance on apparent illumination, transparency, apparent depth and junctions. Grays ordinarily look darker on light backgrounds. Adelson showed that two rows of diamonds looked the same on different backgrounds when the diamond tips crossed into the other background. He added different gray tips, consistent with illumination changes, thus releasing the rest of the diamonds to no-longer looking the same. Grays on light backgrounds reverted to looking darker. New experiments show that the introduction of any edge along the light-dark background boundary releases the diamonds to look different in different surrounds. Whereas Adelson introduced diamond tips consistent with illumination, we used lines, dots, Cornsweet edges and non-consistent tips to release the diamonds to look different. Apparent illumination and apparent depth were not necessary because small abstract displays (one-quarter diamond) affected lightnesses the same as the complex displays. We apply complex 'Early Vision' models to images associated with 'High Vision' hypotheses. The argument is that flat displays, which are perceived as flat, require a quite complex visual mechanism just to account for the properties of flat lightnesses. Any experimental verification of the existence of 'High Vision' lightness mechanisms should be tested first with realistic flat 'Early Vision' models. The results show that different 'Early Vision' approaches account for these 'Diamond Wall' experiments. If 'Early Vision' mechanisms can account for the results, then these experiments cannot be used as evidence for the existence of 'High Vision' mechanisms.

The role of the scanpath in perception, especially in visual imagery experiments, raises questions regarding 'the mind's eye' neurological representation. The structural, symbolic, and sequential bindings required to generate a scanpath are likely located in cortical modules with appropriate connections. The results of symbolic experiments, designed to study these connections, shed light on the top-down visual memory processes.

Our laboratory uses image perception studies to optimize the acquisition and processing of image sequences from x-ray fluoroscopy and interventional MRI (iMRI) both of which are used to guide complex minimally invasive treatments of cancer and vascular disease. Fluoroscopy consists of high frame rate, quantum-limited image sequences. Since it accounts for over half of the diagnostic population x-ray dose, we attempt to reduce dose by optimizing image acquisition and filtering. We quantify image quality using human detection experiments and modeling. Human spatio-temporal processing greatly affects results. For example, spatial noise reduction filtering is significantly more effective on image sequences than on single image frames where it gives relatively little improvement due to the deleterious effect of spatial noise correlation. At CWRU, we use iMRI to guide a radio-frequency probe used for the thermal ablation of cancer. Improving the speed and accuracy of insertion to the target will reduce patient risk and discomfort. We are investigating keyhole imaging whereby one updates only a portion of the Fourier domain at each time step, producing a fast, approximate image sequence. To optimize the very large number of techniques and parameters, we use a perceptual difference model that quantifies the degrading effects introduced by fast MR imaging, including the blurring of interventional devices. Preliminary studies show that a perpendicular frequency encoding direction provides superior image quality in the region of interest compared to other keyhole stripe orientations. Together these two applications illustrate that image perception studies can impact the design of medical imaging systems.

The goal of this research was to understand the influence of experience on how radiologists search medical images for lesions and how search is affected by providing computer-aided detection prompts. Two studies will be discussed. The first study recorded the eye-position of 6 radiologists with different amounts of experience as they searched 20 mammography cases for lesions. Detection performance was measured using Receiver Operating Characteristic (ROC) analysis and visual search patterns were characterized. The second study recorded eye-position as radiologists with different degrees of experience searched mammograms with and without lesion prompts. Radiologists with more experience tend to find lesions faster and have more efficient search patterns than those with less experience. Positive decisions for all observers have longer dwells than negative decisions. Missed lesions receive longer dwells than lesion free areas. Experienced radiologists perform more complete searches prior to prompt availability and access prompts later in search than those with less experience. Visual search is influenced significantly by amount of training and experience reading specific types of images. This skill may or may not be easily transferable from/to different types of images. This is especially important when computer aids are used to help the radiologist detect lesions by providing prompts on the image.

Different shape representation and classification methods for complex medical lesions were compared using oral lesions as a case study. The problem studied was the discrimination between potentially cancerous lesions, called leukoplakia, and other usually harmless lesions, called lichenoid reactions, which can appear in human oral cavities. The classification problem is difficult because these lesions vary in shape within classes and there are no easily recognizable characteristics. The representations evaluated were the centroidal profile function, the curvature function, and polar and complex coordinate functions. From these representations, translation, scale and rotation independent features were derived using Fourier transformations, auto-regressive modeling, and Zernike moments. A nonparametric kNN classifier with the leave-one-out cross-validation method was used as a classifier. An overall classification accuracy of about 84% was achieved using only the shape properties of the lesions, compared with a human visual classification rate of 65%. The best results were obtained using complex representation and Fourier/Zernike methods. In clinical practice, the preliminary diagnosis is based mainly on the visual inspection of the oral cavity, using both color, shape and texture as differentiating parameters. This study showed that machine analysis of shape could also play an important part in diagnosis and decisions regarding future treatment.

We describe a low-impact method for coloring interfaces harmoniously. The method uses a model that characterizes the overall image including the need for distinguishability between interface components. The degree of visual distinction between one component and other components, and its color strength (which increases with its importance and decreases with its size and longevity), are used in generating a rigid ball-and-stick 'color molecule,' which represents the color relationships between the interface components. The shape of the color molecule is chosen to conform to standard principles of color harmony (like colors harmonize, complementary colors harmonize, cycles in the color space harmonize, and so on). The color molecule's shape is fixed, but its position and orientation within the perceptually uniform color solid are not. The end user of the application chooses a new color scheme for the complete interface by repositioning the molecule within the color space. The molecule's shape and rigidity, and the space's perceptual uniformity, ensures the distinguishability and color harmony of the components are maintained. The system produces a selection of color schemes which often include subtle 'nameless' colors that people rarely choose using conventional color controls, but which blend smoothly into a harmonious color scheme. A new set of equally harmonious color schemes only requires repositioning the color molecule within the space.

Many automated quality inspection systems make use of brightness and contrast features of the objects being inspected. This reduces the complexity of the problem solving methods, as well as the demand for computational capacity. Nevertheless a lot of significant information is located in color features of the objects. This paper describes a method, that allows the evaluation of color information in a very compact and efficient way. The described method uses a combination of multi-valued logic and a special color model. We use fuzzy logic as multi-valued logic, and the HSI color model, but any multi-valued logic, that allows rule-based reasoning can be used. The HSI color model can also be exchanged with other color models, if special demands require this.

In visualization, a 'color map' maps a range of data values onto a scale of colors. However, unless a color map is e carefully constructed, visual artifacts can be produced. This problem has stimulated considerable interest in creating perceptually based color maps, that is, color maps where equal steps in data value are perceived as equal steps in the color map [Robertson (1988); Pizer (1981); Green (1992); Lefkowitz and Herman, 1992)]. In Rogowitz and Treinish, (1996, 1998) and in Bergman, Treinish and Rogowitz, (1995), we demonstrated that color maps based on luminance or saturation could be good candidates for satisfying this requirement. This work is based on the seminal work of S.S. Stevens (1966), who measured the perceived magnitude of different magnitudes of physical stimuli. He found that for many physical scales, including luminance (cd/m²) and saturation (the 'redness' of a long-wavelength light source), equal ratios in stimulus value produced equal ratios in perceptual magnitude. He interpreted this as indicating that there exists in human cognition a common scale for representing magnitude, and we scale the effects of different physical stimuli to this internal scale. In Rogowitz, Kalvin, Pelahb and Cohen (1999), we used a psychophysical technique to test this hypothesis as it applies to the creation of perceptually uniform color maps. We constructed color maps as trajectories through three-color spaces, a common computer graphics standard (uncalibrated HSV), a common perceptually-based engineering standard for creating visual stimuli (L*a*b*), and a space commonly used in the graphic arts (Munsell). For each space, we created color scales that varied linearly in hue, saturation, or luminance and measured the detectability of increments in hue, saturation or luminance for each of these color scales. We measured the amplitude of the just-detectable Gaussian increments at 20 different values along the range of each color map. For all three color spaces, we found that luminance-based color maps provided the most perceptually- uniform representations of the data. The just-detectable increment was constant at all points in the color map, with the exception of the lowest-luminance values, where a larger increment was required. The saturation-based color maps provided less sensitivity than the luminance-based color maps, requiring much larger increments for detection. For the hue- based color maps, the size of the increment required for detection varied across the range. For example, for the standard 'rainbow' color map (uncalibrated HSV, hue-varying map), a step in the 'green' region required an increment 16 times the size of the increment required in the 'cyan' part of the range. That is, the rainbow color map would not successfully represent changes in the data in the 'green' region of this color map. In this paper, we extend this research by studying the detectability of spatially-modulated Gabor targets based on these hue, saturation and luminance scales. Since, in visualization, the user is called upon to detect and identify patterns that vary in their spatial characteristics, it is important to study how different types of color maps represent data with varying spatial properties. To do so, we measured modulation thresholds for low-(0.2 c/deg) and high-spatial frequency (4.0 c/deg) Gabor patches and compared them with the Gaussian results. As before, we measured increment thresholds for hue, saturation, and luminance modulations. These color scales were constructed as trajectories along the three perceptual dimensions of color (hue, saturation, and luminance) in two color spaces, uncalibrated HSV and calibrated L*a*b. This allowed us to study how the three perceptual dimensions represent magnitude information for test patterns varying in spatial frequency. This design also allowed us to test the hypothesis that the luminance channel best carries high-spatial frequency information while the saturation channel best represents low spatial-frequency information (Mullen 1985; DeValois and DeValois 1988).

Color misregistration is one of the common artifacts for 3-CCD desktop scanners. The misregistration of red, green and blue image layers causes both color fringing and blur in the scanned images. These effects are quantified by linear system theory analysis. Knowing the bandwidth and peak sensitivity asymmetries in the opponent color representation of the visual system, we developed a method to reduce the color misregistration artifact by attempting to capture signals in an approximate opponent color space. A new sensor arrangement facilitates this goal, in which the luminance and chrominance signals are captured independently. The luminance signal (Y) is captured at the full resolution using one row of the 3-row CCD linear arrays. The first chrominance signal is captured on another row with an interleaved half resolution red (R) and half resolution luminance sensor elements, and the second chrominance signal is similarly captured on a third row using blue (B) and luminance (Y). Since each luminance and chrominance signal is isolated on a single row, and since there is no registration error within a row, color misregistration is theoretically prevented in luminance as well as in the chrominance signals. Simulation shows that the new method does eliminate the blur and reduces the visibility of color fringing. Since residual luminance and chrominance misregistration may occur, a psychophysical experiment was conducted to judge the improvement in the scanned image quality. The experiment shows that this new capture scheme can significantly reduce the perception of misregistration artifacts.

Methods for processing gray scale images cannot be simply extended to color images because they rely on the fact that each gray level is described by a single number and the fact that a relation of full order can be easily established on the set of those numbers. Color spaces (such as RGB or Lab) are on the other hand three dimensional. To make them more amenable to the type of processing used for one-dimensional gray level pictures, we propose a particular space sampling and numbering of sampled points. The sampling scheme provides a uniform quantization of the Euclidean Lab space, and a way to establish a partial order relation on the set of points. The idea is based on several results from number theory and phyllotaxy. We illustrate possible processing of color images by first introducing the basic average and difference operations and then implementing edge detection and compression of color quantized images.

This paper is an attempt to develop a coherent framework for understanding, modeling, and computing color categories. The main assumption is that the structure of color category systems originates from the statistical structure of the perceived color environment. This environment can be modeled as color statistics of natural images in some perceptual and approximately uniform color space (e.g., the CIELUV color space). The process of color categorization can be modeled as the grouping of the color statistics by clustering algorithms (e.g., K-means). The proposed computational model enable to predict the location, order, and number of color categories. The model is examined on the basis of K-means clustering analysis of statistics of 630 natural images in the CIELUV color space. In general, the predictions are consistent with Berlin and Kay, and Boynton and Oslon data.

Military commanders must understand the complexities of the tactical picture, which is made up of many individual entities (microstates) with their own organizational allegiance, capabilities, and attributes. Further, while individual events can occur quickly, a battle can occupy several hours to days. It is inadequate to simply perceive the differentiated data about battlefield entities at some point in time -- the commander must understand the integrated meaning of the entities as they interact across time. We say that the commander is building situation awareness (SA), and doing so using traditional information displays consumes scarce resources of time and cognitive effort. The challenge is to combine the many individual microstates into a few integrated macrostates and display this information to the commander in a way that is instantly meaningful. We suggest a method for reducing the time and cognitive effort necessary to acquire SA by applying principles of visual perception to the design of battlefield visualization displays. Specifically, we exploit the organizational properties of gestalt figures and the phenomenon of global preference to create displays that allow immediate understanding of relevant information on the location, identity, and concentration of friendly and enemy units. The data is then compressed temporally, bringing battle dynamics into a useful time domain.

An important goal in interactive computer graphics is to allow the user to interact dynamically with three-dimensional objects. The computing resources required to represent, transmit and display a three dimensional object depends on the number of polygons used to represent it. Many geometric simplification algorithms have been developed to represent the geometry with as few polygons as possible, without substantially changing the appearance of the rendered object. A popular method for achieving geometric simplification is to replace fine scale geometric detail with texture images mapped onto the simplified geometry. However the effectiveness of replacing geometry with texture has not been explored experimentally. In this paper we describe a visual experiment in which we examine the perceived quality of various representations of textured, geometric objects, viewed under direct and oblique illumination. We used a pair of simple large scale objects with different fine-scale geometric detail. For each object we generated many representations, varying the resources allocated to geometry and texture. The experimental results show that while replacing geometry with texture can be very effective, in some cases the addition of texture does not improve perceived quality, and can sometimes reduce the perceived quality.

A method is proposed for migrating a planar shape through a scale space in which the shape is sampled on an irregular grid induced by curvature events along the shape contour. Because the grid is derived from the shape, the grid pattern itself changes in discrete steps as the shape is simplified. At each step the shape contour is reconstructed with an equal or fewer number of inflection points. An implementation is described based on a sampling grid generated by a Voronoi tessellation, and examples given of the resulting progressive simplification for two shapes. It is argued that the process simulates the simplification of a shape perceived by a human observer as the distance between observer and shape steadily decreases.

We have conducted experiments in which subjects are asked to make judgments about the shape of graphically rendered, rotating convex objects. As compared with monoscopic display, stereographic display enhances the subjective clarity of object shape. There are two obvious candidates for the source of this enhancement. One possibility is that the occluding contour is seen as a rotating 3D outline, providing greater object stability, and therefore better cues to the overall shape. Another possibility is that patches of the shaded surface may be perceived in depth, providing a clearer perception of the local surface curvature, and therefore a better sense of the overall shape. In this paper we present an empirical examination of the effects of including stereopsis in the task of shape understanding, and we isolate and compare these two possible explanations.

We present the results of a study of the plausibility of inverting the image analysis done in the human visual system. We examine the arguments for such inversion in recall and visual imagery, and look at the requirements for creating imagery by inverting one or multiple layers of function in neural nets. We then show how such a reversal of visual system processing can take place in stages, or all at once. This is done by means of finite Radon transforms on certain geometries, and we examine the possibility that such situations exist in the human visual system. We create a dual system to certain feed forward network models of visual processing, and show its application to such processing, and to non-image processing applications.

In this paper, we aim to get to the content-based retrieval of nonuniformly textured objects from natural scenes under varying illumination and viewing conditions. Nonuniformly textured objects are objects containing irregular texture elements such as trees, animals (e.g. lions), walls, and grass. To cope with irregular texture contents, the texture measure is based on comparing feature distributions based on the multidimensional histogram intersection of color ratio derivatives. It is shown that color ratio derivatives are robust to a change in illumination, camera viewpoint, and pose of the textured object. Color ratio derivatives are computed from the RGB color channels of a ccd color camera as well as from spectral data obtained by a spectrograph. To cope with object cluttering, a region-based texture segmentation is applied on the target images in the image database prior to the actual image retrieval process. The region-based segmentation algorithm computes regions or blobs having roughly the same texture content as the query image. After segmenting the target images into blobs, the retrieval process is based on computing the histogram intersection of color ratio derivatives derived from query image and target blobs. Experiments have been conducted on images taken from colored, textured objects. Different light sources have been used to illuminate the objects in the scene. From the theoretical and experimental results, it is concluded that color constant texture matching in image libraries provides high retrieval accuracy and is robust to varying illumination and viewing conditions.

Texture plays an important part in many Content Based Image Retrieval systems. This paper describes the results from a human study, which asked 30 volunteers to classify images from the Brodatz Textures album. We use these results to derive a subset which show good agreement among the different individuals. The results for this subset were used to evaluate the retrieval performance of a range of statistical, Fourier- based, and spatial/spatial filtering methods. However, no one computational method works well for all textures, unlike the human visual system. We show how each of the ten methods correlates with the rankings from the human studies. The results typically match for only about 20% - 25% of the images. Combining two techniques can improve the retrieval performance, as judged by human users. We also identify a further subset of the Brodatz images where no computer method correlates significantly with the composite human ranking. Of the 85 images selected by the human study, only 64 have any significant correlation with one or more of the computational methods in this paper. The excluded images, where human users agree with each other, but none of the methods we evaluated did, provide a further challenge to texture-based image retrieval techniques.

A consumer photograph, or snapshot, is a medium for conveying to a viewer, one's interest in one or more main subjects. A methodology is presented for collecting ground truth data useful for training and evaluating algorithms designed to automatically detect the main subject of a consumer photograph. For a database of 100 images, 16 observers provided polygonal approximations to the image areas that comprise the main subject. Results from all observer are combined to form a truth image that is considered the ideal result of a main subject detector and is analyzed to determine features for main subject detection (MSD). The collected ground truth shows substantial agreement among third-party observers. It also supports conventional wisdom regarding the likely locations of main subjects and the value of 'people' detection as a cue for main subject detection. Training data is created from the truth images for an MSD framework involving image segmentation, feature detection, and probabilistic reasoning. A proposed method for generating region-based training data can be used to retrain a reasoning engine as segmentation algorithms improve, without further observer involvement. Although the subject matter for consumer photographs ranges from sweeping landscapes to close portraits, identification of the main subject is a meaningful task.

Research in facial expression has concluded that at least six emotions, conveyed by human faces, are universally associated with distinct expressions. Sadness, anger, joy, fear, disgust and surprise are categories of expressions that are recognizable across cultures. In this work we form a relation between the description of the universal expressions and the MPEG-4 Facial Definition Parameter Set (FDP). We also investigate the relation between the movement of basic FDPs and the parameters that describe emotion-related words according to some classical psychological studies. In particular Whissel suggested that emotions are points in a space, which seem to occupy two dimensions: activation and evaluation. We show that some of the MPEG-4 Facial Animation Parameters (FAPs), approximated by the motion of the corresponding FDPs, can be combined by means of a fuzzy rule system to estimate the activation parameter. In this way variations of the six archetypal emotions can be achieved. Moreover, Plutchik concluded that emotion terms are unevenly distributed through the space defined by dimensions like Whissel's; instead they tend to form an approximately circular pattern, called 'emotion wheel,' modeled using an angular measure. The 'emotion wheel' can be defined as a reference for creating intermediate expressions from the universal ones, by interpolating the movement of dominant FDP points between neighboring basic expressions. By exploiting the relation between the movement of the basic FDP point and the activation and angular parameters we can model more emotions than the primary ones and achieve efficient recognition in video sequences.

Do all parts of the face contribute equally to face detection or are some parts more detectable than others? The task was to detect the presence of normalized frontal-face images within in aperture windows of varying extent. We performed such a face summation study using two-alternative forced-choice psychophysics. The face stimuli were scaled to equal eye-to- chin distance, foveated on the bridge of the nose. The images were windowed by a fourth-power Gaussian envelope ranging from the center of the nose to the full face width. Eight faces (4 male and 4 female) were presented in randomized order, intermixed with 8 control stimuli consisting of phase- scrambled versions of the images with equal Fourier energy. The integration functions for detection of random images did not deviate significantly from a log-log slope of -0.5, suggesting the operation of a set of ideal integrators with probability summation over all aperture sizes. The data for face detection showed that observers were not ideal integrators for the information in the face images, but integrated linearly up to some small size and failed to gain any improvement for information beyond some larger size. This performance suggested the operation of a specialized face template filter at detection threshold, differing in extent among the observers.

In order to develop a useful and ergonomically attractive remote controller for ISDB (Integrated Services Digital Broadcasting), which will begin very soon in Japan, we performed experiments with elderly and young subjects to evaluate the usability and the training effects of four types of remote controller: a button type, a trackball, a touch panel and a voice recognition system. We set the subjects the task of selecting an icon on a HDTV monitor as quickly and as accurately as possible using each remote controller. Semantic differential and ranked order questionnaire surveys were also conducted, and these results were analyzed statistically. The results showed that the trackball type was the most preferred, with no major differences in preference among the other three types especially for elderly subjects. From the analyses of the questionnaire surveys and operation time, we conclude that the reasons for the rankings obtained are as follows: Users preferred devices which they could operate without having to look down; Users preferred devices with which there was a significant learning effect in a relatively short period. It is considered that these are necessary conditions for an ergonomically attractive remote controller which users will want to use.

We are studying about an intelligent robot camera that can automatically shoot an object and produce images with a powerful sense of reality as if a very skilled cameraman were at work. In this study, we designed a control algorithm based on cameramen's techniques for the control of the robot camera and conducted a series of experiments to understand the effects of camera work on how images look to viewers. The findings were as follows: (1) Evaluation scores are high when actual data by cameraman, especially typical data, are used as the position adjusting velocity curve of the target. (2) Evaluation scores are relatively high for images taken with feedback-feedforward camera control method when the target moves in one direction. (3) When both the direction and velocity of the target change and when the target gets bigger and faster in the view finder, it becomes increasingly difficult to keep the target within the view finder using the control method that imitates the human camera handling. (4) The method with mechanical feedback, on the other hand, is able to cope with rapid changes in the target's direction and velocity, constantly keeping the target within the view finder. Even so, the viewer finds the image rather mechanical than natural.

The most important and fascinating ability of natural vision systems is that they spend most of their time on interesting portions of their input, that is, on those aspects of an image which inform the task at hand. This helps a great deal in estimating the location of the system even under dynamic environmental conditions to which systems are subjected to in everyday life. We propose a model that incorporates such ability in robots. Landmark-based approach to robot navigation requires what we define as 'interest operators' to estimate the utility of a particular image region as an effective representative. We have chosen color as the distinguishing characteristic for landmarks. We present a color interest operator consisting of a weighted combination of heuristic scores which thereby selects those image regions or landmarks likely to be found again, even under a different viewing and/or different illumination conditions. The salient regions yield a robust representation for the recognition of a scene. The ability to reproduce regions selected by the operator can be of great help in eliminating environmental uncertainties. We also incorporate a novel color object algorithm, which surpasses all currently available algorithms in speed, robustness and performance to further quicken the response of the navigating robot.

The concept of presence, i.e. the sensation of 'being there' in a mediated environment, has received substantial attention from the virtual reality community, and is becoming increasingly relevant both to broadcasters and display developers. Although research into presence is still at an early stage of development, there is a consensus that presence has multiple determinants. To identify and test which parameters affect presence, a reliable, robust and valid means of measuring presence is required. In this paper, we describe the categories of factors thought to have an impact on presence. Furthermore, we present an overview of various approaches taken to measuring presence, which can be divided into two general categories: subjective measures and objective corroborative measures. Since presence is a subjective experience, the most direct way of assessment is through users' subjective report. This approach has serious limitations however, and should be used judiciously. Objective measures, such as postural, physiological or social responses to media, can be used to corroborate subjective measures, thereby overcoming some of their limitations. At present, the most promising direction for presence measurement is to develop and use an aggregate measure of presence that is comprised of both subjective and objective components, tailored to the specific medium under study.

In many everyday situations people have to locate a particular object in a cluttered environment, for instance a routing sign among commercial signs in an airport terminal. The object's conspicuity determines the efficiency of the search. The literature on human visual search does not unequivocally answer the question how a cluttered environment affects target conspicuity. A method was developed and validated to measure target conspicuity and the effect of distractors, using the case of routing signs (blue with white lettering) and commercial signs (non-blue) in large public buildings as a vehicle. The stimulus fields are complex, computer-generated images, which mimic natural scenes in public buildings but have no apparent meaning. Target conspicuity is operationalized as the time a subject needs to locate the target; this search time is derived from the subject's eye movements. To avoid artifacts of perceptual learning, the number of trials per subject is limited. Experiments in which this method was used clearly demonstrated that conspicuity results from the combined action of the object's own physical properties in relation to those of its environment and of the observer's perceptual and cognitive properties and current intention. Thus, conspicuity never depends merely on the characteristics of the visual stimulus.

Visual perception, operating below conscious awareness, effortlessly provides the experience of a rich representation of the environment, continuous in space and time. Conscious visual perception is made possible by the 'foveal compromise,' the combination of the high-acuity fovea and a sophisticated suite of eye movements. Our illusory visual experience cannot be understood by introspection, but monitoring eye movements lets us probe the processes of visual perception. Four tasks representing a wide range of complexity were used to explore visual perception; image quality judgments, map reading, model building, and hand-washing. Very short fixation durations were observed in all tasks, some as short as 33 msec. While some tasks showed little variation in eye movement metrics, differences in eye movement patterns and high-level strategies were observed in the model building and hand washing tasks. Performance in the hand washing task revealed a new type of eye movement. 'Planful' eye movements were made to objects well in advance of a subject's interaction with the object. Often occurring in the middle of another task, they provide 'overlapping' temporal information about the environment providing a mechanism to produce our conscious visual experience.

In standard differential geometry, the Fundamental Theorem of Space Curves states that two differential invariants of a curve, namely curvature and torsion, determine its geometry, or equivalently, the isometry class of the curve up to rigid motions in the Euclidean three-dimensional space. Consider a physical model of a space curve made from a sufficiently thin, yet visible rigid wire, and the problem of perceptual identification (by a human observer or a robot) of two given physical model curves. In a previous paper (perceptual geometry) we have emphasized a learning theoretic approach to construct a perceptual geometry of the surfaces in the environment. In particular, we have described a computational method for mathematical representation of objects in the perceptual geometry inspired by the ecological theory of Gibson, and adhering to the principles of Gestalt in perceptual organization of vision. In this paper, we continue our learning theoretic treatment of perceptual geometry of objects, focusing on the case of physical models of space curves. In particular, we address the question of perceptually distinguishing two possibly novel space curves based on observer's prior visual experience of physical models of curves in the environment. The Fundamental Theorem of Space Curves inspires an analogous result in perceptual geometry as follows. We apply learning theory to the statistics of a sufficiently rich collection of physical models of curves, to derive two statistically independent local functions, that we call by analogy, the curvature and torsion. This pair of invariants distinguish physical models of curves in the sense of perceptual geometry. That is, in an appropriate resolution, an observer can distinguish two perceptually identical physical models in different locations. If these pairs of functions are approximately the same for two given space curves, then after possibly some changes of viewing planes, the observer confirms the two are the same.

In this paper, a fractal-based compression approach of wavelet image is presented. The scheme tries to make full use of the sensitivity features of the human visual system. With the wavelet-based multi-resolution representation of image, detail vectors of each high frequency sub-image are constructed in accordance with its spatial orientation in order to grasp the edge information to which human observer is sensitive. Then a multi-level selection algorithm based on human vision's contrast masking effect is proposed to make the decision whether a detail vector is coded or not. Those vectors below the contrast threshold are discarded without introducing visual artifacts because of the ignorance of human vision. As for the redundancy of the retained vectors, different fractal- based methods are employed to decrease the correlation in single sub-image and between the different resolution sub- images with the same orientation. Experimental results suggest the efficiency of the proposed scheme. With the standard test image, our approach outperforms the EZW algorithm and the JPEG method.

Objective image quality assessment techniques are currently based on the properties of the human visual system (HVS) essentially using early vision model. This type of approach allows to get the differences between original and distorted images in a perceptually space, so the outputs of the early vision model are perceptual distortion maps. In order to get one mark for the overall distortion, spatial pooling and frequency pooling in case of spatial frequency decomposition should be applied on these maps. In this paper, we present various methods to do this pooling. In order to represent the distorted image in a perceptual space, we use a multi-channel early vision model including an amplitude nonlinearity, a CSF, a subband decomposition and a masking function. For the pooling, Minkowski summation with various exponents is first tested as it is the most common pooling in literature. The second type of pooling proposed achieves a summation of all the degradations weighted by a function of the probability of their occurrence. Finally we propose a summation taking into account some higher perceptual factors in order to point out the region of interest used to weight the errors. The results are compared measuring the correlation between the distortion marks and the MOS.

Green noise is the mid-frequency component of white noise and has been shown to have visually pleasing attributes when applied to digital halftoning. Unlike blue noise dither patterns, which are composed exclusively of isolated pixels, green noise dither patterns are composed of pixel-clusters making them less susceptible to image degradation from non- ideal printing artifacts such as dot-loss. Clearly, these patterns reduce the spatial variation in tone produced by electrophotographic printers when printing a constant shade of gray, but to date, no study has been presented showing the amount of reduction. In this paper, we address this problem by studying the effects of changing the average cluster size in a green noise dither pattern, measuring the resulting spatial variations for a Lexmark Optra laser printer in 1200 dpi mode. The print quality is evaluated in terms of the visibility of printer mechanism noise and the average change in tone across the printed page.

This paper describes a novel adaptive-hierarchical-filtering technique to achieve high-quality magazine image reproduction on computer peripherals such as color copier, or scanner plus printer. Commercial magazine images are halftoned images. Unacceptable noises and moire distortion may result when halftone images are copied (i.e., scanned and printed). In this paper, we analyze moire patterns formation characterized by the following factors: Moire patterns formation depends on the following factors: (1) halftone screen frequency, how many lines per inch (i.e., lpi) (2) scan frequency, how many dot per inch, (dpi) (3) screen angle which is the screen orientation against the scan direction, (4) scanner aperture (i.e., scanner characteristics), (5) halftoning and printing mechanisms, (6) viewing conditions. Error diffusion tends to randomize the moire patterns, and cluster-dot or line screen show more moire distortion. We developed a set of variable- length low-pass filters that have a nice inheritance of canceling aliased low frequency components (moire distortion). High pass filtering is also applied to sharpen image edges. A hierarchical filter classifier was developed to determine that an edge is either a global true edge (for sharpening enhancement) or a local halftone's micro-structural edge (for LPF for moire reduction). Depending on the classifier, adaptive LPF is applied to achieve the smooth transition between sharp edges and smooth halftone regions. Thus, we achieve overall high-quality output images. Experimental results have been shown the effectiveness of the presented technique that works well on wide combinations of above- mentioned 6 factors for high-quality magazine image reproduction.

In parallel with our studies on human eye movements, we have investigated image processing algorithms that predict where human eyes fixate. These loci of fixations, traditionally named Regions-of-Interest, ROIs, are strategically important both for computer applications and for cognitive studies of human visual processing. A very important aspect of our methodology, beyond the specific image processing algorithms used, is how to select from a large initial set of candidates, usually local maxima in the processed image, a final set of few ROIs. In this paper we analyze this latter aspect, proposing and comparing different clustering procedures and study how different procedures may affect the fidelity of comparisons with human selected ROIs.

A vision-based tele-operating system had been designed and simulated in our lab with model-based supervisory control and model-based, top-down image processing, IP, for vehicle pose recovery. Top-down IP at near real-time feedback was achieved by a pose extraction algorithm based on the scanpath theory of human eye movement. In this paper, we extended our system into a multiple cameras system to improve the accuracy and robustness of the visual system. Another possible approach was to use mobile camera platforms. We contrasted the difference between the two approaches in the context of our system requirement and limitation. OpenGL based simulation was used to illustrate advantages of using more cameras quantitatively to increase system robustness against plant and image noise.

Accurate detection of elementary features such as edges and lines in digital images plays an important role in early vision and is included as a first step in many image processing systems. An efficient and flexible form of feature detection is based on the responses of sets of orientation selective filters. These steerable filters can be designed to posses near Gabor properties and provide a derivable interpolation function over orientation. However, these filters are rarely designed using non-circular receptive fields and cannot be optimized in terms of angular selectivity. A line of reasoning is developed here which allows orientation selective filters to be constructed using elliptical window functions and used within a conventional steerable filter bank system. Results suggest an optimal situation existing between the rejection of unwanted signals and high angular selectivity of filters. Further results are developed which emphasize phase differences between elliptical and circular filter responses as a measure of locally convex curvature maxima.

Models of the human pupil response have been widely investigated. Yet, besides the well known model for the pupil diameter as a function of light intensity and some studies on the influence of wavelength, a model for the dependence of the pupil diameter on the geometry of an observed image is still missing. This paper describes an experimental setup to study the influence of the geometry of a light distribution stimulating the human eye and its pupil. Moreover, experimental results of the human pupil response as a function of the time and of geometric parameters of the stimulating field are presented. To measure the behavior of the pupil, a test person is positioned in front of a back-side illuminated diffusing pane. This pane approximates a Lambert-radiator. The illuminated area is adjustable to different circular aperture sizes. While varying both size and intensity, the pupil response of a test person is observed by an infrared CCD- camera. Electronic image processing is used to determine the diameter of the captured pupil in the image. The first step is to convert the gray-scale image into a binary image with an optimized threshold. Then, the two dimensional image is reduced to a one-dimensional column-image by collecting and adding all pixels of each row (parallel projection). This step enables a very effective frequency-domain analysis (Fourier transform) leading to the detection of the pupil diameter. The actual setup is able to process ten images per second with a resolution of 0.2 mm. Short-time effects of the pupil diameter can therefore be observed as well as long-time effects. It was found that previous results regarding the pupil diameter as function of the light intensity were confirmed. This indicates that the automatic system works well and the psycho-physical experiments are conducted reasonably. However, as a new result, it was found that the pupil response seems to be dependent on the aperture size of the stimulating field.

Pupillary responses of the subjects under chromatic visual stimulation were filmed with video-oculography systems (VOG). Specifically, programmable chromatic visual stimuli were displayed on the monitor of a personal computer (PC) and responses tested in several normal subjects and those with ophthalmic pathologic conditions. Visual excitation utilized a reversal stimulus checkerboard pattern. Image processing techniques were applied in order to evaluate the pupil variations due to chromatic stimulation. In particular, the studies determined a relationship between pupillary response (PR) and steady-state retinal sensitivity (photopic vision). Finally, the retinal illuminance in trolands (td), as a product of the pupil area in mm² and luminance in cd/m² to full-field chromatic stimulation (FFCS) and checkerboard pattern reversal chromatic stimulation (CPRCS) was determined.

Mosquito noise is a time dependent video compression impairment in which the high frequency spatial detail in video images having crisp edges is aliased intermittently. A new synthetic test pattern of moving spirals or circles is described which generates mosquito noise (MN) under Motion Pictures Expert Group (MPEG) compression. The spiral pattern is one of several NIST-developed patterns designed to stress specific features of compression based on motion estimation and quantization. The 'Spirals' pattern has several spirals or circles superimposed on a uniform background. The frames are filtered to avoid interline flicker which may be confounded with MN. Motion of the spirals and changing luminance of the background can be included to reduce the correlation between successive frames. Unexpectedly, even a static pattern of spirals can induce mosquito noise due to the stochastic character of the encoder. We consider metrics which are specific to the impairment being measured. For mosquito noise, we examine two separable detectors: each consists of a temporal (frame-to-frame) computation applied to the output of a spatial impairment detector which is applied to each frame. The two spatial detectors are: FLATS, which detects level 8 X 8 pixel image blocks; and the root-mean-square (RMS) applied to the image differences between original and compressed frames. The test patterns are encoded at low bit rates. We examine the measured mosquito noise as a function of the Group-of-Pictures (GOP) pattern in the MPEG-2 encoding and find that the GOP structure defines the periodicities of the MN.

We have studied four approaches to segmentation of images: three automatic ones using image processing algorithms and a fourth approach, human manual segmentation. We were motivated toward helping with an important NASA Mars rover mission task -- replacing laborious manual path planning with automatic navigation of the rover on the Mars terrain. The goal of the automatic segmentations was to identify an obstacle map on the Mars terrain to enable automatic path planning for the rover. The automatic segmentation was first explored with two different segmentation methods: one based on pixel luminance, and the other based on pixel altitude generated through stereo image processing. The third automatic segmentation was achieved by combining these two types of image segmentation. Human manual segmentation of Martian terrain images was used for evaluating the effectiveness of the combined automatic segmentation as well as for determining how different humans segment the same images. Comparisons between two different segmentations, manual or automatic, were measured using a similarity metric, S_AB. Based on this metric, the combined automatic segmentation did fairly well in agreeing with the manual segmentation. This was a demonstration of a positive step towards automatically creating the accurate obstacle maps necessary for automatic path planning and rover navigation.

'Reverspectives' is one of many titles given to a remarkable class of Patrick Hughes's paintings that share two basic properties: (1) They are painted on a set of intersecting planar surfaces, rather than a single flat surface. (2) The perspective produced by the painting elicits the percept of a depth configuration that is the reverse of the actual depth configuration, as dictated by the construction of the multi- planar combination of surfaces (hence the name reverspective). As a result of these two properties, reverspectives appear to turn vividly as viewers walk past the paintings. This strange motion, which can rarely be observed in flat-canvas paintings unless they elicit strong perspective depth, can be explained by a neural process that matches up kinesthetic information about the viewer's own motion to changes in the retinal images. In this paper I study the role played by the perspective painting, by comparing percepts elicited by conventional Hughes reverspectives to those elicited by the same multi-planar surfaces that are left blank (unpainted). I examine differences in the two precepts in terms of the critical distance, i.e. that viewing distance for which the depth percept switches from veridical to reverse. This paradigm is one example in which the adoption of a depth schema determines the interpretation of motion signals.

The perceptual analysis an artist brings to creating an object is explored in relation to how a viewer perceives a completed art object. Multiple styles, media, and historical images are surveyed to (1) explain the artist/object relationship is interactive, (2) contrast narrative and visual compositional goals and (3) explore underlying solutions that bring superficially dissimilar images together. Within this matrix, the discussion demonstrates how innovative artistic and scientific imaging technologies interface as they revise understandings of how we see surfaces, transparency, light, and even seeing itself.

Presence is a user's subjective sensation of 'being there' in a mediated environment. Its determinants include the extent of sensory information within a mediated environment, the user's ability to control their sensors and modify content within the environment, and individual-difference based factors. Although direct subjective ratings of presence have been used, the construct may be unclear to naive observers, and previous work in our laboratory has shown that ratings of presence may be biased towards different characteristics of the display. In the present study focus groups were used to: (1) establish whether viewers refer to sensations of presence, unprompted, (2) identify viewers' descriptions of their experiences of stereoscopic TV, and (3) identify the program types best suited to advanced broadcast presentations. Results showed that non-experts describe sensations of presence without prompting while watching stereoscopic television, relating it to involvement, realism and naturalness. In addition, there was a general consensus that live events, such as sports matches, theatre and concerts are the program types best suited to high-presence broadcast presentations, through advanced broadcast services such as Immersive Television.

Being present at a live event is undeniably the most exciting way to experience any entertainment. This is true whether we are talking about a musical concert, a theatrical performance, a cricket match, or even a firework display. The ability to direct your gaze where you wish, to hear sounds from all around you, to experience the immediacy and expectation of an unscripted happening, to feel the buzz of the crowd and to smell the grass or smoke, are all sensory cues which contribute to the powerful experience of being there. This paper examines the ways in which entertainment media have attempted to recreate experiences which encourage the viewer to suspend disbelief and become part of a remote or recorded event. We introduce the concept of immersive television and look at some of the research, spanning many disciplines of science and art, which the ITC is conducting to explore the potential of this new medium.

Depth representation, in both its geometric and its more generic forms, has often served as an impetus in artistic development through the millennia. The first historical mentions of art, by Plato and contemporaries in the 5th century BC, were provoked by the dramatic use of perspective in the scenery for the plays of Aeschylus and Sophocles. One of these innovative scene painters, Agatharchus, even wrote a commentary on his use of convergent perspective, whose effects had inspired several contemporary Greek geometers to analyze the productive transform mathematically.

Casual users of color frequently misunderstand the relationships between spatially adjacent colors or even the simpler concepts of value and chroma. In this paper we discuss the design and use of Color Deducto, a computer game that provides an enjoyable way for players to learn about color relationships. Players learn from Color Deducto in two ways. First, they learn by playing the game, which consists of a series of logic puzzles. In each puzzle, players must infer the rule that distinguishes 'matching' grids (arrangements of colors that satisfy the rule) from non-matching grids. Second, players learn by devising their own rules for others to infer.

How an object appears depends on geometrical and physical properties of its surface. Moreover, objects occur in an environment and the context in which objects are embedded affects the perception of their shapes. In this paper the influences of scene geometry and viewing conditions on shape perception are investigated in a systematic way. Stimuli are actual 3D scenes. The objects in the scenes are triaxial ellipsoids placed on a turntable, in a controlled lightfield, and against a distant backdrop. The ellipsoids approximate Lambertian diffuse reflection (finished matte white). Visual shape is measured by probing local surface attitude: a 'gauge figure' has to be adjusted such that it appears as a circle painted upon the surface of the objects. Form and orientation of this gauge figure can be translated into slant and tilt values at the sampled locations. Scene geometry is varied parametrically. Detailed quantitative results of the effects of scene geometry have been gathered by manipulating size, distance and orientation of the objects relative to the observer. Depending on context overestimation of relief by a factor of 2 - 4 occurs. Slant overestimations are less for larger objects and more distant objects.