In multispectral imaging systems, the accuracy of reflectance estimation can be degraded by the nonlinearity in imaging process, which is due to non-Gaussian distribution of the data and nonlinear optoelectronic conversion function of the camera. To deal with nonlinearity, we propose to extend camera responses by high-order polynomials and reduce the overfitting problem by partial least-squares (PLS) regression. Experiment shows that, in terms of both spectral and colorimetric error metrics, the proposed method performs better than Wiener estimation and ordinary polynomial regression, and is similar to polynomial regression with regularization.

We propose a new method that classifies wafer images according to their defect types for automatic defect classification in semiconductor fabrication processes. Conventional image classifiers using global properties cannot be used in this problem, because the defects usually occupy very small regions in the images. Hence, the defects should first be segmented, and the shape of the segment and the features extracted from the region are used for classification. In other words, we need to develop a classification-after-segmentation approach for the use of features from the small regions corresponding to the defects. However, the segmentation of scratch defects is not easy due to the shrinking bias problem when using conventional methods. We propose a new Markov random field-based method for the segmentation of wafer images. Then we design an AdaBoost-based classifier that uses the features extracted from the segmented local regions.

This PDF file contains the editorial “Special Section Guest Editorial: Digital Photography” for JEI Vol. 19 Issue 02

We present a detailed design study for a novel solid-state focal plane array of silicon avalanche photodiodes (APDs), with each detector in the array capable of operating with wide dynamic range in linear or Geiger-mode. The detector array is based on back-illuminated, crystallographically etched, (100) epitaxial silicon on R-plane sapphire substrates using 27 µm pixels, although other sizes could be supported as well. Analysis of the APD pixel design, which includes the photoelectron noise, dark current noise, and electronic circuit noise, shows the capability of imaging with a signal-to-noise ratio greater than 7 at −30 °C and ultralow illuminance of 10⁻⁴ lux. The detector array design will enable large focal planes, capable of imaging with high sensitivity and high speed, over a wide range of natural illumination conditions from direct sunlight to a cloudy moonless night.

We propose a modification to the standard Bayer color filter array (CFA) and photodiode structure for CMOS image sensors, which we call 2PFC^TM (two pixels, full color). The blue and red filters of the Bayer pattern are replaced by a magenta filter. Under each magenta filter are two stacked, pinned photodiodes; the diode nearest the surface absorbs mostly blue light, and the deeper diode absorbs mostly red light. The magenta filter absorbs green light, improving color separation between the blue and red diodes. We first present a frequency-based demosaicing method, which takes advantage of the new 2PFC geometry. Due to the spatial arrangement of red, green, and blue pixels, luminance and chrominance are very well separated in the Fourier space, allowing for computationally inexpensive linear filtering. In comparison with state-of-the-art demosaicing methods for the Bayer CFA, we show that our sensor and demosaicing method outperform the others in terms of color aliasing, peak signal to noise ratio, and zipper effect. As demosaicing alone does not determine image quality, we also analyze the whole system performance in terms of resolution and noise.

Color demosaicing is critical for digital cameras, because it converts a Bayer sensor mosaic output to a full color image, which determines the output image quality of the camera. In this work, an efficient decision-based demosaicing method is presented. This method exploits a new edge-sensing measure called integrated gradient (IG) to effectively extract gradient information in both color intensity and color difference domains simultaneously. This measure is reliable and supports full resolution, which allows one to interpolate the missing samples along an appropriate direction and hence directly improves the demosaicing performance. By sharing it in different demosaicing stages to guide the interpolation of various color planes, it guarantees the consistency of the interpolation direction in different color channels and saves the effort required to repeatedly extract gradient information from intermediate interpolation results at different stages. An IG-based green plane enhancement is also proposed to further improve the method's efficiency. Simulation results confirm that the proposed demosaicing method outperforms up-to-date demosaicing methods in terms of output quality at a complexity of around 80 arithmetic operations per pixel.

Color interpolation solutions drastically influence the quality of the whole image generation pipeline, so they must guarantee the rendering of high quality pictures by avoiding typical artifacts such as blurring, zipper effects, and false colors. Moreover, demosaicing should avoid emphasizing typical artifacts of real sensors data, such as noise and green imbalance effect, which would be further accentuated by the subsequent steps of the processing pipeline. We propose a new adaptive algorithm that decides the interpolation technique to apply to each pixel, according to its neighborhood analysis. Edges are effectively interpolated through a directional filtering approach that interpolates the missing colors, selecting the suitable filter depending on edge orientation. Regions close to edges are interpolated through a simpler demosaicing approach. Thus flat regions are identified and low-pass filtered to eliminate some residual noise and to minimize the annoying green imbalance effect. Finally, an effective false color removal algorithm is used as a postprocessing step to eliminate residual color errors. The experimental results show how sharp edges are preserved, whereas undesired zipper effects are reduced, improving the edge resolution itself and obtaining superior image quality.

Plenoptic cameras, constructed with internal microlens arrays, capture both spatial and angular information, i.e., the full 4-D radiance, of a scene. The design of traditional plenoptic cameras assumes that each microlens image is completely defocused with respect to the image created by the main camera lens. As a result, only a single pixel in the final image is rendered from each microlens image, resulting in disappointingly low resolution. A recently developed alternative approach based on the focused plenoptic camera uses the microlens array as an imaging system focused on the image plane of the main camera lens. The flexible spatioangular trade-off that becomes available with this design enables rendering of final images with significantly higher resolution than those from traditional plenoptic cameras. We analyze the focused plenoptic camera in optical phase space and present basic, blended, and depth-based rendering algorithms for producing high-quality, high-resolution images. We also present our graphics-processing-unit-based implementations of these algorithms, which are able to render full screen refocused images in real time.

We propose a global motion (GM) compensated preprocessing algorithm for block-based frame rate conversion (FRC). The proposed method estimates camera motions such as zooms or rotations between two input frames, and accordingly, produces nontranslational GM-free pictures by performing GM compensation with respect to the temporal location where the FRC method reconstructs an intermediate frame. To reduce the computational load, the proposed method first subsamples input images, and then block-wise motion estimation (ME) is performed. With the ME results, the proposed method detects scene changes and nontranslational GMs. This allows us to determine whether to proceed with the GM estimation and compensation processes. The geometric motion model is adopted to describe camera motions with four parameters, and these values are iteratively found on a motion vector (MV) field. Experimental results show that the proposed algorithm achieves significant performance improvements of subsequent FRC methods.

In telemedicine environments, a standardized and reproducible assessment of wounds, using a simple free-handled digital camera, is an essential requirement. However, to ensure robust tissue classification, particular attention must be paid to the complete design of the color processing chain. We introduce the key steps including color correction, merging of expert labeling, and segmentation-driven classification based on support vector machines. The tool thus developed ensures stability under lighting condition, viewpoint, and camera changes, to achieve accurate and robust classification of skin tissues. Clinical tests demonstrate that such an advanced tool, which forms part of a complete 3-D and color wound assessment system, significantly improves the monitoring of the healing process. It achieves an overlap score of 79.3 against 69.1% for a single expert, after mapping on the medical reference developed from the image labeling by a college of experts.

Artificial color uses the projection of the spectrum into two or more broad, overlapping spectral bands to discriminate, pixel by pixel, among user-defined classes of objects. As initially practiced, it used a sequence of hyperspherical regions of the decision space to define class membership. Of course, a hypersphere is just a degenerate hyperellipsoid; thus, exploring the effect of loosening that degeneracy seemed appropriate. Initially, we use two-foci hyperellipsoids with a hyperellipsoidal distance metric to classify pixels with dramatic improvement in performance. We explore the work even further by allowing many foci and noting the effects of increased complexity of the decision surfaces. In the example case, three foci gave superior performance to one or two foci, but four added little improvement.

Various computerized features extracted from breast ultrasound images are useful in assessing the malignancy of breast tumors. However, the underlying relationship between the computerized features and tumor malignancy may not be linear in nature. We use the decision tree ensemble trained by the cost-sensitive boosting algorithm to approximate the target function for malignancy assessment and to reflect this relationship qualitatively. Partial dependence plots are employed to explore and visualize the effect of features on the output of the decision tree ensemble. In the experiments, 31 image features are extracted to quantify the sonographic characteristics of breast tumors. Patient age is used as an external feature because of its high clinical importance. The area under the receiver-operating characteristic curve of the tree ensembles can reach 0.95 with sensitivity of 0.95 (61/64) at the associated specificity 0.74 (77/104). The partial dependence plots of the four most important features are demonstrated to show the influence of the features on malignancy, and they are in accord with the empirical observations. The results can provide visual and qualitative references on the computerized image features for physicians, and can be useful for enhancing the interpretability of computer-aided diagnosis systems for breast ultrasound.

A method for contrast enhancement is proposed. The algorithm is based on a local and image-dependent exponential correction. The technique aims to correct images that simultaneously present overexposed and underexposed regions. To prevent halo artifacts, the bilateral filter is used as the mask of the exponential correction. Depending on the characteristics of the image (piloted by histogram analysis), an automated parameter-tuning step is introduced, followed by stretching, clipping, and saturation preserving treatments. Comparisons with other contrast enhancement techniques are presented. The Mean Opinion Score (MOS) experiment on grayscale images gives the greatest preference score for our algorithm.

A JPEG image steganalysis scheme based on joint discrete cosine transform (DCT) domain features is proposed. Intrinsic characteristics of DCT coefficients, such as histogram, intrablock correlation, and interblock correlation, are exploited to construct three feature sets. Support vector machine is utilized to learn and discriminate the difference of features between cover and stego images. First, the three feature sets are investigated separately to reveal their individual capability of attacking steganographic methods. Second, the feature sets are combined to form a joint feature set with better performance. Experimental results demonstrate that all three feature sets individually succeed in attacking the four typical steganographic tools to some extent, with the intrablock feature set performing the best. Furthermore, the comparison experiments show that the jointed feature set not only outperforms the three individual feature sets but also proves to be better than a previous state-of-the-art steganalysis method.

In many image-processing applications, observed images are contaminated by a nonstationary noise and no a priori information on noise dependence on local mean or about local properties of noise statistics is available. In order to remove such a noise, a locally adaptive filter has to be applied. We study a locally adaptive filter based on evaluation of image local activity in a "blind" manner and on discrete cosine transform computed in overlapping blocks. Two mechanisms of local adaptation are proposed and applied. The first mechanism takes into account local estimates of noise standard deviation while the second one exploits discrimination of homogeneous and heterogeneous image regions by adaptive threshold setting. The designed filter performance is tested for simulated data as well as for real-life remote-sensing and maritime radar images. Recommendations concerning filter parameter setting are provided. An area of applicability of the proposed filter is defined.

Letter-scanning cameras (LSCs) form the front- end imaging systems for virtually all mail-scanning systems that are currently used to automatically sort mail products. As with any vision-dependent technology, the quality of the images generated by the camera is fundamental to the overall performance of the system. We present novel techniques for objective evaluation of LSCs using comparative imaging-a technique that involves measuring the fidelity of target images produced by a camera with reference to an image of the same target captured at very high quality. Such a framework provides a unique opportunity to directly quantify the camera's ability to capture real-world targets, such as handwritten and printed text. Noncomparative techniques were also used to measure properties such as the camera's modulation transfer function, dynamic range, and signal-to-noise ratio. To simulate real-world imaging conditions, application-specific test samples were designed using actual mail product materials.

We present a machine vision system for simultaneous and objective evaluation of two important functional attributes of a fabric, namely, soil release and shrinkage. Soil release corresponds to the efficacy of the fabric in releasing stains after laundering and shrinkage essentially quantifies the dimensional changes in the fabric postlaundering. Within the framework of the proposed machine vision scheme, the samples are prepared using a prescribed procedure and subsequently digitized using a commercially available off-the-shelf scanner. Shrinkage measurements in the lengthwise and widthwise directions are obtained by detecting and measuring the distance between two pairs of appropriately placed markers. In addition, these shrinkage markers help in producing estimates of the location of the center of the stain on the fabric image. Using this information, a customized adaptive statistical snake is initialized, which evolves based on region statistics to segment the stain. Once the stain is localized, appropriate measurements can be extracted from the stain and the background image that can help in objectively quantifying stain release. In addition, the statistical snakes algorithm has been parallelized on a graphical processing unit, which allows for rapid evolution of multiple snakes. This, in turn, translates to the fact that multiple stains can be detected and segmented in a computationally efficient fashion. Finally, the aforementioned scheme is validated on a sizeable set of fabric images and the promising nature of the results help in establishing the efficacy of the proposed approach.

Although solutions to a particular attack against the quantization index modulation (QIM) method such as valumetric scaling have been proposed, no attempt has been made to a combined form of common attacks, including valumetric scaling, constant change, additive white Gaussian noise (AWGN), and compressions. Of course, the composite attacks are more realistic but more difficult to solve. In this work, we tackle composite attacks in the framework of the QIM. We seek the solution in DC values of image blocks. By embedding watermark bits into DC values, QIM becomes generically robust against AWGN and most image compression attacks. To undo the alterations caused by valumetric scaling and constant change, pilot reference signals can be embedded into some selected image blocks, where the parameters for the affine transform model of the attacks are estimated directly from the watermarked images. Four versions of block-wise QIM algorithms with pilot signals are proposed. Experimental results show their robustness against severe composite attacks, including valumetric scaling, constant change, AWGN, and JPEG compressions.

We propose an adaptively synchronous scalable spread spectrum (A4S) data-hiding strategy to integrate disparate data, needed for a typical 3-D visualization, into a single JPEG2000 format file. JPEG2000 encoding provides a standard format on one hand and the needed multiresolution for scalability on the other. The method has the potential of being imperceptible and robust at the same time. While the spread spectrum (SS) methods are known for the high robustness they offer, our data-hiding strategy is removable at the same time, which ensures highest possible visualization quality. The SS embedding of the discrete wavelet transform (DWT)-domain depth map is carried out in transform domain YCrCb components from the JPEG2000 coding stream just after the DWT stage. To maintain synchronization, the embedding is carried out while taking into account the correspondence of subbands. Since security is not the immediate concern, we are at liberty with the strength of embedding. This permits us to increase the robustness and bring the reversibility of our method. To estimate the maximum tolerable error in the depth map according to a given viewpoint, a human visual system (HVS)-based psychovisual analysis is also presented.

We propose a new method to quantify the clarity of autofocused images by integrating multiple measures of images in both far- and near-focused regions. Generally, the clarity of images varies in a hill-shaped manner with respect to the lens position within a focus region. However, the variation in the far-focused regions is subject to disturbances and not very useful for the design of a focusing control. Using the properties of regional monotonous variations, the proposed method integrates additional measures, which are adequate to quantify the clarity of images in the far-focused region. The properties of monotonous variations of the measures are used to characterize the design process. The proposed focusing algorithm implements a step-size adaptation mechanism that adjusts the step size of lens motion based on the monotonous variation properties. To improve the focusing speed in the far-focused regions, the monotonous variations of multiple measures are combined to approximate the proximity of the lens to the focus position and the corresponding step size. The combined information about monotonous variations is also used to check and correct inconsistencies of the lens motion, which could occur due to motion imaging disturbances. This inconsistency check has proven to be a significant improvement on the robustness of the proposed focusing control. Four case studies demonstrate the feasibility of this algorithm and the effects of modifying individual design parameters, leading to a response time of 0.6 to 1.2 s. This speed is relatively sufficient for general industrial applications, such as LCD panel manufacturing. With the application of the inconsistency check, each of the four cases was run 100 times, and there were no cases of failure; this passes the manufacturing standard of less than 1% failure. The proposed design concept of regional monotonous variation, along with the inconsistency check and step-size adaptation, improves speed of focusing control and reliability.

The Charpy impact test technique assesses the toughness of an engineering material. The test measures the amount of energy a specimen can resist before it is broken by the impact of a heavy pendulum. Estimation of toughness is carried out manually by a skilled operator; they assess the percentage of light-reflective brittle regions on the fracture area. Because this assessment is performed manually, there is some subjectivity in the results. This study proposes a machine-based-learning algorithm to estimate this measure automatically. The method consists of capturing a digital image of the fracture surface after impact, preprocessing it, dividing it up into 10×10 pixel segments, and extracting from each segment features associated with its texture. Feature vectors feed a classifier whose purpose is to distinguish between brittle and ductile images (binary output). To estimate toughness, the classifier's outputs are used to construct a binary image, which is postprocessed to determine the percentage of the brittle region. To assess the accuracy of the algorithm, automatically and manually classified images are compared. Results show that the algorithm proposed was able to distinguish between brittle and ductile regions successfully and could be used instead of the manually performed technique.

We propose an efficient lossless compression scheme for still images based on arithmetic coding. The scheme presents a novel adaptive arithmetic coding that updates the probabilities of pixels only after detecting the last occurrence of each pixel and then removes the redundancy from the original image effectively. The proposed approach has interestingly low computational complexity. In addition, unlike other statistical coding techniques, arithmetic coding in the proposed scheme is not solely dependent on the pixel probability distribution but also on the image block sorting. The proposed method is compared to both static and adaptive order-0 models while taking into account compression ratios and processing time. Experimental results, based on a set of 100 gray-level images, demonstrate that the proposed scheme gives mean compression ratios that are 5.5% higher than those by the conventional arithmetic encoders as well as significantly faster than the order-0 adaptive arithmetic coding.

A novel robotic laser scanning system for the inspection of large underwater hydroelectric structures is proposed. This system has been developed at the Hydro Quebec Research Institute and consists of a laser camera mounted on a 2-D Cartesian manipulator. Mechanical, electronic, and software design aspects; overall operational modalities; and proof of concept results are presented. We evaluated the performances of the system in the course of laboratory experiments and inspection trials carried out under normal operating conditions at the site of three of Hydro Quebec's hydroelectric dams.

The importance of networked automatic target recognition systems for surveillance applications is continuously increasing. Because of the requirement of a low cost and limited payload, these networks are traditionally equipped with lightweight, low-cost sensors such as electro-optical (EO) or infrared sensors. The quality of imagery acquired by these sensors critically depends on the environmental conditions, type and characteristics of sensors, and absence of occluding or concealing objects. In the past, a large number of efficient detection, tracking, and recognition algorithms have been designed to operate on imagery of good quality. However, detection and recognition limits under nonideal environmental and/or sensor-based distortions have not been carefully evaluated. We introduce a fully automatic target recognition system that involves a Haar-based detector to select potential regions of interest within images, performs adjustment of detected regions, segments potential targets using a region-based approach, identifies targets using Bessel K form−based encoding, and performs clutter rejection. We investigate the effects of environmental and camera conditions on target detection and recognition performance. Two databases are involved. One is a simulated database generated using a 3-D tool. The other database is formed by imaging 10 die-cast models of military vehicles from different elevation and orientation angles. The database contains imagery acquired both indoors and outdoors. The indoors data set is composed of clear and distorted images. The distortions include defocus blur, sided illumination, low contrast, shadows, and occlusions. All images in this database, however, have a uniform (blue) background. The indoors database is applied to evaluate the degradations of recognition performance due to camera and illumination effects. The database collected outdoors includes a real background and is much more complex to process.

Image interpretation is particularly important in many real applications (video monitoring, biometrics, etc.). Due to the proliferation of image interpretation systems in the literature, their evaluation still remains a crucial stake. Among all the tasks in this field, the quality of object localization is often evaluated through an evaluation metric. We propose to review these techniques and study their reliability. We first propose a generic definition of a localization algorithm. Then, different state of the art techniques to evaluate image interpretation results are detailed. Secondly, we focus on metrics that enable us to evaluate localization results. We propose a general methodology to analyze the behavior of an evaluation metric, considered here as a black box (its definition is not even supposed to be known). We define the properties that these metrics should fulfill. We then perform a comparative study of 33 localization metrics from the state of the art. Experimental results conducted on a large and significant image database permit us to determine metrics that should be used in the future for the evaluation of object localization results.

The interframe interpolation method presented consists of partial changing, using the growing size of structuring elements, of the morphological skeleton decomposition subsets of a binary input or grayscale frame with the morphological skeleton decomposition subsets of a binary output or grayscale frame. One of the interpolated frames obtained by this method is similar to the reference frame. Computer simulations illustrate the results.

We propose an image-understanding algorithm for identifying and ranking regions of perceptually relevant content in digital images. Global features that characterize relations between image regions are fused in a probabilistic framework to generate a region ranking map (RRM) of an arbitrary image. Features are introduced as maps for spatial position, weighted similarity, and weighted homogeneity for image regions. Further analysis of the RRM, based on the receiver operating characteristic curve, has been utilized to generate a binary map that signifies region of interest in the test image. The algorithm includes modules for image segmentation, feature extraction, and probabilistic reasoning. It differs from prior art by using machine learning techniques to discover the optimum Bayesian Network structure and probabilistic inference. It also eliminates the necessity for semantic understanding at intermediate stages. Experimental results indicate an accuracy rate of ~90% on a set of ~4000 color images that are publicly available and compare favorably to state-of-the-art techniques. Applications of the proposed algorithm include smart image and document rendering, content-based image retrieval, adaptive image compression and coding, and automatic image annotation.

This PDF file contains the editorial “The Essential Guide to Image Processing” for JEI Vol. 19 Issue 02