This PDF file contains the editorial “Special Section Guest Editorial:Advances on Distributed Smart Cameras” for JEI Vol. 25 Issue 04

Visual sensor networks (VSNs) consist of several camera nodes with wireless communication capabilities that can perform visual analysis tasks such as object identification, recognition, and tracking. Often, VSN deployments result in many camera nodes with overlapping fields of view. In the past, such redundancy has been exploited in two different ways: (1) to improve the accuracy/quality of the visual analysis task by exploiting multiview information or (2) to reduce the energy consumed for performing the visual task, by applying temporal scheduling techniques among the cameras. We propose a game theoretic framework based on the Nash bargaining solution to bridge the gap between the two aforementioned approaches. The key tenet of the proposed framework is for cameras to reduce the consumed energy in the analysis process by exploiting the redundancy in the reciprocal fields of view. Experimental results in both simulated and real-life scenarios confirm that the proposed scheme is able to increase the network lifetime, with a negligible loss in terms of visual analysis accuracy.

Recent advancements in visual sensor technologies have made behavior analysis practical for in-home monitoring systems. The current in-home monitoring systems face several challenges: (1) visual sensor calibration is a difficult task and not practical in real-life because of the need for recalibration when the visual sensors are moved accidentally by a caregiver or the senior citizen, (2) privacy concerns, and (3) the high hardware installation cost. We propose to use a network of cheap low-resolution visual sensors (30×30  pixels) for long-term behavior analysis. The behavior analysis starts by visual feature selection based on foreground/background detection to track the motion level in each visual sensor. Then a hidden Markov model (HMM) is used to estimate the user’s locations without calibration. Finally, an activity discovery approach is proposed using spatial and temporal contexts. We performed experiments on 10 months of real-life data. We show that the HMM approach outperforms the k-nearest neighbor classifier against ground truth for 30 days. Our framework is able to discover 13 activities of daily livings (ADL parameters). More specifically, we analyze mobility patterns and some of the key ADL parameters to detect increasing or decreasing health conditions.

Image sensors with integrated, programmable signal processing execute computationally intensive processing steps during or immediately after image acquisition, thereby allowing for reducing output data to relevant features only. In contrast to conventional image processing systems, the tasks of image acquisition and actual image processing in such a “vision chip” cannot be viewed independently of each other. Both for validating the architecture and supporting programming in the course of application development, modeling on the system level has been performed as part of the design process of the vision-system-on-chip. Apart from the implementation of all essential components of the integrated control unit as well as digital and analog signal processing, special attention has been paid to the integration into the development environment. Being able to purposefully insert parameter deviations and/or defects at different points of the analog processing enables investigations with respect to their influence on image processing algorithms performed on the image sensor. Due to its high simulation speed and compatibility to the real system, especially regarding the to-be-executed programs, the resulting simulation model is very well suited for use in application development.

Multiple-object tracking constitutes a major step in several computer vision applications, such as surveillance, advanced driver assistance systems, and automatic traffic monitoring. Because of the number of cameras used to cover a large area, these applications are constrained by the cost of each node, the power consumption, the robustness of the tracking, the processing time, and the ease of deployment of the system. To meet these challenges, the use of low-power and low-cost embedded vision platforms to achieve reliable tracking becomes essential in networks of cameras. We propose a tracking pipeline that is designed for fixed smart cameras and which can handle occlusions between objects. We show that the proposed pipeline reaches real-time processing on a low-cost embedded smart camera composed of a Raspberry-Pi board and a RaspiCam camera. The tracking quality and the processing speed obtained with the proposed pipeline are evaluated on publicly available datasets and compared to the state-of-the-art methods.

Embedded smart cameras are gaining in popularity for a number of real-time outdoor surveillance applications. However, there are still challenges, i.e., computational latency, variation in illumination, and occlusion. To solve these challenges, multimodal systems, integrating multiple imagers can be utilized. However, trade-off is more stringent requirements on processing and communication for embedded platforms. To meet these challenges, we investigated two low-complexity and high-performance preprocessing architectures for a multiple imagers’ node on a field-programmable gate array (FPGA). In the proposed architectures, majority of the tasks are performed on the thermal images because of the lower spatial resolution. Analysis with different sets of images show that the system with proposed architectures offers better detection performance and can reduce output data from 1.7 to 99 times as compared with full-size images. The proposed architectures can achieve a frame rate of 53 fps, logics utilization from 2.1% to 4.1%, memory consumption 987 to 148 KB and power consumption in the range of 141 to 163 mW on Artix-7 FPGA. This concludes that the proposed architectures offer reduced design complexity and lower processing and communication requirements while retaining the configurability of the system.

We describe a software library we developed for efficiently using the Kinect v2, a time-of-flight RGB-D sensor, with an embedded system, the NVidia Jetson TK1, as a cost-efficient RGB-D smart camera for people detection and tracking. The speed-up needed for achieving real-time operation has been obtained using NVidia CUDA to concurrently generate and process the raw depth and infrared data and to create the three-dimensional point cloud. This library has been released as open source and the smart camera has been tested in real-world scenarios, as a people-detection node in an open-source multinode RGB-D tracking system (OpenPTrack) and onboard a service robot for endowing it with robust people-following capabilities. Moreover, we show that nonembedded computers also can benefit from our library in terms of people-detection frame rate.

Many applications in visual sensor networks (VSNs) demand the low-cost wireless transmission of video data. In this context, distributed video coding (DVC) has proven its potential to achieve state-of-the-art compression performance while maintaining low computational complexity of the encoder. Despite their proven capabilities, current DVC solutions overlook hardware constraints, and this renders them unsuitable for practical implementations. This paper introduces a DVC architecture that offers highly efficient wireless communication in real-world VSNs. The design takes into account the severe computational and memory constraints imposed by practical implementations on low-resolution visual sensors. We study performance-complexity trade-offs for feedback-channel removal, propose learning-based techniques for rate allocation, and investigate various simplifications of side information generation yielding real-time decoding. The proposed system is evaluated against H.264/AVC intra, Motion-JPEG, and our previously designed DVC prototype for low-resolution visual sensors. Extensive experimental results on various data show significant improvements in multiple configurations. The proposed encoder achieves real-time performance on a 1k-pixel visual sensor mote. Real-time decoding is performed on a Raspberry Pi single-board computer or a low-end notebook PC. To the best of our knowledge, the proposed codec is the first practical DVC deployment on low-resolution VSNs.

A method is proposed for fused three-dimensional (3-D) shape estimation and visibility analysis of an unknown, markerless, deforming object through a multicamera vision system. Complete shape estimation is defined herein as the process of 3-D reconstruction of a model through fusion of stereo triangulation data and a visual hull. The differing accuracies of both methods rely on the number and placement of the cameras. Stereo triangulation yields a high-density, high-accuracy reconstruction of a surface patch from a small surface area, while a visual hull yields a complete, low-detail volumetric approximation of the object. The resultant complete 3-D model is, then, temporally projected based on the tracked object’s deformation, yielding a robust deformed shape prediction. Visibility and uncertainty analyses, on the projected model, estimate the expected accuracy of reconstruction at the next sampling instant. In contrast to common techniques that rely on a priori known models and identities of static objects, our method is distinct in its direct application to unknown, markerless, deforming objects, where the object model and identity are unknown to the system. Extensive simulations and comparisons, some of which are presented herein, thoroughly demonstrate the proposed method and its benefits over individual reconstruction techniques.

Active contour model is widely and popularly used in the field of image segmentation because of its superior theoretical properties and efficient numerical methods. An algorithm to segment a ship target in infrared (IR) images using Chan–Vese (C–V) active contour model is proposed here. The method effectively integrates both image regional and boundary information by an adaptable weight function. The method can segment IR ship images, which usually contain noises, blurry boundaries, and heterogeneous regions. In addition, compared with the state-of-the-art methods, experiment results demonstrate the performance and effectiveness of this method.

A graphics processing unit (GPU)-based implementation of a space carving method for the reconstruction of the photo hull is presented. In particular, the generalized voxel coloring with item buffer approach is transferred to the GPU. The fast computation on the GPU is realized by an incrementally calculated standard deviation within the likelihood ratio test, which is applied as color consistency criterion. A fast and efficient computation of complete voxel-pixel projections is provided using volume rendering methods. This generates a speedup of the iterative carving procedure while considering all given pixel color information. Different volume rendering methods, such as texture mapping and raycasting, are examined. The termination of the voxel carving procedure is controlled through an anytime concept. The photo hull algorithm is examined for its applicability to real-world surveillance scenarios as an online reconstruction method. For this reason, a GPU-based redesign of a visual hull algorithm is provided that utilizes geometric knowledge about known static occluders of the scene in order to create a conservative and complete visual hull that includes all given objects. This visual hull approximation serves as input for the photo hull algorithm.

Tracking of a handheld device’s three-dimensional (3-D) position and orientation is fundamental to various application domains, including augmented reality (AR), virtual reality, and interaction in smart spaces. Existing systems still offer limited performance in terms of accuracy, robustness, computational cost, and ease of deployment. We present a low-cost, accurate, and robust system for handheld pose tracking using fused vision and inertial data. The integration of measurements from embedded accelerometers reduces the number of unknown parameters in the six-degree-of-freedom pose calculation. The proposed system requires two light-emitting diode (LED) markers to be attached to the device, which are tracked by external cameras through a robust algorithm against illumination changes. Three data fusion methods have been proposed, including the triangulation-based stereo-vision system, constraint-based stereo-vision system with occlusion handling, and triangulation-based multivision system. Real-time demonstrations of the proposed system applied to AR and 3-D gaming are also included. The accuracy assessment of the proposed system is carried out by comparing with the data generated by the state-of-the-art commercial motion tracking system OptiTrack. Experimental results show that the proposed system has achieved high accuracy of few centimeters in position estimation and few degrees in orientation estimation.

Object detection is a major step in several computer vision applications and a requirement for most smart camera systems. Recent advances in hardware acceleration for real-time object detection feature extensive use of reconfigurable hardware [field programmable gate arrays (FPGAs)], and relevant research has produced quite fascinating results, in both the accuracy of the detection algorithms as well as the performance in terms of frames per second (fps) for use in embedded smart camera systems. Detecting objects in images, however, is a daunting task and often involves hardware-inefficient steps, both in terms of the datapath design and in terms of input/output and memory access patterns. We present how a visual-feature-directed search cascade composed of motion detection, depth computation, and edge detection, can have a significant impact in reducing the data that needs to be examined by the classification engine for the presence of an object of interest. Experimental results on a Spartan 6 FPGA platform for face detection indicate data search reduction of up to 95%, which results in the system being able to process up to 50 1024×768  pixels images per second with a significantly reduced number of false positives.

An effective anisotropic diffusion (AD) mean filter variant is proposed for filtering of salt-and-pepper impulse noise. The implemented filter is robust to impulse noise ranging from low to high density levels. The algorithm involves a switching scheme in addition to utilizing the unsymmetric trimmed mean/median deviation to filter image noise while greatly preserving image edges, regardless of impulse noise density (ND). It operates with threshold parameters selected manually or adaptively estimated from the image statistics. It is further combined with the partial differential equations (PDE)-based AD for edge preservation at high NDs to enhance the properties of the trimmed mean filter. Based on experimental results, the proposed filter easily and consistently outperforms the median filter and its other variants ranging from simple to complex filter structures, especially the known PDE-based variants. In addition, the switching scheme and threshold calculation enables the filter to avoid smoothing an uncorrupted image, and filtering is activated only when impulse noise is present. Ultimately, the particular properties of the filter make its combination with the AD algorithm a unique and powerful edge-preservation smoothing filter at high-impulse NDs.

This paper presents an efficient method for computing a stack of images digitally focused at various lengths from a four-dimensional light field (LF). The main contribution of this work is a fast and algebraically exact method that does not require interpolation in the frequency or spatial domains as alternative methods do. The proposed imaging operator combines two-dimensional (2-D) fast Fourier transform with 2-D fractional Fourier transform and has computational complexity of O(N log N), where N is the number of pixels in the LF tesseract of dimension N=n_x×n_y×n_u×n_v. The whole method consists of unitary vector-based operations; therefore, parallel implementation is easy and can contribute additional speed up. While current state of the art methods suffer from inherent tradeoff between the reconstruction quality and computational complexity, the proposed method benefits of both low-computational complexity and high-reconstruction quality. We also offer a solution for refocusing at distances that are not included in the reconstructed images stack. For such a case, we provide a modified version of our method, which is also algebraically exact and has lower computational complexity than other exact methods.

In order to address the problems of discontinuity and block effect for the dehazing method based on dark channel prior, we improved this method using wavelet decomposition, fast kernel regression model, and bicubic interpolation. First, spatial resolution of the hazy image was reduced by the downsampling method with Haar wavelet decomposition. Second, the fast kernel regression model was proposed to smooth the central transmission with local neighbor transmissions. Last, the smoothed transmission for the approximation image was resized to the hazy image by the bicubic interpolation method. Experiments were carried out on synthetic hazy images with known ground truth and real-world hazy images without ground truth. The regions of sudden change of depth in the dehazed images by our method were more smooth and continuous than those of several state-of-the-art methods, and contrast of our method was higher than that of other methods. Indexes based on the concept of visibility level, mean squared error, and structural similarity of our method were better than those of other methods.

Finger vein recognition technology has significant advantages over other methods in terms of accuracy, uniqueness, and stability, and it has wide promising applications in the field of biometric recognition. We propose using finger creases to locate and extract an object region. Then we use linear fitting to overcome the problem of finger rotation in the plane. The method of modular adaptive histogram equalization (MAHE) is presented to enhance image contrast and reduce computational cost. To extract the finger vein features, we use a fusion method, which can obtain clear and distinguishable vein patterns under different conditions. We used the Hausdorff average distance algorithm to examine the recognition performance of the system. The experimental results demonstrate that MAHE can better balance the recognition accuracy and the expenditure of time compared with three other methods. Our resulting equal error rate throughout the total procedure was 3.268% in a database of 153 finger vein images.

A neighborhood-dependent component feature learning method for regression analysis in single-image superresolution is presented. Given a low-resolution input, the method uses a directional Fourier phase feature component to adaptively learn the regression kernel based on local covariance to estimate the high-resolution image. The unique feature of the proposed method is that it uses image features to learn about the local covariance from geometric similarity between the low-resolution image and its high-resolution counterpart. For each patch in the neighborhood, we estimate four directional variances to adapt the interpolated pixels. This gives us edge information and Fourier phase gives features, which are combined to interpolate using kernel regression. In order to compare quantitatively with other state-of-the-art techniques, root-mean-square error and measure mean-square similarity are computed for the example images, and experimental results show that the proposed algorithm outperforms similar techniques available in the literature, especially at higher resolution scales.

In recent years, many studies consider visual tracking as a two-class classification problem. The key problem is to construct a classifier with sufficient accuracy in distinguishing the target from its background and sufficient generalize ability in handling new frames. However, the variable tracking conditions challenges the existing methods. The difficulty mainly comes from the confused boundary between the foreground and background. This paper handles this difficulty by generalizing the classifier’s learning step. By introducing the distribution data of samples, the classifier learns more essential characteristics in discriminating the two classes. Specifically, the samples are represented in a multiscale visual model. For features with different scales, several large margin distribution machine (LDMs) with adaptive kernels are combined in a Baysian way as a strong classifier. Where, in order to improve the accuracy and generalization ability, not only the margin distance but also the sample distribution is optimized in the learning step. Comprehensive experiments are performed on several challenging video sequences, through parameter analysis and field comparison, the proposed LDM combined ensemble tracker is demonstrated to perform with sufficient accuracy and generalize ability in handling various typical tracking difficulties.

Robust skin color-based moving object detection for video surveillance is proposed. The objective of the proposed algorithm is to detect and track the target under complex situations. The proposed framework comprises four stages, which include preprocessing, skin color-based feature detection, feature classification, and target localization and tracking. In the preprocessing stage, the input image frame is smoothed using averaging filter and transformed into YCrCb color space. In skin color detection, skin color regions are detected using Otsu’s method of global thresholding. In the feature classification, histograms of both skin and nonskin regions are constructed and the features are classified into foregrounds and backgrounds based on Bayesian skin color classifier. The foreground skin regions are localized by a connected component labeling process. Finally, the localized foreground skin regions are confirmed as a target by verifying the region properties, and nontarget regions are rejected using the Euler method. At last, the target is tracked by enclosing the bounding box around the target region in all video frames. The experiment was conducted on various publicly available data sets and the performance was evaluated with baseline methods. It evidently shows that the proposed algorithm works well against slowly varying illumination, target rotations, scaling, fast, and abrupt motion changes.

Most existing intraframe rate control algorithms for the high efficiency video coding (HEVC) standard do not consider the extremely low end-to-end delay required by various applications. An intraframe extremely low-delay rate control scheme for HEVC is presented. First, compared with existing intraframe complexity estimation methods in HEVC, the proposed scheme provides a more accurate complexity estimation method by considering spatial–temporal complexity correlations. Second, based on the proposed complexity estimation method, a linear relationship is modeled between the encoding bit rate and the complexity estimated at the largest coding units (LCU) level. Finally, in order to meet the extremely low-delay constraint, an adaptive bit allocation scheme is proposed, in which the rate control starts as soon as the first LCU row of each frame becomes available. Compared with existing intraframe rate control schemes in HEVC, in which the buffer size is far greater than the number of bits needed for encoding one frame, the proposed rate control scheme has a buffer size that is approximately one-third the average number of bits for encoding one frame in order to achieve extremely low delay. Experimental results show that the proposed scheme can keep the buffer in an extreme low status along with reliable coding performance.

We take a look at crowdsourcing for subjective image quality evaluation using real image stimuli with nonsimulated distortions. Our aim is to scale the task of subjectively rating images while ensuring maximal data validity and accuracy. While previous work has begun to explore crowdsourcing for quality assessment, it has either used images that are not representative of popular consumer scenarios or used crowdsourcing to collect data without comparison to experiments in a controlled environment. Here, we address the challenges imposed by the highly variable online environment, using stimuli that are subtle and more complex than has traditionally been used in quality assessment experiments. In a series of experiments, we vary different design parameters and demonstrate how they impact the subjective responses obtained. Of the parameters examined are stimulus display mode, study length, stimulus habituation, and content homogeneity/heterogeneity. Our method was tested on a database that was rated in a laboratory test previously. Once our design parameters were chosen, we rated a database of consumer photographs and are making this data available to the research community.

Convolutional neural networks (ConvNets) coupled with long short term memory (LSTM) networks have been recently shown to be effective for video classification as they combine the automatic feature extraction capabilities of a neural network with additional memory in the temporal domain. This paper shows how multiview fusion can be applied to such a ConvNet LSTM architecture. Two different fusion techniques are presented. The system is first evaluated in the context of a driver activity recognition system using data collected in a multicamera driving simulator. These results show significant improvement in accuracy with multiview fusion and also show that deep learning performs better than a traditional approach using spatiotemporal features even without requiring any background subtraction. The system is also validated on another publicly available multiview action recognition dataset that has 12 action classes and 8 camera views.

A reduced-reference video quality assessment (VQA) method was proposed by using structural similarity (SSIM) index as a tool to extract features from both the original and the target video sequences, using a reference video pattern. The method is suitable for monitoring the video quality in real time and across the service provision chain. The performance of the proposed method was evaluated using a large experimental set of reference and nonreference video sequences and achieves an accuracy higher than 2.56% in comparison to SSIM. Additionally, comparison to subjectively evaluated scores of Laboratory for Image and Video Engineering video quality dataset, based on difference mean opinion scores, shows that the performance of the proposed method is within the range of the full reference VQA methods.

According to the background, we divide motion detection into two categories: the detection under static background (static camera) and the detection under moving background (moving camera). The motion detection algorithm under a moving camera is more complex than that under a static camera due to the mixture of the motions of target and background. We propose a spatiotemporal image edge analysis method (STIEA) for motion detection under a pan-tilt (PT) camera. First, we obtain the spatiotemporal image (STI) by extracting the corresponding rows from each frame and displaying them one above another after motion estimation. Subsequently, the distribution characteristic (the slope of each edge) of STI is further analyzed theoretically. Overall, the pixels with the same gray value in STI are almost distributed in the same line (linear characteristic). Based on this characteristic, we cluster the edges and detect the abnormal edges. The moving objects exist in the columns with abnormal edges. In addition, the influence of parallax on edges slope is also analyzed. Finally, our method is compared with several existing algorithms. The experimental results demonstrate that our algorithm performs better than others in various scenarios, which provides a unique thought for motion detection under a PT camera.

A multimodal sparse reconstruction approach is proposed for localizing defects in thin plates in Lamb wave-based structural health monitoring. The proposed approach exploits both the sparsity of the defects and the multimodal nature of Lamb wave propagation in plates. It takes into account the variation of the defects’ aspect angles across the various transducer pairs. At low operating frequencies, only the fundamental symmetric and antisymmetric Lamb modes emanate from a transmitting transducer. Asymmetric defects scatter these modes and spawn additional converted fundamental modes. Propagation models are developed for each of these scattered and spawned modes arriving at the various receiving transducers. This enables the construction of modal dictionary matrices spanning a two-dimensional array of pixels representing potential defect locations in the region of interest. Reconstruction of the region of interest is achieved by inverting the resulting linear model using the group sparsity constraint, where the groups extend across the various transducer pairs and the different modes. The effectiveness of the proposed approach is established with finite-element scattering simulations of the fundamental Lamb wave modes by crack-like defects in a plate. The approach is subsequently validated with experimental results obtained from an aluminum plate with asymmetric defects.

Face recognition is a widely used biometric technology due to its convenience but it is vulnerable to spoofing attacks made by nonreal faces such as photographs or videos of valid users. The antispoof problem must be well resolved before widely applying face recognition in our daily life. Face liveness detection is a core technology to make sure that the input face is a live person. However, this is still very challenging using conventional liveness detection approaches of texture analysis and motion detection. The aim of this paper is to propose a feature descriptor and an efficient framework that can be used to effectively deal with the face liveness detection problem. In this framework, new feature descriptors are defined using a multiscale directional transform (shearlet transform). Then, stacked autoencoders and a softmax classifier are concatenated to detect face liveness. We evaluated this approach using the CASIA Face antispoofing database and replay-attack database. The experimental results show that our approach performs better than the state-of-the-art techniques following the provided protocols of these databases, and it is possible to significantly enhance the security of the face recognition biometric system. In addition, the experimental results also demonstrate that this framework can be easily extended to classify different spoofing attacks.

Three-dimensional (3-D) holoscopic imaging, also known as integral imaging, light field imaging, or plenoptic imaging, can provide natural and fatigue-free 3-D visualization. However, a large amount of data is required to represent the 3-D holoscopic content. Therefore, efficient coding schemes for this particular type of image are needed. A 3-D holoscopic image coding scheme with kernel-based minimum mean square error (MMSE) estimation is proposed. In the proposed scheme, the coding block is predicted by an MMSE estimator under statistical modeling. In order to obtain the signal statistical behavior, kernel density estimation (KDE) is utilized to estimate the probability density function of the statistical modeling. As bandwidth estimation (BE) is a key issue in the KDE problem, we also propose a BE method based on kernel trick. The experimental results demonstrate that the proposed scheme can achieve a better rate-distortion performance and a better visual rendering quality.

This study investigates the directed acyclic subgraph (DAS) algorithm, which is used to solve discrete labeling problems much more rapidly than other Markov-random-field-based inference methods but at a competitive accuracy. However, the mechanism by which the DAS algorithm simultaneously achieves competitive accuracy and fast execution speed, has not been elucidated by a theoretical derivation. We analyze the DAS algorithm by comparing it with a message passing algorithm. Graphical models, inference methods, and energy-minimization frameworks are compared between DAS and message passing algorithms. Moreover, the performances of DAS and other message passing methods [sum-product belief propagation (BP), max-product BP, and tree-reweighted message passing] are experimentally compared.

The Consultative Committee for Space Data Systems proposed an efficient image compression standard that can do lossless compression (CCSDS-ICS). CCSDS-ICS is the most widely utilized standard for satellite communications. However, the original CCSDS-ICS is weak in terms of error resilience with even a single incorrect bit possibly causing numerous missing pixels. A restoration algorithm based on the neighborhood similar pixel interpolator is proposed to fill in missing pixels. The linear regression model is used to generate the reference image from other panchromatic or multispectral images. Furthermore, an adaptive search window is utilized to sieve out similar pixels from the pixels in the search region defined in the neighborhood similar pixel interpolator. The experimental results show that the proposed methods are capable of reconstructing missing regions with good visual quality.

The responses of cortical neurons to a stimulus in a classical receptive field (CRF) can be modulated by stimulating the non-CRF (nCRF) of neurons in the primary visual cortex (V1). In the very early stages (at around 40 ms), a neuron in V1 exhibits strong responses to a small set of stimuli. Later, however (after 100 ms), the neurons in V1 become sensitive to the scene’s global organization. As per these visual cortical mechanisms, a contour detection model based on the spatial summation properties is proposed. Unlike in previous studies, the responses of the nCRF to the higher visual cortex that results in the inhibition of the neuronal responses in the primary visual cortex by the feedback pathway are considered. In this model, the individual neurons in V1 receive global information from the higher visual cortex to participate in the inhibition process. Computationally, global Gabor energy features are involved, leading to the more coherent physiological characteristics of the nCRF. We conducted an experiment where we compared our model with those proposed by other researchers. Our model explains the role of the mutual inhibition of neurons in V1, together with an approach for object recognition in machine vision.

An image denoising method via adaptive eigenvectors of graph Laplacian (EGL) is proposed. Unlike the trivial parameter setting of the used eigenvectors in the traditional EGL method, in our method, the eigenvectors are adaptively selected in the whole denoising procedure. In detail, a rough image is first built with the eigenvectors from the noisy image, where the eigenvectors are selected by using the deviation estimation of the clean image. Subsequently, a guided image is effectively restored with a weighted average of the noisy and rough images. In this operation, the average coefficient is adaptively obtained to set the deviation of the guided image to approximately that of the clean image. Finally, the denoised image is achieved by a group-sparse model with the pattern from the guided image, where the eigenvectors are chosen in the error control of the noise deviation. Moreover, a modified group orthogonal matching pursuit algorithm is developed to efficiently solve the above group sparse model. The experiments show that our method not only improves the practicality of the EGL methods with the dependence reduction of the parameter setting, but also can outperform some well-developed denoising methods, especially for noise with large deviations.

Most energy minimization-based restoration methods are developed for signal-independent Gaussian noise. The assumption of Gaussian noise distribution leads to a quadratic data fidelity term, which is appealing in optimization. When an image is acquired with a photon counting device, it contains signal-dependent Poisson or mixed Poisson–Gaussian noise. We quantify the loss in performance that occurs when a restoration method suited for Gaussian noise is utilized for mixed noise. Signal-dependent noise can be treated by methods based on either classical maximum a posteriori (MAP) probability approach or on a variance stabilization approach (VST). We compare performances of these approaches on a large image material and observe that VST-based methods outperform those based on MAP in both quality of restoration and in computational efficiency. We quantify improvement achieved by utilizing Huber regularization instead of classical total variation regularization. The conclusion from our study is a recommendation to utilize a VST-based approach combined with regularization by Huber potential for restoration of images degraded by blur and signal-dependent noise. This combination provides a robust and flexible method with good performance and high speed.

Three-dimensional modeling of scene or object requires registration of multiple range scans, which are obtained by range sensor from different viewpoints. An approach is proposed for scaling registration of multiview range scans via motion averaging. First, it presents a method to estimate overlap percentages of all scan pairs involved in multiview registration. Then, a variant of iterative closest point algorithm is presented to calculate relative motions (scaling transformations) for these scan pairs, which contain high overlap percentages. Subsequently, the proposed motion averaging algorithm can transform these relative motions into global motions of multiview registration. In addition, it also introduces the parallel computation to increase the efficiency of multiview registration. Furthermore, it presents the error criterion for accuracy evaluation of multiview registration result, which can make it easy to compare results of different multiview registration approaches. Experimental results carried out with public available datasets demonstrate its superiority over related approaches.

In visual object tracking, occlusions significantly undermine the performance of tracking algorithms. RGB-D cameras, such as Microsoft Kinect or the related PrimeSense camera, are widely available to consumers. Great attention has been focused on exploiting depth information for object tracking in recent years. We propose an algorithm that improves the existing correlation filter-based tracker for scale-adaptive tracking. Moreover, we utilize depth information provided by the Kinect camera to handle various types of occlusions. First, the optimal location of the target is obtained by the conventional kernelized correlation filter tracker. Then, we make use of the discriminative correlation filter for scale estimation as an independent part. At last, to further improve the tracking performance under occlusions, we present a simple yet effective occlusion handling mechanism to detect occlusion and recovery. In this mechanism, cluster analysis and object segmentation by K-means method have been applied to depth data. Numerous experiments on Princeton RGB-D tracking dataset demonstrate that the proposed algorithm outperforms several state-of-the-art trackers by successfully dealing with occlusions.

For digital imagery, face detection and identification are functions of great importance in wide-ranging applications, including full facial recognition systems. The development and evaluation of unique and existing face detection and face identification applications require a significant amount of data. Increased availability of such data volumes could benefit the formulation and advancement of many biometric algorithms. Here, the utility of using synthetically generated face data to evaluate facial biometry methodologies to a precision that would be unrealistic for a parametrically uncontrolled dataset, is demonstrated. Particular attention is given to similarity metrics, symmetry within and between recognition algorithms, discriminatory power and optimality of pan and/or tilt in reference images or libraries, susceptibilities to variations, identification confidence, meaningful identification mislabelings, sensitivity, specificity, and threshold values. The face identification results, in particular, could be generalized to address shortcomings in various applications and help to inform the design of future strategies.

A single-lens stereo vision system employing a biprism placed in front of the camera will generate unusual distortion in the captured image. Different from the typical image distortions due to lenses, this distortion is mainly induced by the thick biprism and appears to be incompatible with existing lens distortion models. A fully constrained and model-free distortion correction method is proposed. It employs all the projective invariants of a planar checkerboard template as the correction constraints, including straight lines, cross-ratio, and convergence at vanishing point, along with the distortion-free reference point as an additional constraint from the system. The extracted sample points are corrected by minimizing the total cost function formed by all these constraints. With both sets of distorted and corrected points, and the intermediate points interpolated by a local transformation, the correction maps are determined. Thereafter, all the subsequent images could be distortion corrected by the correction maps. This method performs well on the distorted image data captured by the system and shows improvements in accuracy on the camera calibration and depth recovery compared with other correction methods.

Reversible denoising and lifting steps (RDLS) are lifting steps integrated with denoising filters in such a way that, despite the inherently irreversible nature of denoising, they are perfectly reversible. We investigated the application of RDLS to reversible color space transforms: RCT, YCoCg-R, RDgDb, and LDgEb. In order to improve RDLS effects, we propose a heuristic for image-adaptive denoising filter selection, a fast estimator of the compressed image bitrate, and a special filter that may result in skipping of the steps. We analyzed the properties of the presented methods, paying special attention to their usefulness from a practical standpoint. For a diverse image test-set and lossless JPEG-LS, JPEG 2000, and JPEG XR algorithms, RDLS improves the bitrates of all the examined transforms. The most interesting results were obtained for an estimation-based heuristic filter selection out of a set of seven filters; the cost of this variant was similar to or lower than the transform cost, and it improved the average lossless JPEG 2000 bitrates by 2.65% for RDgDb and by over 1% for other transforms; bitrates of certain images were improved to a significantly greater extent.

We propose that both colorimetric and spectral distortion in compressed multispectral images can be reduced by a composite model, named OLCP(W)-X (OptimalLeaders_Color clustering-PCA-W weighted-X coding). In the model, first the spectral–colorimetric clustering is designed for sparse equivalent representation by generating spatial basis. Principal component analysis (PCA) is subsequently used in the manipulation of spatial basis for spectral redundancy removal. Then error compensation mechanism is presented to produce predicted difference image, and finally combined with visual characteristic matrix W, and the created image is compressed by traditional multispectral image coding schemes. We introduce four model-based algorithms to explain their validity. The first two algorithms are OLCPWKWS (OLC-PCA-W-KLT-WT-SPIHT) and OLCPKWS, in which Karhunen–Loeve transform, wavelet transform, and set partitioning in hierarchical trees coding are applied for the created image compression. And the latter two methods are OLCPW-JPEG2000-MCT and OLCP-JPEG2000-MCT. Experimental results show that, compared with the corresponding traditional coding, the proposed OLCPW-X schemes can significantly improve the colorimetric accuracy of rebuilding images under various illumination conditions and generally achieve satisfactory peak signal-to-noise ratio under the same compression ratio. And OLCP-X methods could always ensure superior spectrum reconstruction. Furthermore, our model has excellent performance on user interaction.

Image-based road detection is a vital task for many real-world applications, such as autonomous driving and obstacle detection. We propose a method for segmenting road regions from a single image based on horizon line estimation and clustering technology. The key idea is to leverage normalized cross correlation to search for the line separating the road image. Additionally, we divide the lower part of the road image into several identical parts horizontally and utilize a density-peak clustering algorithm in terms of gray and HSV value of each pixel. Clustering results are further labeled as road and nonroad based on the assumption that two adjacent horizontal parts share similar clustering size and average gray value. For road images with shadows, we also propose a new shadow-free space derived from HSV space. By calculating maximum entropy and reconstructing images, we show that shadows are eliminated to achieve better road detection. Experimental results on four datasets and shadow images demonstrate the effectiveness and robustness of our method.

We improved classic retinal modeling to alleviate the adverse effect of complex illumination on face recognition and extracted robust image features. Our improvements on classic retinal modeling included three aspects. First, a combined filtering scheme was applied to simulate functions of horizontal and amacrine cells for accurate local illumination estimation. Second, we developed an optimal threshold method for illumination classification. Finally, we proposed an adaptive factor acquisition model based on the arctangent function. Experimental results on the combined Yale B; the Carnegie Mellon University poses, illumination, and expression; and the Labeled Face Parts in the Wild databases show that the proposed method can effectively alleviate illumination difference of images under complex illumination conditions, which is helpful for improving the accuracy of face recognition and that of facial feature point detection.

Simple and efficient modifications of the original error-diffusion algorithm are proposed to remove the structural artifact and enhance the real structures. The input image is separated into the flatten and detailed areas based on a gradient-based measure. While a randomization is applied to remove the structural artifact in the flatten area, a gradient-based diffusion modulation is applied to enhance the real structures in the detailed area. Furthermore, tone modulations are introduced to achieve sufficient structural-artifact breaking without producing notable noise. These algorithms are simple and maintain the computational efficiency of the original methods. Numerical experiments show that this method is not only outperforming previous simple approaches but also able to achieve comparable visual quality with those obtained by previous sophisticated and computationally expensive approaches.