A hybrid distortion function for JPEG steganography exploiting block fluctuation and quantization steps is proposed. To resist multidomain steganalysis, both spatial domain and discrete cosine transformation (DCT) domain are involved in the proposed distortion function. In spatial domain, a distortion value is allotted for each 8×8 block according to block fluctuation. In DCT domain, quantization steps are employed to allot distortion values for DCT coefficients in a block. The two elements, block distortion and quantization steps, are combined together to measure the embedding risk. By employing the syndrome trellis coding to embed secret data, the embedding changes are constrained in complex regions, where modifications are hard to be detected. When compared to current state-of-the-art steganographic methods for JPEG images, the proposed method presents less detectable artifacts.

We present a graph-based classification approach called sparse graph-based inductive learning (SGIL). Different to the conventional graph-based classifiers, which perform the classification in a semisupervised way, SGIL is a purely supervised method whose classifier is totally learned in an inductive fashion instead of transductive fashion. Similar to the idea of sparse graph-based classifier, SGIL constructs a sparse graph to encode the correlations of training samples, and considers the classification issue as a regularized sparse graph partition issue where the optimal graph cut should not only minimize the correlation loss of the training samples but also minimize the classification errors. Essentially, the learned graph cut plays a role as the predicted labels here. Thus, a linear classifier can be inductively derived by learning a mapping between the training samples and the graph cuts. Since SGIL is purely supervised, it enjoys several desirable properties over the semisupervised ones in graph construction and model training. We evaluate our work on several popular image datasets. The experimental results demonstrate its superiority.

The depth modeling mode (DMM) is a new coding tool in the three-dimensional extension of high-efficiency video coding (3D-HEVC) standard to improve the coding of depth maps. In DMM, the current depth block is partitioned into two nonrectangular partitions and each partition is predicted by a constant partition value (CPV). A searching algorithm for the optimal CPVs has been provided in the 3D-HEVC reference software, which, however, has high complexity and is time-consuming. This letter shows that the distortion function in DMM is convex and proposes a fast early termination algorithm for searching the optimal CPVs without losing its optimality. Theoretical analysis of the method is provided, and experimental results show that our method can save 20% of search time with the same coding performance.

This PDF file contains the editorial “Special Section Guest Editorial:Intelligent Surveillance for Transport Systems” for JEI Vol. 25 Issue 05

The multiview appearance of road signs in uncontrolled environments has made the detection of road signs a challenging problem in computer vision. We propose a road sign detection method to detect multiview road signs. This method is based on several algorithms, including the classical cascaded detector, the self-adaptive weighted Gaussian color model (SW-Gaussian model), and a shape context matching method. The classical cascaded detector is used to detect the frontal road signs in video sequences and obtain the parameters for the SW-Gaussian model. The proposed SW-Gaussian model combines the two-dimensional Gaussian model and the normalized red channel together, which can largely enhance the contrast between the red signs and background. The proposed shape context matching method can match shapes with big noise, which is utilized to detect road signs in different directions. The experimental results show that compared with previous detection methods, the proposed multiview detection method can reach higher detection rate in detecting signs with different directions.

This work presents a vision-based data collection system for pedestrian behavior analysis at intersections that include crossing counts, waiting time, crossing speed, and facility utilization. The tracking system uses contextual fusion of motion and appearance cues to more reliably track pedestrians during stop-and-go movements at intersections. Moreover, the pedestrian tracking system is improved through cooperation of two different tracking algorithms: bipartite graph match and optical flow algorithms. The performance of the proposed system is evaluated separately at the detection and tracking steps followed by behavior analyses of pedestrians for three different intersection videos of Las Vegas. The experimental results show the efficiency of the proposed system and intersection utilization is depicted through heat maps.

In traffic engineering, vehicle detectors are trained on limited datasets, resulting in poor accuracy when deployed in real-world surveillance applications. Annotating large-scale high-quality datasets is challenging. Typically, these datasets have limited diversity; they do not reflect the real-world operating environment. There is a need for a large-scale, cloud-based positive and negative mining process and a large-scale learning and evaluation system for the application of automatic traffic measurements and classification. The proposed positive and negative mining process addresses the quality of crowd sourced ground truth data through machine learning review and human feedback mechanisms. The proposed learning and evaluation system uses a distributed cloud computing framework to handle data-scaling issues associated with large numbers of samples and a high-dimensional feature space. The system is trained using AdaBoost on 1,000,000 Haar-like features extracted from 70,000 annotated video frames. The trained real-time vehicle detector achieves an accuracy of at least 95% for 1/2 and about 78% for 19/20 of the time when tested on ∼7,500,000 video frames. At the end of 2016, the dataset is expected to have over 1 billion annotated video frames.

The number of vehicles in circulation in modern urban centers has greatly increased, which motivates the development of automatic traffic monitoring systems. Consequently, camera-based traffic monitoring systems are becoming more widely used, since they offer important technological advantages in comparison with traditional traffic monitoring systems (e.g., simpler maintenance and more flexibility for the design of practical configurations). The segmentation of the foreground (i.e., vehicles) is a fundamental step in the workflow of a camera-based traffic monitoring system. However, foreground segmentation can be negatively affected by vehicle shadows. This paper discusses the types of shadow detection methods available in the literature, their advantages, disadvantages, and in which situations these methods can improve camera-based vehicle detection for traffic monitoring. In order to compare the performance of these different types of shadow detection methods, experiments are conducted with typical methods of each category using publicly available datasets. This work shows that shadow detection definitely can improve the reliability of traffic monitoring systems, but the choice of the type of shadow method depends on the system specifications (e.g., tolerated error), the availability of computational resources, and prior information about the scene and its illumination in regular operation conditions.

This paper presents a state-of-the-art, vision-based vehicle detection and type classification to perform traffic surveys from a roadside closed-circuit television camera. Vehicles are detected using background subtraction based on a Gaussian mixture model that can cope with vehicles that become stationary over a significant period of time. Vehicle silhouettes are described using a combination of shape and appearance features using an intensity-based pyramid histogram of orientation gradients (HOG). Classification is performed using a support vector machine, which is trained on a small set of hand-labeled silhouette exemplars. These exemplars are identified using a model-based preclassifier that utilizes calibrated images mapped by Google Earth to provide accurately surveyed scene geometry matched to visible image landmarks. Kalman filters track the vehicles to enable classification by majority voting over several consecutive frames. The system counts vehicles and separates them into four categories: car, van, bus, and motorcycle (including bicycles). Experiments with real-world data have been undertaken to evaluate system performance and vehicle detection rates of 96.45% and classification accuracy of 95.70% have been achieved on this data.

The article is dedicated to the presentation of a vision-based system for road vehicle counting and classification. The system is able to achieve counting with a very good accuracy even in difficult scenarios linked to occlusions and/or presence of shadows. The principle of the system is to use already installed cameras in road networks without any additional calibration procedure. We propose a robust segmentation algorithm that detects foreground pixels corresponding to moving vehicles. First, the approach models each pixel of the background with an adaptive Gaussian distribution. This model is coupled with a motion detection procedure, which allows correctly location of moving vehicles in space and time. The nature of trials carried out, including peak periods and various vehicle types, leads to an increase of occlusions between cars and between cars and trucks. A specific method for severe occlusion detection, based on the notion of solidity, has been carried out and tested. Furthermore, the method developed in this work is capable of managing shadows with high resolution. The related algorithm has been tested and compared to a classical method. Experimental results based on four large datasets show that our method can count and classify vehicles in real time with a high level of performance (98%) under different environmental situations, thus performing better than the conventional inductive loop detectors.

This paper presents a tracking method to detect and track independently moving targets, attempting to traverse the railway, in monocular camera sequences. This method is capable of tracking the maximum number of pixels belonging to an object. The method starts by detecting and separating moving objects due to background subtraction and an energy vector-based clustering. Next, the method performs the step of tracking locally. Tracking starts by generating initial optical flow of all object pixels by propagating the optical flow of Harris corner points (calculated by Lucas–Kanade technique) using normal distribution. An iterative procedure, including Kalman filtering with adaptive parameters, color intensity difference-based optimization, and validation constraints, is then implemented to reach precise and robust optical flow estimation for the majority of the pixels of the tracked objects. Different experimental results are presented, evaluated, and discussed to show the effectiveness of the method of tracking objects that may move in complex and overlapping trajectories.

A gyroscope and visual fusion solution for digital video stabilization (DVS) is presented. The solution classifies DVS-related motions into three types: the object motion (OM) in the world space, the camera motion in the camera space (CS), and the pixel motion in the image space (IS). The camera rotation is estimated by gyroscope and smoothed in the CS, while the camera translation is compounded with the OM and smoothed together in the IS. The main contributions of this paper lie in two aspects: (1) propose an inertial and visual fusion method to stabilize both rotational and translational jitters and (2) the fusion method is simple and fast in computation and can be suitable for smart terminals. Experimental results show that the proposed solution performs well in video stabilization.

We propose a robust local L_2,1 tracker based on red-green-blue (RGB) color channel fusion. In this tracker, the object is first divided into some patches with overlap region, and then the local sparse optimization from RGB color channels in each patch is solved via the L_2,1 mixed-norm regularization, which can realize the fusion of multicolor channel information. In the calculation of candidate object confidence, the confidences from RGB color channels are fused to obtain the total confidence of candidate object, this can accurately select the best candidate object. In the update module of template set and dictionary database, we design an adaptive update mechanism. The template and dictionary to delete are determined by sorting the cosine similarities between the tracking result and templates while the update of dictionary database is completed by replacing the old dictionaries with the reconstruction results of all the patches corresponding to the tracking result. This update method can effectively adapt to the appearance change of the object, and it can alleviate the tracking drift. Both qualitative and quantitative evaluations on challenging video sequences demonstrate that the tracker proposed is reliable and effective. It performs favorably against several state-of-the-art methods.

Depth edges play an important role in depth image upsampling. Many recent upsampling methods rely on the prior images of depth edges to preserve sharp depth edges in restored depth images. However, recent depth edge detection methods are not robust against the noise in depth images. Some methods are also too time-consuming. We propose a method to efficiently detect edges in depth images. The proposed method is very simple but very robust against the noise in depth images. It is also fast and has near δ(1) implementation. We apply the proposed method to the existing edge guided depth image upsampling. Experimental results on both simulated and real data show the effeδctiveness of the proposed method.

The emerging compressive sensing (CS) theory provides a paradigm for image compression. Most current efforts in CS-based image compression have been focused on enhancing the objective coding efficiency. In order to achieve a maximal perceptual quality under the measurements budget constraint, we propose a perceptual rate-distortion optimized (RDO) CS-based image codec in this paper. By incorporating both the human visual system characteristics and the signal sparsity into a RDO model designed for the block compressive sensing framework, the measurements allocation for each block is formulated as an optimization problem, which can be efficiently solved by the Lagrangian relaxation method. After the optimal measurement number is determined, each block is adaptively sampled using an image-dependent measurement matrix. To make our proposed codec applicable to different scenarios, we also propose two solutions to implement the perceptual RDO measurements allocation technique: one at the encoder side and the other at the decoder side. The experimental results show that our codec outperforms the other existing CS-based image codecs in terms of both objective and subjective performances. In particular, our codec can also achieve a low complexity encoder by adopting the decoder-based solution for the perceptual RDO measurements allocation.

Character segmentation (CS) plays an important role in automatic license plate recognition and has been studied for decades. A method using multiscale template matching is proposed to settle the problem of CS for Chinese license plates. It is carried out on a binary image integrated from maximally stable extremal region detection and Otsu thresholding. Afterward, a uniform harrow-shaped template with variable length is designed, by virtue of which a three-dimensional matching space is constructed for searching of candidate segmentations. These segmentations are detected at matches with local minimum responses. Finally, the vertical boundaries of each single character are located for subsequent recognition. Experiments on a data set including 2349 license plate images of different quality levels show that the proposed method can achieve a higher accuracy at comparable time cost and is robust to images in poor conditions.

Building a robust appearance model is useful to improve tracking performance. We propose a hierarchical multiview learning framework to construct the appearance model, which has two layers for tracking. On the top layer, two different views of features, grayscale value and histogram of oriented gradients, are adopted for representation under the cotraining framework. On the bottom layer, for each view of each feature, three different random subspaces are generated to represent the appearance from multiple views. For each random view submodel, the least squares support vector machine is employed to improve the discriminability for concrete and efficient realization. These two layers are combined to construct the final appearance model for tracking. The proposed hierarchical model assembles two types of multiview learning strategies, in which the appearance can be described more accurately and robustly. Experimental results in the benchmark dataset demonstrate that the proposed method can achieve better performance than many existing state-of-the-art algorithms.

Gray-level co-occurrence matrix (GLCM) is one of the most used methods for texture representation. As it can be computed only from gray-level images, a significant amount of information that could be provided by color is totally ignored. We propose a generalization of GLCM from gray level to hue saturation value color space, which we refer to as modified integrative color intensity co-occurrence matrix (MICICM). To reach such a generalization, a mapping function, which determines for each pixel value the bin it falls into, is needed. In many previous studies, this function uses a hard mapping where all pixel values that fall in a bin are considered as the same, regardless of their values. This presents a number of drawbacks. To fix them, we introduce a color and gray-level mapping scheme based on a set of weight assignment functions we propose. In our scheme, each pixel is mapped to more than one possible color (and gray-level) bin, to avoid the drawbacks of hard mapping. Although a fuzzy-based scheme has been recently proposed, our MICICM has successfully outperformed it and those of the state of the art. Our findings make several noteworthy contributions to image representation.

Image blending plays an important role in video mosaicking, which has a high demand for real-time performance and visual quality. This paper proposes a fast blending method based on Bresenham algorithm, which realizes blending by controlling the storing addresses of source pixels. The starting storing location is accurately computed based on the coordinates of the middle pixel of the seam instead of the first pixel’s, reducing the accumulated error along the seam significantly. The other storing addresses are acquired using a variable-step Bresenham method, which takes advantage of burst mode operation of a dynamic memory and can achieve a good trade-off between the operation convenience and memory requirement. By the proposed method, complicated calculations of storing addresses are simplified into integer additions and subtractions, which is more suitable for hardware implementation. A hardware architecture based on field programmable gate array is presented to evaluate the proposed method with clock frequency analysis and resource assessment. The experimental results show that the proposed method achieves good performance of high image quality, low computational complexity, and low memory requirement.

Object segmentation is an important but highly challenging problem in computer vision and image processing. An attention shift-based multiple saliency object segmentation model, called ASMSO, is introduced. The proposed ASMSO could produce a pool of potential object regions for each saliency object and be applicable to multiple saliency object segmentation. The potential object regions are produced by combing the methods of gPb-owt-ucm and min-cut graph, whereas the saliency objects are detected by a visual attention model with an attention shift mechanism. In order to deal with various scenes, the model attention shift-based multiple saliency object segmentation (ASMSO) contains different features which include not only traditional features, such as color, uniform, and texture, but also a new position feature originating from proximity of Gestalt theory. Experiments on the training set of PASCAL VOC2012 segmentation dataset not only show that traditional color feature and the proposed position feature work much better than features of texture and uniformity, but also prove that ASMSO is suitable for multiple object segmentation. In addition, experiments on a traditional saliency dataset show that ASMSO could also be applied to traditional saliency object segmentation and performs much better than the state-of-the-art method.

Thousands of pavement images are collected by road authorities daily for condition monitoring surveys. These images typically have intensity variations and texture nonuniformities that make their segmentation challenging. The automated segmentation of such pavement images is crucial for accurate, thorough, and expedited health monitoring of roads. In the pavement monitoring area, well-known texture descriptors, such as gray-level co-occurrence matrices and local binary patterns, are often used for surface segmentation and identification. These, despite being the established methods for texture discrimination, are inherently slow. This work evaluates Laws texture energy measures as a viable alternative for pavement images for the first time. k-means clustering is used to partition the feature space, limiting the human subjectivity in the process. Data classification, hence image segmentation, is performed by the k-nearest neighbor method. Laws texture energy masks are shown to perform well with resulting accuracy and precision values of more than 80%. The implementations of the algorithm, in both MATLAB® and OpenCV/C++, are extensively compared against the state of the art for execution speed, clearly showing the advantages of the proposed method. Furthermore, the OpenCV-based segmentation shows a 100% increase in processing speed when compared to the fastest algorithm available in literature.

Single-video-based temporal super-resolution reconstruction (SRR) can get rid of constraints imposed on multiple-video-based temporal SRR, including device scarcity, temporal synchronization, complicated registration, and high cost. But frame interpolation in single-video-based temporal SRR cannot deblur well, and the selection or determination related to interpolation function has a lack of evidence, which reduces the fidelity. Additionally, the subsequent spatial deblurring is suboptimal for the motion blur in video because the formation of the motion blur is different from that of the spatial blur. This paper proposes a temporal SRR based on pixel stream and featured prior model to increase the frame rate and the definition of motion blurred single video. The proposed temporal SRR views the single video as a bundle of pixel streams and implements maximum-a-posteriori-based pixel stream SRR stream-by-stream without the selection or determination related to interpolation function. What is more, a featured temporal prior about the pixel stream is proposed and introduced into the pixel stream SRR, which can well remove the motion blur. Experimental results show the proposed temporal SRR is effective.

Reduced-reference (RR) and no-reference (NR) models for video quality estimation, using features that account for the impact of coding artifacts, spatio-temporal complexity, and packet losses, are proposed. The purpose of this study is to analyze a number of potentially quality-relevant features in order to select the most suitable set of features for building the desired models. The proposed sets of features have not been used in the literature and some of the features are used for the first time in this study. The features are employed by the least absolute shrinkage and selection operator (LASSO), which selects only the most influential of them toward perceptual quality. For comparison, we apply feature selection in the complete feature sets and ridge regression on the reduced sets. The models are validated using a database of H.264/AVC encoded videos that were subjectively assessed for quality in an ITU-T compliant laboratory. We infer that just two features selected by RR LASSO and two bitstream-based features selected by NR LASSO are able to estimate perceptual quality with high accuracy, higher than that of ridge, which uses more features. The comparisons with competing works and two full-reference metrics also verify the superiority of our models.

Blind image deblurring is one of the challenging problems in image processing and computer vision. The main purpose of blind image deblurring is to estimate the correct blur kernel and restore the latent image with edge-preservation, details-protection, and ringing suppression. In order to achieve ideal results, an innovative compound L₀-regularized model is proposed to estimate the blur kernel by regularizing the sparsity property of natural images and two characteristics of blur kernel, such as continuity and sparsity. In the alternating direction framework, the split Bregman algorithm and half-quadratic splitting rule are alternatively employed to optimize the proposed kernel estimation model. Finally, a nonblind restoration method with ringing suppression is developed to obtain the ultimate latent image. Extensive experiments demonstrate the efficiency and viability of the proposed method compared with some state-of-the-art blind deblurring methods.

Predicting human visual perception has several applications such as compression, rendering, editing, and retargeting. Current approaches, however, ignore the fact that the human visual system compensates for geometric transformations, e.g., we see that an image and a rotated copy are identical. Instead, they will report a large, false-positive difference. At the same time, if the transformations become too strong or too spatially incoherent, comparing two images gets increasingly difficult. Between these two extrema, we propose a system to quantify the effect of transformations, not only on the perception of image differences but also on saliency and motion parallax. To this end, we first fit local homographies to a given optical flow field, and then convert this field into a field of elementary transformations, such as translation, rotation, scaling, and perspective. We conduct a perceptual experiment quantifying the increase of difficulty when compensating for elementary transformations. Transformation entropy is proposed as a measure of complexity in a flow field. This representation is then used for applications, such as comparison of nonaligned images, where transformations cause threshold elevation, detection of salient transformations, and a model of perceived motion parallax. Applications of our approach are a perceptual level-of-detail for real-time rendering and viewpoint selection based on perceived motion parallax.

Various approaches have been proposed for robust visual tracking, among which compressive tracking (CT) yields promising performance. In CT, Haar-like features are efficiently extracted with a very sparse measurement matrix and modeled as an online updated naïve Bayes classifier to account for target appearance change. The naïve Bayes classifier ignores overlap between Haar-like features and assumes that Haar-like features are independently distributed, which leads to drift in complex scenario. To address this problem, we present an extended CT algorithm, which assumes that all Haar-like features are correlated with each other and have multivariate Gaussian distribution. The mean vector and covariance matrix of multivariate normal distribution are incrementally updated with constant computational complexity to adapt to target appearance change. Each frame is associated with a temporal weight to expend less modeling power on old observation. Based on temporal weight, an update scheme with changing but convergent learning rate is derived with strict mathematic proof. Compared with CT, our extended algorithm achieves a richer representation of target appearance. The incremental multivariate Gaussian distribution is integrated into the particle filter framework to achieve better tracking performance. Extensive experiments on the CVPR2013 tracking benchmark demonstrate that our proposed tracker achieves superior performance both qualitatively and quantitatively over several state-of-the-art trackers.

Techniques which exploit properties such as sparsity and total variation have provided the ability to reconstruct images that surpass the conventional limits of imaging. This leads to difficulties in assessing the result, as conventional metrics for resolution are no longer valid. We develop a numerical approach to evaluating the second-order statistics of the estimate by relating a confidence interval on the solution to a confidence interval on a pixel value, and from this we formulate an approach to estimating the spatial resolution. With this estimate, we can calculate the resolution at each point subject to chosen bounds on the desired precision and confidence. We demonstrate the method for limited-angle tomographic reconstructions utilizing nonnegativity, sparse regularization, total-variation minimization, and their combinations. This provides a means to visualize and understand the effect on the image inherent in these penalties and constraints. Examples are provided using simulated data for different methods, and the results are shown to agree with resolution calculated empirically via the local edge response.

Targets of interest are different in various applications in which manmade targets, such as aircraft, ships, and buildings, are given more attention. Manmade target extraction methods using synthetic aperture radar (SAR) images are designed in response to various demands, which include civil uses, business purposes, and military industries. This plays an increasingly vital role in monitoring, military reconnaissance, and precision strikes. Achieving accurate and complete results through traditional methods is becoming more challenging because of the scattered complexity of polarization in polarimetric synthetic aperture radar (PolSAR) image. A multistage decision-based method is proposed composed of power decision, dominant scattering mechanism decision, and reflection symmetry decision. In addition, the theories of polarimetric contrast enhancement, generalized Y decomposition, and maximum eigenvalue ratio are applied to assist the decision. Fully PolSAR data are adopted to evaluate and verify the approach. Experimental results show that the method can achieve an effective result with a lower false alarm rate and clear contours. Finally, on this basis, a universal framework of change detection for manmade targets is presented as an application of our method. Two sets of measured data are also used to evaluate and verify the effectiveness of the change-detection algorithm.

Scene text recognition has gained significant attention in the computer vision community. Character detection and recognition are the promise of text recognition and affect the overall performance to a large extent. We proposed a good initialization model for scene character recognition from cropped text regions. We use constrained character’s body structures with deformable part-based models to detect and recognize characters in various backgrounds. The character’s body structures are achieved by an unsupervised discriminative clustering approach followed by a statistical model and a self-build minimum spanning tree model. Our method utilizes part appearance and location information, and combines character detection and recognition in cropped text region together. The evaluation results on the benchmark datasets demonstrate that our proposed scheme outperforms the state-of-the-art methods both on scene character recognition and word recognition aspects.

An adaptive edge detection and mapping (AEDM) algorithm to address the challenging one-dimensional barcode recognition task with the existence of both image degradation and barcode shape deformation is presented. AEDM is an edge detection-based method that has three consecutive phases. The first phase extracts the scan lines from a cropped image. The second phase involves detecting the edge points in a scan line. The edge positions are assumed to be the intersecting points between a scan line and a corresponding well-designed reference line. The third phase involves adjusting the preliminary edge positions to more reasonable positions by employing prior information of the coding rules. Thus, a universal edge mapping model is established to obtain the coding positions of each edge in this phase, followed by a decoding procedure. The Levenberg–Marquardt method is utilized to solve this nonlinear model. The computational complexity and convergence analysis of AEDM are also provided. Several experiments were implemented to evaluate the performance of AEDM algorithm. The results indicate that the efficient AEDM algorithm outperforms state-of-the-art methods and adequately addresses multiple issues, such as out-of-focus blur, nonlinear distortion, noise, nonlinear optical illumination, and situations that involve the combinations of these issues.

Interregion contrast is crucial for image segmentation, but it has not been explicitly incorporated into popular region-based active contour models, such as Chan–Vese model and region-scalable fitting model. We integrate the interregion contrast into the active contour model in a level set formulation. The major contribution of this paper is weighting the interregion contrast by a labeling function to define a difference energy. The labeling function controls the integration region of the difference energy. It enables the proposed method to be applicable to images with intensity inhomogeneity, since the high contrast between local regions on the two sides of the object boundary. Moreover, the major part of the data term can be computed out of iterations with the labeling function, which reduces much computation time. The integration of global and local contrast into the energy functional also accelerates the curve propagation. Experiments and comparisons conducted on synthetic and real images demonstrate both accuracy and efficiency of the proposed method.

Video stereolization has received much attention in recent years due to the lack of stereoscopic three-dimensional (3-D) contents. Although video stereolization can enrich stereoscopic 3-D contents, it is hard to achieve automatic two-dimensional-to-3-D conversion with less computational cost. We proposed rapid learning-based video stereolization using a graphic processing unit (GPU) acceleration. We first generated an initial depth map based on learning from examples. Then, we refined the depth map using saliency and cross-bilateral filtering to make object boundaries clear. Finally, we performed depth-image-based-rendering to generate stereoscopic 3-D views. To accelerate the computation of video stereolization, we provided a parallelizable hybrid GPU–central processing unit (CPU) solution to be suitable for running on GPU. Experimental results demonstrate that the proposed method is nearly 180 times faster than CPU-based processing and achieves a good performance comparable to the-state-of-the-art ones.

Nowadays, typical active contour models are widely applied in image segmentation. However, they perform badly on real images with inhomogeneous subregions. In order to overcome the drawback, this paper proposes an edge-preserving smoothing image segmentation algorithm. At first, this paper analyzes the edge-preserving smoothing conditions for image segmentation and constructs an edge-preserving smoothing model inspired by total variation. The proposed model has the ability to smooth inhomogeneous subregions and preserve edges. Then, a kind of clustering algorithm, which reasonably trades off edge-preserving and subregion-smoothing according to the local information, is employed to learn the edge-preserving parameter adaptively. At last, according to the confidence level of segmentation subregions, this paper constructs a smoothing convergence condition to avoid oversmoothing. Experiments indicate that the proposed algorithm has superior performance in precision, recall, and F-measure compared with other segmentation algorithms, and it is insensitive to noise and inhomogeneous-regions.

Geometric consistency is, usually, considered as a postprocessing step to filter matched sets of local features in order to discard outliers. In this work, it is used to propose an adaptive feature that describes the geometric dispersion of keypoints. It is based on a distribution computed by a nonparametric estimator so that no assumption is made about the data. We investigated and discussed the invariance properties of our descriptor under the most common two- and three-dimensional transformations. Then, we applied it to flower recognition. The classification is performed using the precomputed kernel of support vector machines classifier. Indeed, a similarity computing framework that uses the Kullback–Leibler divergence is presented. Furthermore, a customized layout for each flower image is designed to describe and compare separately the boundary and the central area of flowers. Experimentations made on the Oxford flower-17 dataset prove the efficiency of our method in terms of classification accuracy and computational complexity. The limits of our descriptor are also discussed on a 10-class subset of the Oxford flower-102 dataset.

Human fatigue is an important cause of traffic accidents. To improve the safety of transportation, we propose, in this paper, a framework for fatigue expression recognition using image-based facial dynamic multi-information and a bimodal deep neural network. First, the landmark of face region and the texture of eye region, which complement each other in fatigue expression recognition, are extracted from facial image sequences captured by a single camera. Then, two stacked autoencoder neural networks are trained for landmark and texture, respectively. Finally, the two trained neural networks are combined by learning a joint layer on top of them to construct a bimodal deep neural network. The model can be used to extract a unified representation that fuses landmark and texture modalities together and classify fatigue expressions accurately. The proposed system is tested on a human fatigue dataset obtained from an actual driving environment. The experimental results demonstrate that the proposed method performs stably and robustly, and that the average accuracy achieves 96.2%.

Texture information plays an important role in rendering true objects, especially with the wide application of image-based three-dimensional (3-D) reconstruction and 3-D laser scanning. This paper proposes a seamless texture mapping algorithm to achieve a high-quality visual effect for 3-D reconstruction. At first, a series of image sets is produced by analyzing the visibility of triangular facets, the image sets are clustered and segmented into a number of optimal reference texture patches. Second, the generated texture patches are sequenced to create a rough texture map, then a weighting process is adopted to reduce the color discrepancies between adjacent patches. Finally, a multiresolution decomposition and fusion technique is used to generate the transition section and eliminate the boundary effect. Experiments show that the proposed algorithm is effective and practical for obtaining high-quality 3-D texture mapping for 3-D reconstruction. Compared with traditional methods, it maintains the texture clarity while eliminating the color seams, in addition, it also supports 3-D texture mapping for big data application.

A set of invariant quaternion moments based on an adaptation of the three-dimensional (3-D) spherical harmonic transform (SHT) for describing two-dimensional color shapes is proposed. The use of quaternions to deal with the color part is beneficial in the way the three color components are integrated in a single feature. An adequate mapping from the 3-D SHT to the unit disc allows a fast and accurate computation of the proposed moments. Experiments are conducted to evaluate the performance of the obtained moments in terms of color image reconstruction, robustness to geometric and photometric transformations, content-based color shape retrieval, and computation time. For this purpose, two image databases (COIL-100 and ALOI) are used. Results illustrate the effectiveness of the proposed moments in dealing with the color information.

We propose power constrained contrast enhancement based on brightness compensated contrast-tone mapping operation (BCCTMO). We adopt brightness compensation to restore the original perceptual luminance and combine tone mapping with contrast enhancement to improve image quality. First, we increase pixel values to compensate for the reduced brightness caused by backlight dimming while maintaining the perceived luminance. Then we perform a contrast-tone mapping operation to reduce the information loss caused by the brightness compensation and enhance contrast in images. Finally, we conduct color scaling to reproduce natural colors. We implement BCCTMO on an Android tablet with an LCD display using mathematical approximation. Experimental results demonstrate that BCCTMO produces high-quality images while minimizing distortions and saving power compared with state-of-the-art methods.

Currently, most video resources on line are encoded in the H.264/AVC format. More fluent video transmission can be obtained if these resources are encoded in the newest international video coding standard: high efficiency video coding (HEVC). In order to improve the video transmission and storage on line, a transcoding method from H.264/AVC to HEVC is proposed. In this transcoding algorithm, the coding information of intraprediction, interprediction, and motion vector (MV) in H.264/AVC video stream are used to accelerate the coding in HEVC. It is found through experiments that the region of interprediction in HEVC overlaps that in H.264/AVC. Therefore, the intraprediction for the region in HEVC, which is interpredicted in H.264/AVC, can be skipped to reduce coding complexity. Several macroblocks in H.264/AVC are combined into one PU in HEVC when the MV difference between two of the macroblocks in H.264/AVC is lower than a threshold. This method selects only one coding unit depth and one prediction unit (PU) mode to reduce the coding complexity. An MV interpolation method of combined PU in HEVC is proposed according to the areas and distances between the center of one macroblock in H.264/AVC and that of the PU in HEVC. The predicted MV accelerates the motion estimation for HEVC coding. The simulation results show that our proposed algorithm achieves significant coding time reduction with a little loss in bitrates distortion rate, compared to the existing transcoding algorithms and normal HEVC coding.

Obstacle avoidance is an important and challenging task for the autonomous flight of unmanned aerial vehicles. Obstacle regions extraction from image sequences is a critical prerequisite in obstacle avoidance. We propose an obstacle regions extraction method based on space–time tensor descriptor. In our method, first, the space–time tensor descriptor is defined and a criterion function based on the descriptor of extracting space–time interest points (STIPs) is designed. Then a self-adaptive clustering of STIPs approach is presented to locate the possible obstacle regions. Finally, an improved level set algorithm is applied with the result of clustering to extract the obstacle regions. We demonstrate the experiments of obstacle regions extraction by our method on image sequences. Sequences are captured in indoor simulative obstacle avoidance environments and outdoor real flight obstacle avoidance environments. Experimental results validate that our method can effectively complete extraction and segmentation of obstacle region with captured images. Compared with the state-of-the-art methods, our method performs well to extract the contours of obstacle regions on the whole and significantly improves segmentation speed.

To improve object recognition capabilities in applications, we used orthogonal design (OD) to choose a group of optimal parameters in the parameter space of scale invariant feature transform (SIFT). In the case of global optimization (GOP) and local optimization (LOP) objectives, our aim is to show the operation of OD on the SIFT method. The GOP aims to increase the number of correctly detected true matches (NoCDTM) and the ratio of NoCDTM to all matches. In contrast, the LOP mainly aims to increase the performance of recall–precision. In detail, we first abstracted the SIFT method to a 9-way fixed-effect model with an interaction. Second, we designed a mixed orthogonal array, MA(64,2³4²⁰,2), and its header table to optimize the SIFT parameters. Finally, two groups of parameters were obtained for GOP and LOP after orthogonal experiments and statistical analyses were implemented. Our experiments on four groups of data demonstrate that compared with the state-of-the-art methods, GOP can access more correct matches and is more effective against object recognition. In addition, LOP is favorable in terms of the recall–precision.

Image recoloring is the process of modification and adjustment of color appearance in images. Existing methods address the recoloring of a single image. We propose a method for recoloring stereoscopic images. Naively recoloring each image independently will require a pair of strokes in the source stereoscopic image pair. However, it is difficult to require consistent strokes on both the left and right views. We show how to extend a single image recoloring to work on stereoscopic images. Our method requires only a few user strokes on the left view and automatically transfers the corresponding strokes to the right view. Then a nonlocal color linear model optimization assumption is designed. Our nonlocal color linear model inherits the advantages of global and local color propagation methods. Our approach can propagate color cues in a global manner which can propagate color relatively far from the provided color constraints, while it provides the user with good local control. The experimental results show that the recolorized image pair is geometrically consistent with the original one.

Addressing the problems of moving small target detection in infrared image sequence caused by background clutter and target size variation with time, an approach for moving small target detection is proposed under a pipeline framework with an optimization strategy based on reinforcement learning. The pipeline framework is composed by pipeline establishment, target–background images separation, and target confirmation, in which the pipeline is established by designating several successive images with temporal sliding window, target–background images separation is dealt with low-rank and sparse matrix decomposition via robust principal component analysis, and target confirmation is achieved by employing a voting mechanism over more than one separated target images of the same input image. For unremitting optimization of target–background images separation, the weighting parameter of low-rank and sparse matrix decomposition is dynamically regulated by the way of reinforcement learning in consecutive detection, in which the complexity evaluation from sequential infrared images and results assessment of moving small target detection are integrated. The experiment results over four infrared small target image sequences with different cloudy sky backgrounds demonstrate the effectiveness and advantages of the proposed approach in both background clutter suppression and small target detection.

We propose a nonconvex higher-order total variation (TV) method for blind motion image deblurring. First, we introduce a nonconvex higher-order TV differential operator to define a new model of the blind motion image deblurring, which can effectively eliminate the staircase effect of the deblurred image; meanwhile, we employ an image sparse prior to improve the edge recovery quality. Second, to improve the accuracy of the estimated motion blur kernel, we use L₁ norm and H¹ norm as the blur kernel regularization term, considering the sparsity and smoothing of the motion blur kernel. Third, because it is difficult to solve the numerically computational complexity problem of the proposed model owing to the intrinsic nonconvexity, we propose a binary iterative strategy, which incorporates a reweighted minimization approximating scheme in the outer iteration, and a split Bregman algorithm in the inner iteration. And we also discuss the convergence of the proposed binary iterative strategy. Last, we conduct extensive experiments on both synthetic and real-world degraded images. The results demonstrate that the proposed method outperforms the previous representative methods in both quality of visual perception and quantitative measurement.

Automatic license plate recognition (ALPR) has been the focus of many researches in the past years. In general, ALPR is divided into the following problems: detection of on-track vehicles, license plate detection, segmentation of license plate characters, and optical character recognition (OCR). Even though commercial solutions are available for controlled acquisition conditions, e.g., the entrance of a parking lot, ALPR is still an open problem when dealing with data acquired from uncontrolled environments, such as roads and highways when relying only on imaging sensors. Due to the multiple orientations and scales of the license plates captured by the camera, a very challenging task of the ALPR is the license plate character segmentation (LPCS) step, because its effectiveness is required to be (near) optimal to achieve a high recognition rate by the OCR. To tackle the LPCS problem, this work proposes a benchmark composed of a dataset designed to focus specifically on the character segmentation step of the ALPR within an evaluation protocol. Furthermore, we propose the Jaccard-centroid coefficient, an evaluation measure more suitable than the Jaccard coefficient regarding the location of the bounding box within the ground-truth annotation. The dataset is composed of 2000 Brazilian license plates consisting of 14000 alphanumeric symbols and their corresponding bounding box annotations. We also present a straightforward approach to perform LPCS efficiently. Finally, we provide an experimental evaluation for the dataset based on five LPCS approaches and demonstrate the importance of character segmentation for achieving an accurate OCR.

We propose an optimization scheme that achieves fast yet accurate computation of superpixels from an image. Our optimization is designed to improve the efficiency and robustness for the minimization of a composite energy functional in the expectation–minimization (EM) framework where we restrict the update of an estimate to avoid redundant computations. We consider a superpixel energy formulation that consists of L²-norm for the spatial regularity and L¹-norm for the data fidelity in the demonstration of the robustness of the proposed algorithm. The quantitative and qualitative evaluations indicate that our superpixel algorithm outperforms SLIC and SEEDS algorithms. It is also demonstrated that our algorithm guarantees the convergence with less computational cost by up to 89% on average compared to the SLIC algorithm while preserving the accuracy. Our optimization scheme can be easily extended to other applications in which the alternating minimization is applicable in the EM framework.

Proximal K-singular value decomposition (PK-SVD) is a dictionary updating algorithm that incorporates proximal point method into K-SVD. The attempt of combining proximal method and K-SVD has achieved promising result in such areas as sparse approximation, image denoising, and image compression. However, the optimization procedure of PK-SVD is complicated and, therefore, limits the algorithm in both theoretical analysis and practical use. This article proposes a simple but effective optimization approach to the formulation of PK-SVD. We cast this formulation as a fitting problem and relax the constraint on the direction of the k’th row in the sparse coefficient matrix. This relaxation strengthens the regularization effect of the proximal point. The proposed algorithm needs fewer steps to implement and further boost the performance of PK-SVD while maintaining the same computational complexity. Experimental results demonstrate that the proposed algorithm outperforms conventional algorithms in reconstruction error, recovery rate, and convergence speed for sparse approximation and achieves better results in image denoising.

The engine flame is an important representation of the combustion process in the cylinder, and the three-dimensional (3-D) shape reconstruction of the flame can provide more information for the quantitative analysis of the flame, so as to contribute to further research on the mechanism of the combustion flame. One important method of 3-D shape reconstruction is to reconstruct the two-dimensional (2-D) projection image of the flame, so the optimization problem of the flame 2-D slice reconstruction algorithm is studied in this paper. According to the gradient sparsity characteristics in the total variation (TV) domain and radial diffusion characteristics of the engine combustion flame, a flame 2-D slice algebraic reconstruction technique (ART) reconstruction algorithm based on radial TV (ART-R-TV) is proposed. Numerical simulation results show that the new proposed ART-R-TV algorithm can reconstruct flame slice images more stably and have a better robustness than the two traditional ART algorithms especially in a limited-angle situation.

This paper proposes an improved exemplar-based inpainting using image segmentation, image quilting, and spatial blending to reduce the excessively propagating texture and pathological geometric configurations. A graph-based segmentation is used to generate an initial segmented map. The segmented regions are merged by a K-means approach with texture similarity to generate a further texture-based segmented map, which reduces the candidate source space and determines whether to utilize image quilting in the iterative updating process. An improved spatial blending is introduced as a postprocess to smooth the filled image when the filling process is completed. Experimental results show that this exemplar-based inpainting approach has a good visual effect and reduces the computing time.

Attention to tampering by median filtering (MF) has recently increased in digital image forensics. For the MF detection (MFD), this paper presents a feature vector that is extracted from two kinds of variations between the neighboring line pairs: the row and column directions. Of these variations in the proposed method, one is defined by a gradient difference of the intensity values between the neighboring line pairs, and the other is defined by a coefficient difference of the Fourier transform (FT) between the neighboring line pairs. Subsequently, the constructed 19-dimensional feature vector is composed of these two parts. One is the extracted 9-dimensional from the space domain of an image and the other is the 10-dimensional from the frequency domain of an image. The feature vector is trained in a support vector machine classifier for MFD in the altered images. As a result, in the measured performances of the experimental items, the area under the receiver operating characteristic curve (AUC, ROC) by the sensitivity (P_TP: the true positive rate) and 1-specificity (P_FP: the false-positive rate) are above 0.985 and the classification ratios are also above 0.979. P_e (a minimal average decision error) ranges from 0 to 0.024, and P_TP at P_FP=0.01 ranges from 0.965 to 0.996. It is confirmed that the grade evaluation of the proposed variation-based MF detection method is rated as “Excellent (A)” by AUC is above 0.9.