In this paper we present two novel techniques developed in the context of the stereo to multi-view conversion research at
Philips in support of the introduction of stereoscopic and auto-stereoscopic. First, we show that we can use a relatively
simple filtering approach, based on the recently popular bilateral filters, to address the correspondence problem, which is
at the heart of depth and motion estimation. The proposed recursive filter uses Gaussian kernels to filter best matches
and to incorporate image-based constraints. It iteratively refines the depth values starting from a random initialization
and converges in a limited number of iterations to a time-stable high-quality depth map. The second contribution of the
paper is an occlusion detection method that uses robust filtering for the detection of occlusion that is primarily based on
the analysis of the variation of the matching metric used in the disparity estimation process. The basic underlying ideas
behind the occlusion detection method are (1) that occluded areas are highly likely to be located near image boundaries
(where luminance or color changes abruptly), and (2) occluded regions are characterized by a large decrease in the
quality of the matching metric across these boundaries. The two algorithms were tested on real-world stereoscopic video
content showing promising results.
This research focuses on the conversion of stereoscopic video material into an image + depth format which is suitable for
rendering on the multiview auto-stereoscopic displays of Philips. The recent interest shown in the movie industry for 3D
significantly increased the availability of stereo material. In this context the conversion from stereo to the input formats
of 3D displays becomes an important task. In this paper we present a stereo algorithm that uses multiple footprints
generating several depth candidates for each image pixel. We characterize the various matching windows and we devise
a robust strategy for extracting high quality estimates from the resulting depth candidates. The proposed algorithm is
based on a surface filtering method that employs simultaneously the available depth estimates in a small local
neighborhood while ensuring correct depth discontinuities by the inclusion of image constraints. The resulting highquality
image-aligned depth maps proved an excellent match with our 3D displays.
KEYWORDS: 3D modeling, Image registration, 3D image processing, Systems modeling, Data modeling, Unmanned vehicles, 3D acquisition, Imaging systems, Sensors, Image sensors
The focus of this paper is on the reconstruction of 3D representations of real world scenes and objects using multiple sensors with, as one of its main applications, the enhancement of the autonomy and mobility of unmanned vehicles. The sensors considered are primarily range acquisition devices (such as laser scanners and stereo systems) that allow the recovery of 3D geometry. One of the most important technical challenges that we are addressing is the registration task in both its multi-modal and single modality aspects. Our work is based on a unified approach that formulates the correspondence problem as an optimization task. In this context we developed a criterion that can be used for 3D free-form shape registration. The new criterion is derived from simple Boolean matching principles by approximation and relaxation. Technically, one of the main advantages of the proposed approach is convexity in the neighborhood of the alignment parameters and continuous differentiability, allowing for the use of standard gradient-based optimization techniques. The proposed algorithm allows also for a significant automation of the scene modeling task by reducing the intervention of human operators in the tedious image registration task. Furthermore, we show that the criterion can be computed in linear time complexity which permits the fast implementation critical in many applications of autonomous mobile platforms.
In this paper we present a new method for the registration of multiple sensors applied to a mobile robotic inspection platform. Our main technical challenge is automating the integration process for various multimodal inputs, such as depth maps, and multi-spectral images. This task is approached through a unified framework based on a new registration criterion that can be employed for both 3D and 2D datasets. The system embedding this technology reconstructs 3D models of scenes and objects that are inspected by an autonomous platform in high security areas. The models are processed and rendered with corresponding multi-spectral textures, which greatly enhances both human and machine identification of threat objects.
KEYWORDS: 3D modeling, Data modeling, Motion models, Video, 3D image processing, Cameras, Systems modeling, Laser scanners, 3D video streaming, Atomic force microscopy
In this paper we describe a new method for the modeling of objects with know generic shape such as human faces from video and range data. The method combines the strengths of active laser scanning and passive Shape from Motion techniques. Our approach consists of first reconstructing a few feature-points that can be reliably tracked throughout a video sequence of the object. These features are mapped to corresponding 3D points in a generic 3D model reconstructed from dense and accurate range data acquired only once. The resulting 3D-3D set of matches is used to warp the generic model into the actual object visible in the video stream using thin-plate splines interpolation. Our method avoids the problems of dense matching encountered in stereo algorithms. Furthermore, in the case of face reconstruction, this method provides dense models while not requiring the invasive laser scanning of faces.
KEYWORDS: Image registration, 3D modeling, Cameras, Sensors, Optimization (mathematics), 3D image processing, Image processing, Medical imaging, Reflectivity, Data modeling
In this paper, we present a method for automatically registering a 3D range image and a 2D color image using the (chi) 2-similarity metric. The goal of this registration is to allow the reconstruction of a scene using multi-sensor information. Traditional registration algorithms use invariant image features to drive the registration process. This approach limits the applicability to multi-modal data since features of interest may not appear in each modality. However, the (chi) 2-similarity metric is an intensity- based approach that has interesting multi-modal characteristics. We explore this metric as a mechanism to govern the registration search. Using range data from a Perceptron laser camera and color data form a Kodak digital camera, we present result using this automatic registration with the (chi) 2-similarity metric.
Towards photo-realistic 3D scene reconstruction form range and color images, we present a statistical technique for multi-modal image registration. Statistical tools are employed to measure the dependence of tow imags, considered as random distributions of pixels, and to find the pose of one imaging system relative to the other. The similarity metrics used in our automatic registration algorithm are based on the chi-squared measure of dependence, which is presented as an alternative to the standard mutual information criterion. These two criteria belong to the class of information-theoretic similarity measures that quantify the dependence in terms of information provided by one image about the other. This approach requires the use of a robust optimization scheme for the maximization of the similarity measure. To achieve accurate reslut, we investigated the use of heuristics such as genetic algorithms. The retrieved pose parameters are used to generate a texture map from the color image, and the occluded areas in this image are determined and labeled. Finally the 3D scene is rendered as a triangular mesh with texture.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.