Image-guided depth propagation for 2-D-to-3-D video conversion using superpixel matching and adaptive autoregressive model

Jiji Cai; Cheolkon Jung

doi:10.1117/1.JEI.26.5.053019

10 October 2017 Image-guided depth propagation for 2-D-to-3-D video conversion using superpixel matching and adaptive autoregressive model

Jiji Cai, Cheolkon Jung

Author Affiliations +

Journal of Electronic Imaging, Vol. 26, Issue 5, 053019 (October 2017). https://doi.org/10.1117/1.JEI.26.5.053019

Abstract

We propose image-guided depth propagation for two-dimensional (2-D)-to-three-dimensional (3-D) video conversion using superpixel matching and the adaptive autoregressive (AR) model. We adopt key frame-based depth propagation that propagates the depth map in the key frame to nonkey frames. Moreover, we use the adaptive AR model for depth refinement to penalize depth-color inconsistency. First, we perform superpixel matching to estimate motion vectors at the superpixel level instead of block matching based on the fixed block size. Then, we conduct depth compensation based on motion vectors to generate the depth map in the nonkey frame. However, the size of two superpixels is not exactly the same due to the segment-based matching, which causes matching errors in the compensated depth map. Thus, we introduce an adaptive image-guided AR model to minimize matching errors and produce the final depth map by minimizing AR prediction errors. Finally, we employ depth-image-based rendering to generate stereoscopic views from 2-D videos and their depth maps. Experimental results demonstrate that the proposed method successfully performs depth propagation and produces high-quality depth maps for 2-D-to-3-D video conversion.

Citation Download Citation

Jiji Cai and Cheolkon Jung "Image-guided depth propagation for 2-D-to-3-D video conversion using superpixel matching and adaptive autoregressive model," Journal of Electronic Imaging 26(5), 053019 (10 October 2017). https://doi.org/10.1117/1.JEI.26.5.053019

Received: 24 May 2017; Accepted: 14 September 2017; Published: 10 October 2017

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available