Video segmentation and occlusion detection over multiple frames

Janusz Konrad; Mirko Ristivojevic

doi:10.1117/12.476652

7 May 2003 Video segmentation and occlusion detection over multiple frames

Janusz Konrad, Mirko Ristivojevic

Proceedings Volume 5022, Image and Video Communications and Processing 2003; (2003) https://doi.org/10.1117/12.476652
Event: Electronic Imaging 2003, 2003, Santa Clara, CA, United States

Abstract

Spatial segmentation of image sequences is usually performed based on motion between two frames, and then followed by tracking. Some recent approaches extend this to joint segmentation in space-time; the resulting 3-D segmentation (in x-y-t space) can be interpreted as a volume 'carved out by a moving object in the image sequence domain. We call such volumes 'object tunnels'. In this paper, we propose a new approach to occlusion analysis and characterization that is based on object tunnels. It results from the observation that object-tunnel wall for a fully visible object has different shape than that for an object undergoing occlusion or exposure. Walls of tunnels associated with moving objects have tangent planes that are, in general, non-parallel to the time axis. When an object gets occluded or exposed by a static feature, part of the object tunnel wall stops evolving freely; its spatial coordinates remain fixed (static occlusion boundary) while the temporal coordinate increases linearly (time evolution). This forces part of the wall to be comprised of lines parallel to the time axis, each line defined by a single point on the occlusion boundary. In case this boundary is a straight-line edge, the occluding part of the wall becomes planar. We propose to detect occlusions by searching for such characteristic surfaces of object tunnel walls. We formulate the problem for planar occlusion walls based on a robust distance metric, and we show experimental results for various occlusion types on synthetic and camera-acquired image sequences.

Citation Download Citation

Janusz Konrad and Mirko Ristivojevic "Video segmentation and occlusion detection over multiple frames", Proc. SPIE 5022, Image and Video Communications and Processing 2003, (7 May 2003); https://doi.org/10.1117/12.476652

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available