Panoptic segmentation method based on pixel-level instance perception

Yuhao Wu; Jun Sun

doi:10.1117/12.2640223

1 August 2022 Panoptic segmentation method based on pixel-level instance perception

Yuhao Wu, Jun Sun

Proceedings Volume 12257, 4th International Conference on Information Science, Electrical, and Automation Engineering (ISEAE 2022); 122571A (2022) https://doi.org/10.1117/12.2640223
Event: 4th International Conference on Information Science, Electrical, and Automation Engineering (ISEAE 2022), 2022, Guangzhou, China

Abstract

Deep learning-based panoptic segmentation technology has enabled in-depth understanding of images as well as scene descriptions and provides solutions for most computer vision tasks. The post-processing fusion of instance branches and semantic branches in the top-down approach often causes conflicts between branches. In order to solve this problem, we propose a single-stage panoptic segmentation network based on pixel-level instance perception with FCOS object detector. Specifically, the FCOS output extraction network is extended by using three branches, namely the object detection branch, the semantic branch, and the panoptic branch. The object detection branch uses the FCOS object detector to provide the instance center for the panoptic branch to generate a pixel-level instance perception mask, and the semantic branch provides semantical information for the panoptic branch. The panoptic segmentation output comes from the pixel-wise product of the semantic branch and the panoptic branch, avoiding the heuristic post-processing fusion in end-to-end learning. A good trade-off between accuracy and inference time was achieved on standard datasets Cityscapes, with PQ at 59.4%, mIoU at 76.2% and running time 94ms on Cityscapes validation dataset. Compared to other methods, our method achieves better panoptic performance within a reasonable runtime range. At the same time, the inference process is concise and can be optimized by using the deep learning inference engine to reduce the difficulty of deployment in actual application scenarios.

Citation Download Citation

Yuhao Wu and Jun Sun "Panoptic segmentation method based on pixel-level instance perception", Proc. SPIE 12257, 4th International Conference on Information Science, Electrical, and Automation Engineering (ISEAE 2022), 122571A (1 August 2022); https://doi.org/10.1117/12.2640223

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
8 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Image segmentation

Network architectures

RGB color model

Convolution

Detection and tracking algorithms

Panoramic photography

Target detection

Show All Keywords

Keywords/Phrases

Search In:

Publication Years