Knowledge distillation with attention mechanism for anomaly detection

Mohan Li; Xiang Lyu; Xuan Guo

doi:10.1117/12.3012320

21 December 2023 Knowledge distillation with attention mechanism for anomaly detection

Mohan Li, Xiang Lyu, Xuan Guo

Proceedings Volume 12970, Fourth International Conference on Signal Processing and Computer Science (SPCS 2023); 1297021 (2023) https://doi.org/10.1117/12.3012320
Event: Fourth International Conference on Signal Processing and Computer Science (SPCS 2023), 2023, Guilin, China

Abstract

In the industrial domain, accurate detection and localization of abnormal images are crucial factors for ensuring production efficiency and quality. In recent years, methods in the field of unsupervised anomaly detection have predominantly centered around knowledge distillation models with a teacher-student structure. Although the application of knowledge distillation has significantly improved the precise identification of abnormal regions compared to traditional methods, the currently best-performing knowledge distillation models exhibit identical architectures for both the teacher and student networks. This limitation hampers the effective lightweighting of the student network and poses challenges in distinguishing between background and target objects in certain data with backgrounds, thereby affecting accuracy and wasting computational resources. To address this issue, we propose an innovative approach that integrates knowledge distillation with attention mechanisms for identifying abnormal regions in industrial anomaly images. Our method comprises two key features: firstly, transferring knowledge from a pretrained teacher network to a student network to enhance performance; secondly, incorporating attention mechanisms to direct the model’s focus towards potential abnormal regions, thereby enhancing detection accuracy. Our approach innovatively combines knowledge distillation with attention mechanisms, offering a novel solution for industrial anomaly image recognition. Through experimental validation, our method demonstrates superior performance compared to the original models, providing fresh insights for the field of industrial anomaly image analysis.

(2023) Published by SPIE. Downloading of the abstract is permitted for personal use only.

Citation Download Citation

Mohan Li, Xiang Lyu, and Xuan Guo "Knowledge distillation with attention mechanism for anomaly detection", Proc. SPIE 12970, Fourth International Conference on Signal Processing and Computer Science (SPCS 2023), 1297021 (21 December 2023); https://doi.org/10.1117/12.3012320

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
8 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Education and training

Performance modeling

Visual process modeling

Computer vision technology

Data modeling

Industrial applications

Convolutional neural networks

Show All Keywords

Keywords/Phrases

Search In:

Publication Years