Ranked dropout for handwritten digit recognition

Yue Tang; Zhuonan Liang; Huaze Shi; Peng Fu; Quansen Sun

doi:10.1117/12.2589394

27 January 2021 Ranked dropout for handwritten digit recognition

Yue Tang, Zhuonan Liang, Huaze Shi, Peng Fu, Quansen Sun

Proceedings Volume 11720, Twelfth International Conference on Graphics and Image Processing (ICGIP 2020); 117201Q (2021) https://doi.org/10.1117/12.2589394
Event: Twelfth International Conference on Graphics and Image Processing, 2020, Xi'an, China

Abstract

Overfitting is a common problem in training of neural network with small training sets, which leads to worse performance on the new samples. Dropout has been proved to be an effective method to avoid overfitting, which prevents co-adaptation of features detectors by randomly discarding nodes from hidden layers of network. Inspired by dropout, we proposed a ranked dropout method to remove randomness of standard dropout mask, which discards a part of active nodes and forces the inactive nodes to learn more features to improve generalization ability. We apply the proposed ranked dropout to a stacked autoencoder network and compare it with standard dropout, gaussian dropout, uniform dropout and DropConnect on MNIST dataset. Experimental results of handwritten digit recognition demonstrate that the ranked strategy leads to better classification performance and the proposed ranked dropout can effectively reduce interference of overfitting and improve model‟s generalization ability.

Citation Download Citation

Yue Tang, Zhuonan Liang, Huaze Shi, Peng Fu, and Quansen Sun "Ranked dropout for handwritten digit recognition", Proc. SPIE 11720, Twelfth International Conference on Graphics and Image Processing (ICGIP 2020), 117201Q (27 January 2021); https://doi.org/10.1117/12.2589394

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available