Convolutional neural network weights regularization via orthogonalization

Alexander V. Gayer; Alexander V. Sheshkus

doi:10.1117/12.2559346

31 January 2020 Convolutional neural network weights regularization via orthogonalization

Alexander V. Gayer, Alexander V. Sheshkus

Proceedings Volume 11433, Twelfth International Conference on Machine Vision (ICMV 2019); 1143326 (2020) https://doi.org/10.1117/12.2559346
Event: Twelfth International Conference on Machine Vision, 2019, Amsterdam, Netherlands

Abstract

Regularization methods play an important role in artificial neural networks training, improving generalization performance and preventing them from overfitting. In this paper, we introduce a new regularization method, based on the orthogonalization of convolutional layer filters. Proposed method is easy to implement and it has plug-and-play compatibility with modern training approaches, without any changes or adaptations on their part. Experiments with MNIST and CIFAR10 datasets showed that the effectiveness of the suggested method depends on number of filters in the layer, and maximum increase in quality is achieved for architectures with small number of parameters, which is important for training fast and lightweight neural networks.

Citation Download Citation

Alexander V. Gayer and Alexander V. Sheshkus "Convolutional neural network weights regularization via orthogonalization", Proc. SPIE 11433, Twelfth International Conference on Machine Vision (ICMV 2019), 1143326 (31 January 2020); https://doi.org/10.1117/12.2559346

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available