This paper develops a neural network optimization algorithm: the rectified L2 regularization, which can be used to train ternary neural networks with weights of all layers constrained to −1, 0 and +1. It will analyze how to set the learning rate and penalty coefficient during the training phase. Compared with previous approaches, the rectified L2 regularization algorithm can be directly implemented on the open source machine learning framework, such as TensorFlow and PyTorch. The accuracy of the MNIST and Fashion-MNIST test datasets is 99.40% and 92.21%, respectively, which is close to the state-of-the-art accuracy of full precision neural networks with the model compression rate guaranteed.
Training Ternary Neural Networks By Rectified L2 Regularization
01.10.2019
92864 byte
Aufsatz (Konferenz)
Elektronische Ressource
Englisch
British Library Online Contents | 2009
|NTRS | 1963
|Europäisches Patentamt | 2022
|Ternary Synaptic Weights Algorithm: Neural Network Training With Don't Care Attributes
British Library Conference Proceedings | 1994
|