This paper develops a neural network optimization algorithm: the rectified L2 regularization, which can be used to train ternary neural networks with weights of all layers constrained to −1, 0 and +1. It will analyze how to set the learning rate and penalty coefficient during the training phase. Compared with previous approaches, the rectified L2 regularization algorithm can be directly implemented on the open source machine learning framework, such as TensorFlow and PyTorch. The accuracy of the MNIST and Fashion-MNIST test datasets is 99.40% and 92.21%, respectively, which is close to the state-of-the-art accuracy of full precision neural networks with the model compression rate guaranteed.


    Zugriff

    Zugriff prüfen

    Verfügbarkeit in meiner Bibliothek prüfen

    Bestellung bei Subito €


    Exportieren, teilen und zitieren



    Titel :

    Training Ternary Neural Networks By Rectified L2 Regularization


    Beteiligte:
    Han, Qiankun (Autor:in) / Fan, Yuanning (Autor:in) / Ge, Jiexian (Autor:in) / Cui, Xiaoxin (Autor:in) / Yu, Dunshan (Autor:in)


    Erscheinungsdatum :

    01.10.2019


    Format / Umfang :

    92864 byte




    Medientyp :

    Aufsatz (Konferenz)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch



    Rectified Surface Mosaics

    Carroll, R. E. / Seitz, S. M. | British Library Online Contents | 2009


    Rectified lunar atlas

    Hartmann, W. K. / Kuiper, G. P. / Spradley, L. H. et al. | NTRS | 1963




    Ternary Synaptic Weights Algorithm: Neural Network Training With Don't Care Attributes

    Ulgen, F. / Akamatsu, N. / IEEE; Hong Kong Chapter of Signal Processing | British Library Conference Proceedings | 1994