We propose a method for improving adversarial robustness by addition of a new bounded function just before softmax. Recent studies hypothesize that small logits (inputs of softmax) by logit regularization can improve adversarial robustness of deep learning. Following this hypothesis, we analyze norms of logit vectors at the optimal point under the assumption of universal approximation and explore new methods for constraining logits by addition of a bounded function before softmax. We theoretically and empirically reveal that small logits by addition of a common activation function, e.g., hyperbolic tangent, do not improve adversarial robustness since input vectors of the function (pre-logit vectors) can have large norms. From the theoretical findings, we develop the new bounded function. The addition of our function improves adversarial robustness because it makes logit and pre-logit vectors have small norms. Since our method only adds one activation function before softmax, it is easy to combine our method with adversarial training. Our experiments demonstrate that our method is comparable to logit regularization methods in terms of accuracies on adversarially perturbed datasets without adversarial training. Furthermore, it is superior or comparable to logit regularization methods and a recent defense method (TRADES) when using adversarial training.


    Zugriff

    Download


    Exportieren, teilen und zitieren



    Titel :

    Constraining Logits by Bounded Function for Adversarial Robustness


    Beteiligte:

    Erscheinungsdatum :

    2020


    Anmerkungen:

    19 pages, 16 figures


    Medientyp :

    Preprint


    Format :

    Elektronische Ressource


    Sprache :

    Englisch




    SWAP: Exploiting Second-Ranked Logits for Adversarial Attacks on Time Series

    Dong, Chang George / Zheng, Liangwei Nathan / Chen, Weitong et al. | IEEE | 2023


    Principal components analysis of the logits of the survivorship function

    Mitra, S. / Levin, M. L. / International Union for the Scientific Study of Population | British Library Conference Proceedings | 1993


    Knowledge Distillation with Refined Logits

    Sun, Wujie / Chen, Defang / Lyu, Siwei et al. | ArXiv | 2024

    Freier Zugriff

    NormKD: Normalized Logits for Knowledge Distillation

    Chi, Zhihao / Zheng, Tu / Li, Hengjia et al. | ArXiv | 2023

    Freier Zugriff

    Logits DeConfusion with CLIP for Few-Shot Learning

    Li, Shuo / Liu, Fang / Hao, Zehua et al. | ArXiv | 2025

    Freier Zugriff