On intelligent vehicles, Deep Neural Networks (DNNs) may run on devices whose computational load varies over time. Within the context of variable network architectures, that can be used to constrain the inference latency for real-time deployment with varying system resources, we introduce LeAF (Legacy Augmentation for Flexible inference), a novel paradigm to augment a pre-trained DNN with trainable, shallow execution paths that can run in place of the legacy ones. While preserving the legacy DNN weights, LeAF allows changing the DNN architecture with minimal overhead to effectively adapt to different system performance targets. LeAF-ResNet-50 has less than 14% storage overhead over the legacy DNN; its accuracy varies from the legacy 76.1% to 70.15% (up to 5% better than Slimmable [1] with a latency that is 37% better than OFA [2] on an A100 GPU with batch size 256). Our analysis shows the importance of considering not only the target device, but also the batch size and the temporal dynamic of the DNN configuration to optimize the performances of variable architecture DNNs, LeAF in particular.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Augmenting Legacy Networks for Flexible Inference


    Contributors:


    Publication date :

    2023-06-04


    Size :

    1325805 byte





    Type of media :

    Conference paper


    Type of material :

    Electronic Resource


    Language :

    English





    Augmenting Adaptive Approach to Control of Flexible Systems

    Anthony Calise / Bong-Jun Yang / James Craig | AIAA | 2004


    Shape memory alloys for augmenting damping of flexible structures

    Thomson, P. / Balas, G. / Nalbantoglu, V. | AIAA | 1996


    An Augmenting Adaptive Approach to Control of Flexible Systems

    Calise, Anthony / Yang, Bong-Jun / Craig, James | AIAA | 2002