Exploiting Quantization and Mapping Synergy in Hardware-Aware Deep Neural Network Accelerators

• https://arxiv.org/abs/2404.05368

Quantization, Neural networks, Hardware accelerator

Energy efficiency and memory footprint of a convolutional neural network (CNN)
implemented on a CNN inference accelerator depend on many factors, including
a weight quantization strategy (i.e., data types and bit-widths) and mapping
(i.e., placement and scheduling of DNN elementary operations on hardware units of
the accelerator). We show that enabling rich mixed quantization schemes during
the implementation can open a previously hidden space of mappings that utilize
the hardware resources more effectively. CNNs utilizing quantized weights and
activations and suitable mappings can significantly improve trade-offs among the
accuracy, energy, and memory requirements compared to less carefully optimized
CNN implementations. To find, analyze, and exploit these mappings, we: (i) extend
a general-purpose state-of-the-art mapping tool (Timeloop) to support mixed
quantization, which is not currently available; (ii) propose an efficient
multi-objective optimization algorithm to find the most suitable bit-widths and
mapping for each DNN layer executed on the accelerator; and (iii) conduct
a detailed experimental evaluation to validate the proposed method. On two CNNs
(MobileNetV1 and MobileNetV2) and two accelerators (Eyeriss and Simba) we show
that for a given quality metric (such as the accuracy on ImageNet), energy
savings are up to 37% without any accuracy drop.