Low-bit Neural Network Computing: Algorithms and Hardware
Date:
In recent years, deep learning technologies achieved excellent performance and many breakthroughs in both academia and industry. However, the state-of-the-art deep models are computationally expensive and consume large storage space. Deep learning is also strongly demanded by numerous applications from areas such as mobile platforms, wearable devices, autonomous robots, and IoT devices. How to efficiently apply deep models on such low power devices becomes a challenging research problem. In recent years, Low-bit Neural Network (NN) Computing has received much attention, due to its potential to reduce the storage and computation complexity of NN inference and training. This tutorial introduces the existing efforts on low-bit NN computing. This tutorial includes three parts: 1) Algorithms towards more accurate low-bit NNs; 2) Binary neural network design and inference; 3) Low-bit training of NNs.