A stochastic three-block splitting algorithm and its application to quantized deep neural networks

arXiv2204.11065MaRDI QIDQ6397270

Fengmiao Bian, Xiaoqun Zhang, Author name not available (Why is that?)

Publication date: 23 April 2022

Abstract: Deep neural networks (DNNs) have made great progress in various fields. In particular, the quantized neural network is a promising technique making DNNs compatible on resource-limited devices for memory and computation saving. In this paper, we mainly consider a non-convex minimization model with three blocks to train quantized DNNs and propose a new stochastic three-block alternating minimization (STAM) algorithm to solve it. We develop a convergence theory for the STAM algorithm and obtain an $epsilon$-stationary point with optimal convergence rate $mathcalO(epsilo{n}^{-4})$. Furthermore, we apply our STAM algorithm to train DNNs with relaxed binary weights. The experiments are carried out on three different network structures, namely VGG-11, VGG-16 and ResNet-18. These DNNs are trained using two different data sets, CIFAR-10 and CIFAR-100, respectively. We compare our STAM algorithm with some classical efficient algorithms for training quantized neural networks. The test accuracy indicates the effectiveness of STAM algorithm for training relaxed binary quantization DNNs.

This page was built for publication: A stochastic three-block splitting algorithm and its application to quantized deep neural networks

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6397270)