The BeMi Stardust: a Structured Ensemble of Binarized Neural Networks

arXiv2212.03659MaRDI QIDQ6419844

Author name not available (Why is that?)

Publication date: 7 December 2022

Abstract: Binarized Neural Networks (BNNs) are receiving increasing attention due to their lightweight architecture and ability to run on low-power devices. The state-of-the-art for training classification BNNs restricted to few-shot learning is based on a Mixed Integer Programming (MIP) approach. This paper proposes the BeMi ensemble, a structured architecture of BNNs based on training a single BNN for each possible pair of classes and applying a majority voting scheme to predict the final output. The training of a single BNN discriminating between two classes is achieved by a MIP model that optimizes a lexicographic multi-objective function according to robustness and simplicity principles. This approach results in training networks whose output is not affected by small perturbations on the input and whose number of active weights is as small as possible, while good accuracy is preserved. We computationally validate our model using the MNIST and Fashion-MNIST datasets using up to 40 training images per class. Our structured ensemble outperforms both BNNs trained by stochastic gradient descent and state-of-the-art MIP-based approaches. While the previous approaches achieve an average accuracy of 51.1% on the MNIST dataset, the BeMi ensemble achieves an average accuracy of 61.7% when trained with 10 images per class and 76.4% when trained with 40 images per class.

Has companion code repository: https://github.com/informsjoc/2023.0281

This page was built for publication: The BeMi Stardust: a Structured Ensemble of Binarized Neural Networks

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6419844)