Weight Matrix Dimensionality Reduction in Deep Learning via Kronecker Multi-layer Architectures
From MaRDI portal
Publication:6396061
arXiv2204.04273MaRDI QIDQ6396061
Author name not available (Why is that?)
Publication date: 8 April 2022
Abstract: Deep learning using neural networks is an effective technique for generating models of complex data. However, training such models can be expensive when networks have large model capacity resulting from a large number of layers and nodes. For training in such a computationally prohibitive regime, dimensionality reduction techniques ease the computational burden, and allow implementations of more robust networks. We propose a novel type of such dimensionality reduction via a new deep learning architecture based on fast matrix multiplication of a Kronecker product decomposition; in particular our network construction can be viewed as a Kronecker product-induced sparsification of an "extended" fully connected network. Analysis and practical examples show that this architecture allows a neural network to be trained and implemented with a significant reduction in computational time and resources, while achieving a similar error level compared to a traditional feedforward neural network.
Has companion code repository: https://github.com/jaromhogue/kdlayers
No records found.
This page was built for publication: Weight Matrix Dimensionality Reduction in Deep Learning via Kronecker Multi-layer Architectures
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6396061)
