Successfully and efficiently training deep multi-layer perceptrons with logistic activation function simply requires initializing the weights with an appropriate negative mean

DOI10.1016/J.NEUNET.2022.05.030zbMath1530.68234OpenAlexW4282025272WikidataQ113868229 ScholiaQ113868229MaRDI QIDQ6077039

Ahmet Yilmaz, Riccardo Poli

Publication date: 17 October 2023

Published in: Neural Networks (Search for Journal in Brave)

Full work available at URL: http://repository.essex.ac.uk/32958/1/manuscript.pdf

zbMATH Keywords

supervised learningdeep neural networksweights initializationvanishing gradientlogistic activation function

Mathematics Subject Classification ID

Artificial neural networks and deep learning (68T07)

Cites Work

• Unnamed Item
• Multilayer feedforward networks are universal approximators
• Reducing the Dimensionality of Data with Neural Networks
• The Vanishing Gradient Problem During Learning Recurrent Neural Nets and Problem Solutions
• Learning representations by back-propagating errors
• A Fast Learning Algorithm for Deep Belief Nets
• Approximation by superpositions of a sigmoidal function

This page was built for publication: Successfully and efficiently training deep multi-layer perceptrons with logistic activation function simply requires initializing the weights with an appropriate negative mean