On the storage capacity of Hopfield models with correlated patterns
type: Item
instance of: scholarly article
title: On the storage capacity of Hopfield models with correlated patterns
zbMATH Open document ID: 0941.60090
published in: The Annals of Applied Probability
publication date: 1999-11-29T00:00:00Z
review text: It is well known that the Hopfield model of a fully connected neural network with \(N\) neurons with the original Hebbian learning rule can store a number of patters equal to \(C N/\ln N\) without error, and equal to \(\alpha_c N\) (with \(\alpha_c\approx 0.138\) according to numerical and replica results) if a small fraction of errors is tolerated. These results hold with probability tending to one rapidly if the patterns are assumed to be i.i.d. random variables with each component a symmetric i.i.d. Rademacher random variable. The present paper presents a rigorous investigation or the question what becomes of these results if the patterns are correlated, either ``semantically'' or ``spatially''. More specifically, it is supposed that either for any \(i\), \(\xi^\mu_i\), \(\mu\in {\mathbb N}\), form a Markov chain with state space \(\{-1,+1\}\), which are in turn independent for different \(i\), or vice versa that for each \(\mu\), \(\xi^\mu_i,i\in {\mathbb N}\), is an family of independent Markov chains. The results obtained are the following: 1) If perfect storage is required, the storage capacity decreases linearly the degree of correlation increases in the semantic case, while the opposite is true in the case of spatial correlation. 2) If storage with error is tolerated, the bounds obtained for \(a_c\) decrease with the correlation in both cases.
Mathematics Subject Classification ID: 60K35
Mathematics Subject Classification ID: 82C32
zbMATH DE Number: 1319910
zbMATH Keywords: Hopfield model
zbMATH Keywords: storage capacity
zbMATH Keywords: correlated patterns
zbMATH Keywords: neural networks
author: M. Löwe
reviewed by: Anton Bovier
MaRDI profile type: Publication
full work available at URL: https://doi.org/10.1214/aoap/1028903378
OpenAlex ID: W1965496542
cites work: Rigorous bounds on the storage capacity of the dilute Hopfield model
cites work: An almost sure large deviation principle for the Hopfield model
cites work: The retrieval phase of the Hopfield model: A rigorous analysis of the overlap distribution
cites work: Gibbs states of the Hopfield model in the regime of perfect memory
cites work: Large deviation principles for the Hopfield model and the Kac-Hopfield model
cites work: Gibbs states of the Hopfield model with extensively many patterns.
cites work: Neural Network Design for Efficient Information Retrieval
cites work: A central limit theorem for the overlap in the Hopfield model
cites work: Gibbs measures and phase transitions
cites work: Neural networks and physical systems with emergent collective computational abilities.
cites work: Q4291240
cites work: Lower bounds on the restitution error in the Hopfield model
cites work: The capacity of the Hopfield associative memory
cites work: Properties of neural networks storing spatially correlated patterns
cites work: Q4372475
cites work: Q4863579
cites work: Rigorous results for the Hopfield model with many patterns
cites work: Storage of sets of correlated data in neural network memories
cites work: Q4027092
DOI: 10.1214/AOAP/1028903378
description: scientific article; zbMATH DE number 1319910