A novel chaotic hetero-associative memoryby Z.H. Aghajari, M. Teshnehlab, M.R. Jahed Motlagh



Cognitive Neuroscience / Artificial Intelligence / Computer Science Applications


A novel chaotic hetero-associative memory

Z.H. Aghajari a,n, M. Teshnehlab a, M.R. Jahed Motlagh b a Khaje Nasir University of Technology, Electronic and Computer Engineering Faculty, Tehran, Iran b Iran University of Science and Technology, Computer Engineering Faculty, Tehran, Iran a r t i c l e i n f o

Article history:

Received 2 November 2013

Received in revised form 7 March 2015

Accepted 23 April 2015

Communicated by He Huang


Hetero-associative memory

Associative memory

Neural network

Chaos a b s t r a c t

In this study, a novel hetero-associative memory with dynamic behavior is proposed. The proposed hetero-associative memory can store as twice as a regular hetero-associative memory using a new extension of sparse learning method. The new learning method gives the network ability of successive learning, therefore it can store new patterns even after learning phase. In other words, learning step and recall step are not separated in this method. We also add chaos searching in recall step in order to make the network be able to converge into the best possible solution among whole search space. Chaotic behavior helps the network jumps from local minimums. Simulation result shows higher storage capacity and also better recall performance in comparison with regular hetero-associative memory with the presence of noisy input data. & 2015 Elsevier B.V. All rights reserved. 1. Introduction

One of the most interesting features of human brain is its ability in association of information. We associate human faces with names, we can also recognize people even if they get older.

This primary function of the brain is called Associative Memory.

This feature is useful in many different fields especially in data mining.

To implement associative memory some methods were introduced and the most interesting and efficient one is the Associative

Memory Networks [1]. Associative memory networks are single layer nets that can store and recall patterns based on data content rather than data address. Associative memory stores pattern associations and each association is a pair of input/output vector (s, t). If s vector and t vector are the same, then the associative memory is called Auto-Associative Memory and if they are different vectors, then it is called Hetero-Associative Memory.

Hetero-associative memories can store the association between two different types. For example, hetero-associative memories can store alphabet sounds that are related to their associated alphabet graphic patterns. Although associative memory can learn and store associated patterns successfully, its capacity is restricted by neuron size and the number of learning patterns.

To improve capacity storage of the associative memory some new learning methods are introduced which are more efficient rather than the conventional associative memory [1,2]. It says that studies have shown chaotic behavior of real neurons and it is considered that chaos plays an important role in information processing of human brain [3–5]. Consequently, chaos was noticed as a new solution to be used in associative memory. Several articles have been introduced based on chaotic theory which show improvement of associative memories performance especially in deal with noisy data.

There is also another kind of associative memories that use matrix operation instead of algebra called Lattice associative memory. Recently they have become more interesting as they can store more patterns than conventional hetero-associative [6].

Ritter et al. present morphological associative memories based on morphological neural networks which converge in one step, and also have unlimited storage capacity for perfect recall. It has been shown that morphological auto-associative memories can exhibit superior performance for noisy inputs and carefully chosen kernels [6]. In what follows, morphological bidirectional associative memories have been introduced by Ritter et al. [7], which have the ability to reconstruct input patterns using associated output as well as recalling outputs using input samples. Although associative morphological memories have excellent recall properties, they suffer from the sensitivity to specific noise models [8]. Raducanu et al. proposed a construction method to improve Morphological memory robustness to noise [9].

In this paper a novel hetero-associative memory with a new learning method and chaotic dynamic behavior of its neurons is proposed. A new weight structure is introduced to make the network robust to noise by employing chaotic behavior. There are two weight vectors, one for internal association of input patterns and the other one is proposed to keep input/output association. The first weight vector is trained based on Hebbian

Contents lists available at ScienceDirect journal homepage: www.elsevier.com/locate/neucom

Neurocomputing http://dx.doi.org/10.1016/j.neucom.2015.04.060 0925-2312/& 2015 Elsevier B.V. All rights reserved. n Corresponding author.

E-mail addresses: zhila.aghajari@ee.kntu.ac.ir (Z.H. Aghajari), teshnehlab@eetd.kntu.ac.ir (M. Teshnehlab), jahedmr@iust.ac.ir (M.R. Jahed Motlagh).

Please cite this article as: Z.H. Aghajari, et al., A novel chaotic hetero-associative memory, Neurocomputing (2015), http://dx.doi.org/ 10.1016/j.neucom.2015.04.060i

Neurocomputing ∎ (∎∎∎∎) ∎∎∎–∎∎∎ rule and the second input/output weight vector is adjusted based on input/output correlations due to an extension of sparse learning method which is called Less Correlation Less Effect (LCLE). The structure of weight vectors gives the network ability of successive learning, therefore, it can learn new data after training step and does not need to retrain all previous stored patterns. In the recall step, chaotic neurons and a chaos control method are proposed to help the network converge to the best possible associated input/ output stored pattern. A series of computer simulation shows the effectiveness of the proposed method and significant improvements of the network capacity and noise resistance.