Holographic associative memory
Holographic associative memory (HAM) Is a form of information storage where two pieces of information are saved and retrieved by associating them with one another in a pattern such that any part of the pattern contains them both and either piece can be used to retrieve the other. It has its roots in the principles of holography. Holograms are made by using two beams of light, called a "reference beam" and an "object beam". They produce a pattern on the film that contains them both. Afterwards, by reproducing the reference beam, the hologram recreates a visual image of the original object. In theory, one could use the object beam to do the same thing: reproduce the original reference beam. In HAM, the pieces of information act like the two beams. Each can be used to retrieve the other from the pattern.
Definition
HAM is part of the family of analog, correlation-based, associative, stimulus-response memories, where information is mapped onto the phase orientation of complex numbers operating. It can be considered as a complex valued artificial neural network. The holographic associative memory exhibits some remarkable characteristics. Holographs have been shown to be effective for associative memory tasks, generalization, and pattern recognition with changeable attention. Ability of dynamic search localization is central to natural memory. For example, in visual perception, humans always tend to focus on some specific objects in a pattern. Humans can effortlessly change the focus from object to object without requiring relearning. HAM provides a computational model which can mimic this ability by creating representation for focus. At the heart of this new memory lies a novel bi-modal representation of pattern and a hologram-like complex spherical weight state-space. Besides the usual advantages of associative computing, this technique also has excellent potential for fast optical realization because the underlying hyper-spherical computations can be naturally implemented on optical computations.
It is based on principle of information storage in the form of stimulus-response patterns where information is presented by phase angle orientations of complex numbers on a Riemann surface.[1] A very large number of stimulus-response patterns may be superimposed or "enfolded" on a single neural element. Stimulus-response associations may be both encoded and decoded in one non-iterative transformation. The mathematical basis requires no optimization of parameters or error backpropagation, unlike connectionist neural networks. The principal requirement is for stimulus patterns to be made symmetric or orthogonal in the complex domain. HAM typically employs sigmoid pre-processing where raw inputs are orthogonalized and converted to Gaussian distributions.
Principles of operation
1) Stimulus-response associations are both learned and expressed in one non-iterative transformation. No backpropagation of error terms or iterative processing required.
2) The method forms a non-connectionist model in which the ability to superimpose a very large set of analog stimulus-response patterns or complex associations exists within the individual neuron cell.
3) The generated phase angle communicates response information, and magnitude communicates a measure of recognition (or confidence in the result).
4) The process permits a capability with neural system to establish dominance profile of stored information, thus exhibiting a memory profile of any range - from short-term to long-term memory.
5) The process follows the non-disturbance rule, that is prior stimulus-response associations are minimally influenced by subsequent learning.
6) The information is presented in abstract form by a complex vector which may be expressed directly by a waveform possessing frequency and magnitude. This waveform is analogous to electrochemical impulses believed to transmit information between biological neuron cells.
References
- Sutherland, John G. (1 January 1990). "A holographic model of memory, learning and expression". International Journal of Neural Systems. 01 (3): 259–267. doi:10.1142/S0129065790000163.
Bibliography
- Sutherland, J., Holographic Models of Memory, Learning and Expression, International Journal of Neural Systems, 1(3), 1990, pp356–267
- J. I. Khan. Attention Modulated Associative Computing and Content-Associative Search in Image Archive. PhD thesis, University of Hawaii, August 1995.
- K. I. Khan and D. Y. Yun. Characteristics of Multidimensional Holographic Associative Memory in Retrieval with Dynamically Localizable Attention. IEEE Transactions on Neural Networks, 9(3):389–406, May 1998.
- HE Michel, AAS Awwal, Enhanced artificial neural networks using complex numbers, Neural Networks, 1999. Proceedings. 1999 IEEE International Joint Conference on
- R Stoop, J Buchli, G Keller, WH Steeb, Stochastic resonance in pattern recognition by a holographic neuron model, Physical Review E, 2003.
- Y Hendra, RP Gopalan, MG Nair, A method for dynamic indexing of large image databases, Systems, Man, and Cybernetics, 1999. IEEE SMC'99.
- HE Michel, S Kunjithapatham, Processing Landsat TM data using complex-valued neural networks, Proceedings of SPIE, the International Society for Optical, 2002.
- RP Gopalan, G Lee, Indexing of Image Databases Using Untrained 4D Holographic Memory Model, 15th Australian Joint Conference on Artificial Intelligence, - Springer Page 1. RI McKay and J. Slaney (Eds.): AI 2002, LNAI 2557, pp. 237–248.
- RWTH Aachen, IH Ney, Approaches to Invariant Image Object Recognition,