Chapter 1
Introduction to ANS Technology 1
1.1 Elementary Neurophysiology 8
1.2 From Neurons to ANS 17
1.3 ANS Simulation 30
Bibliography 4 1
Chapter 2
Adaline and Madaline 45
2.1 Review of Signal Processing 45
2.2 Adaline and the Adaptive Linear Combiner 55
2.3 Applications of Adaptive Signal Processing 68
2.4 The Madaline 72
2.5 Simulating the Adaline 79
Bibliography 86
Chapter 3
Backpropagation 89
3.1 The Backpropagation Network 89
3.2 The Generalized Delta Rule 93
3.3 Practical Considerations 103
3.4 BPN Applications 106
3.5 The Backpropagation Simulator 114
Bibliography 124
Chapter 4
The BAM and the Hopfield Memory 727
4.1 Associative-Memory Definitions 128
4.2 The BAM 731
xi
xii Contents
4.3 The Hopfield Memory 141
4.4 Simulating the BAM 156
Bibliography 167
Chapter 5
Simulated Annealing 769
5.1 Information Theory and Statistical Mechanics 171
5.2 The Boltzmann Machine 179
5.3 The Boltzmann Simulator 189
5.4 Using the Boltzmann Simulator 207
Bibliography 212
Chapter 6
The Counterpropagation Network 273
6.7 CPN Building Blocks 215
6.2 CPN Data Processing 235
6.3 An Image-Classification Example 244
6.4 the CPN Simulator 247
Bibliography 262
Chapter 7
Self-Organizing Maps 263
7.1 SOM Data Processing 265
7.2 Applications of Self-Organizing Maps 274
7.3 Simulating the SOM 279
Bibliography 289
Chapter 8
Adaptive Resonance Theory 297
8.1 ART Network Description 293
8.2 ART1 298
8.3 ART2 316
8.4 The ART 1 Simulator 327
8.5 ART2 Simulation 336
Bibliography 338
Chapter 9
Spatiotemporal Pattern Classification 347
9.1 The Formal Avalanche 342
9.2 Architectures of Spatiotemporal Networks (STNS) 345
Contents xiii
9.3 The Sequential Competitive Avalanche Field 355
9.4 Applications of STNS 363
9.5 STN Simulation 364
Bibliography 371
Chapter 10
The Neocognitron 373
10.1 Neocognitron Architecture 376
10.2 Neocognitron Data Processing 381
10.3 Performance of the Neocognitron 389
10.4 Addition of Lateral Inhibition and Feedback to the
Neocognitron 390
Bibliography 393