Computational Intelligence

by Konar, Amit

• ISBN13: 9783540208983
• ISBN10: 3540208984
• Edition: CD
• Format: Hardcover
• Copyright: 2005-06-30
• Publisher: Springer-Verlag New York Inc

Summary

Computational Intelligence: Principles, Techniques and Applications presents both theories and applications of computational intelligence in a clear, precise and highly comprehensive style. The textbook addresses the fundamental aspects of fuzzy sets and logic, neural networks, evolutionary computing and belief networks. The application areas include fuzzy databases, fuzzy control, image understanding, expert systems, object recognition, criminal investigation, telecommunication networks, and intelligent robots. The book contains many numerical examples and homework problems with sufficient hints so that the students can solve them on their own. A CD-ROM containing the simulations is supplied with the book, to enable interested readers to develop their own application programs with the supplied C/ C++ toolbox.

Table of Contents

Chapter 1: An Introduction to Computational Intelligence
1.1 Artificial Intelligence- a Brief Review	1	(2)
1.2 Pitfalls of the Traditional AI	3	(1)
1.3 Computational Intelligence- an Emergence of a New Computational Paradigm	4	(3)
1.4 Computational Intelligence- a Formal Definition	7	(1)
1.5 Soft Computing- Definitions	8	(1)
1.6 Fundamental Elements of Soft Computing	9	(14)
1.6.1 The Logic of Fuzzy Sets	9	(1)
1.6.2 Computational Models of Neural Nets	10	(7)
1.6.3 Genetic Algorithms	17	(3)
1.6.4 Belief Networks	20	(3)
1.7 Computational Learning Theory	23	(2)
1.8 Synergism in Soft Computing	25	(2)
1.8.1 Neuro-Fuzzy Synergism	26	(1)
1.8.2 Neuro-GA Synergism	26	(1)
1.8.3 Fuzzy-GA Synergism	26	(1)
1.8.4 Neuro-Belief Network Synergism	26	(1)
1.8.5 GA-Belief Network Synergism	27	(1)
1.8.6 Neuro-Fuzzy-GA Synergism	27	(1)
1.9 Conclusions	27	(1)
Exercise	27	(2)
References	29	(8)
Chapter 2: Fuzzy Sets and Relations
2.1 Conventional Sets	37	(1)
2.2 Fuzzy Sets	38	(2)
2.3 Membership Functions	40	(2)
2.4 Continuous and Discrete Membership Functions	42	(1)
2.5 Typical Membership Functions	43	(5)
2.5.1 The γ-Function	43	(1)
2.5.2 The s-Function	44	(1)
2.5.3 The L-Function	45	(1)
2.5.4 The Triangular Membership Function	46	(1)
2.5.5 The II-Function	47	(1)
2.5.6 The Gaussian Membership Function	47	(1)
2.6 Operations on Fuzzy Sets	48	(2)
2.6.1 Fuzzy T-Norm	48	(1)
2.6.2 Fuzzy S-Norm	49	(1)
2.6.3 Fuzzy Complement	50	(1)
2.7 Basic Concepts Associated with Fuzzy Sets	50	(2)
2.8 Extension Principle of Fuzzy Sets	52	(2)
2.9 Fuzzy Relations	54	(2)
2.10 Projection of Fuzzy Relations	56	(1)
2.11 Cylindrical Extension of Fuzzy Relations	57	(1)
2.12 Fuzzy Max-Min and Max-Product Composition Operation	58	(2)
2.13 Fuzzy Linguistic Hedges	60	(2)
2.14 Summary	62	(1)
Exercise	63	(3)
References	66	(1)
Chapter 3: Fuzzy Logic and Approximate Reasoning
3.1 Production Systems	67	(2)
3.2 Conflict Resolution in Production Systems	69	(1)
3.3 Drawbacks of Traditional Production Systems	69	(1)
3.4 Fuzzy Implication Rules	70	(1)
3.5 Fuzzy Implication Relations	71	(2)
3.6 Fuzzy Logic	73	(2)
3.6.1 Typical Propositional Inference Rules	73	(1)
3.6.2 Fuzzy Extension of the Inference Rules	74	(1)
3.7 The Composition Rule of Inference	75	(5)
3.7.1 Computing Fuzzy Inferences in GMP	75	(3)
3.7.2 Computing Fuzzy Inferences Using GMT	78	(1)
3.7.3 Computing Fuzzy Inferences Using GHS	79	(1)
3.8 Approximate Reasoning with Multiple Antecedent Clauses	80	(3)
3.9 Approximate Reasoning with Multiple Rules	83	(3)
3.10 Scope of Parallelism in Approximate Reasoning Using Mamdani Implication Function	86	(1)
3.11 Realization of Fuzzy Inference Engine on VLSI Architecture	87	(2)
3.12 Approximate Reasoning with Multiple Rules Each with Multiple Antecedent Clauses	89	(1)
3.13 Fuzzy Abductive Reasoning	90	(3)
3.14 Conclusions	93	(1)
Exercise	93	(2)
References	95	(2)
Chapter 4: Fuzzy Logic in Process Control
4.1 Process Control	97	(2)
4.2 Advantages of Fuzzy Control	99	(1)
4.3 Typical Fuzzy Control Systems	99	(2)
4.4 Architecture of Typical Fuzzy control Systems	101	(1)
4.5 Reasoning in Mamdani Type Fuzzy Control Systems	101	(5)
4.6 Reasoning in T-S Fuzzy Control Systems	106	(3)
4.7 Stability Analysis of Dynamic Systems Using Lyapunov Energy Functions	109	(1)
4.8 Stability Analysis of T-S Fuzzy Systems	110	(4)
4.9 Application in Power Control of a Nuclear Reactor	114	(5)
4.10 Defuzzification Techniques	119	(1)
4.11 Conclusions	120	(1)
Exercise	120	(2)
References	122	(3)
Chapter 5: Fuzzy Pattern Recognition
5.1 Introduction	125	(2)
5.2 The Fuzzy C-Means Clustering Algorithm	127	(5)
5.3 Image Segmentation Using Fuzzy C-Means Clustering Algorithm	132	(4)
5.4 Conclusions	136	(1)
Exercise	136	(1)
References	137	(2)
Chapter 6: Fuzzy Databases and Possibilistic Reasoning
6.1 Introduction to Relational Database Systems	139	(4)
6.1.1 The Relational Model	140	(3)
6.2 Issues in Relational database Design	143	(3)
6.2.1 Lossless Join Decomposition	143	(1)
6.2.2 Functional Dependency Preservation	144	(2)
6.3 Possibility Distribution and Fuzzy Sets	146	(1)
6.4 Fuzziness in Relational Models	147	(5)
6.4.1 Type-1 Fuzzy Relational Data Model	147	(2)
6.4.2 Type-2 Fuzzy Relational Data Model	149	(3)
6.5 Fuzzy Relational Operators	152	(3)
6.5.1 Projection of Fuzzy Relations	152	(1)
6.5.2 Cylindrical Extension of Fuzzy Relations	153	(1)
6.5.3 Natural Join of Fuzzy Relations	154	(1)
6.5.4 Lossy Fuzzy Joins	155	(1)
6.6 Fuzzy Integrity Constraints	155	(3)
6.6.1 Conjunction of Fuzzy Propositions	156	(1)
6.6.2 The Modifier Rule	157	(1)
6.6.3 Possibility Distribution for Conditional Fuzzy Propositions	158	(1)
6.7 Fuzzy Functional Dependency	158	(3)
6.7.1 Equality as a Fuzzy Relation	158	(2)
6.7.2 Representing Functional Dependency Using Equality Relation	160	(1)
6.8 Fuzzy Lossless Join	161	(1)
6.9 Design of Fuzzy Relational Databases	162	(1)
6.10 Conclusions	162	(1)
Exercise	163	(2)
References	165	(2)
Chapter 7: Introduction to Machine Learning Using Neural Nets
7.1 Biological Neural Networks	167	(2)
7.2 Artificial Neural Networks	169	(1)
7.3 Principles of Learning in a Neural Net	170	(9)
7.4 Stability and Convergence	179	(1)
7.5 Three Important Theorems for Stability Analysis of Neural Dynamics	180	(13)
7.6 Conclusions	193
Exercise	184	(10)
References	194	(3)
Chapter 8: Supervised Neural Learning Algorithms
8.1 Introduction	197	(1)
8.2 McCulloch-Pitts Model	198	(3)
8.3 The Perceptron Learning Model	201	(8)
8.3.1 Linear Classification by Perceptrons	203	(4)
8.3.2 Multilayered Perceptron Classifier	207	(2)
8.4 Widrow-Hoff s ADALINE Model	209	(8)
8.5 The Back-propagation Learning Algorithm	217	(6)
8.6 The Radial Basis Function Neural Net	223	(2)
8.7 Modular Neural Nets	225	(5)
8.8 Conclusions	230	(1)
Exercise	230	(3)
References	233	(4)
Chapter 9: Unsupervised Neural Learning Algorithms
9.1 Introduction	237	(1)
9.2 Recurrent Neural Nets	238	(15)
9.2.1 Discrete Hopfield Network	239	(2)
9.2.2 Continuous Hopfield Neural Net	241	(4)
9.2.3 A Near-Equilibrium Consideration for Classification by Continuous Hopfield Nets	245	(1)
9.2.4 Optimization Using Hopfield (Continuous) Neural Nets	246	(2)
9.2.5 Application of Hopfield Neural Nets in Engineering Problems	248	(4)
9.2.6 Boltzman Machines	252	(1)
9.3 Bi-directional Associative Memory	253	(3)
9.4 Adaptive Resonance Theory	256	(4)
9.5 Fuzzy Associative Memory	260	(2)
9.6 Discussions	262	(1)
Exercise	263	(2)
References	265	(2)
Chapter 10: Competitive Learning Using Neural Nets
10.1 Introduction	267	(3)
10.2 Two Layered Recurrent Networks for Competitive Learning	270	(2)
10.3 Components of a Competitive Learning Network	272	(4)
10.3.1 The Pre-Processing Layer	272	(1)
10.3.2 The Instar Connectivity from Neurons of the Input to the Output Layer	273	(1)
10.3.3 Competitive Learning in the Output Layer	274	(2)
10.4 Hebbian Learning in the Competitive Layer	276	(3)
10.5 Analysis of Pattern Clustering Network	279	(2)
10.6 Principal Component Analysis	281	(1)
10.7 Self-Organizing Feature Map	282	(2)
10.8 Application in Face Recognition	284	(5)
10.9 Conclusions	289	(1)
Exercise	289	(3)
References	292	(3)
Chapter 11: Neuro-dynamic Programming by Reinforcement Learning
11.1 Introduction	295	(3)
11.2 Formulation of the Reinforcement Learning Paradigm	298	(4)
11.3 Q-Learning	302	(5)
11.4 Convergence of Q-Learning Algorithm	307	(1)
11.5 Nondeterministic Rewards and Actions	308	(2)
11.6 Temporal Difference Learning	310	(1)
11.7 Neural Reinforcement Learning	311	(4)
11.8 Multi-Agent Reinforcement Learning	315	(2)
11.9 Conclusions	317	(1)
Exercise	318	(3)
References	321	(2)
Chapter 12: Evolutionary Computing Algorithms
12.1 Introduction	323	(1)
12.2 Genetic Algorithm: How does it work?	324	(7)
12.3 Deterministic Explanation of Holland's Observation	331	(1)
12.4 Stochastic Explanation of GA	332	(3)
12.5 The Markov Model for Convergence Analysis	335	(4)
12.6 Application of GA in Optimization Problems	339	(2)
12.7 Application of GA in Machine Learning	341	(4)
12.7.1 GA as an Alternative to Back-propagation Learning	342	(1)
12.7.2 Adaptation of the Learning Rule/ Control Law by GA	342	(3)
12.8 Application of GA in Intelligent Search	345	(1)
12.8.1 Navigational Planning of Robots	345	(1)
12.9 Genetic Programming	346	(2)
12.10 Conclusions	348	(1)
Exercise	349	(1)
References	350	(3)
Chapter 13: Belief Calculus and Probabilistic Reasoning
13.1 Introduction	353	(1)
13.2 Elements of Probability Theory	354	(7)
13.2.1 Bayes' Law on Conditional Probability	356	(5)
13.3 Belief Propagation on a Causal Tree	361	(8)
13.4 Pearl's Belief Propagation Scheme on a Polytree	369	(3)
13.5 Dempster-Shafer Theory for Uncertainty Management	372	(5)
13.6 Conclusions	377	(1)
Exercise	377	(14)
References	391	(3)
Chapter 14: Reasoning in Expert Systems Using Fuzzy Petri Nets
14.1 Introduction	394	(2)
14.2 Imprecision Management in an Acyclic FPN	396	(8)
14.2.1 Formal Definitions and the Proposed Model	396	(1)
14.2.2 Proposed Model for Belief Propagation	396	(2)
14.2.3 Proposed Algorithm for Belief Propagation	398	(6)
14.3 Imprecision and Inconsistency Management in a Cyclic FPN	404	(11)
14.3.1 Proposed Model for Belief Revision	404	(1)
14.3.2 Stability Analysis of the Belief Revision Model	405	(6)
14.3.3 Detection and Elimination of Limit cycles	411	(3)
14.3.4 Nonmonotonic Reasoning in an FPN	414	(1)
14.4 Conclusions	415	(4)
Exercise	419	(1)
References	419	(4)
Chapter 15: Image Matching Using Fuzzy Moment Descriptors
15.1 Introduction	423	(2)
15.2 Image Features and their Membership Distributions	425	(4)
15.2.1 Fuzzy Membership Distributions	426	(2)
15.2.2 Fuzzy Production Rules	428	(1)
15.3 Fuzzy Moment Descriptors	429	(2)
15.4 Image Matching Algorithm	431	(2)
15.5 Rotation and Size Invariant Matching	433	(1)
15.6 Noise Insensitive Matching	433	(1)
15.7 Computer Simulation	434	(1)
15.8 Implication of the Results	435	(1)
15.9 Template Matching Using Interleaved Search	435	(4)
15.10 Computer Simulation of Template Matching	439	(1)
15.11 Human Mood Detection from Facial Expressions	440	(6)
15.11.1 Image Segmentation and Localization of Facial Components	441	(1)
15.11.2 Facial Extracts and their Measurements for Mood Analysis	442	(4)
15.12 Conclusions	446	(1)
Exercise	447	(3)
References	450	(3)
Chapter 16: Behavioral Synergism of Soft Computing Tools
16.1 Introduction	453	(2)
16.2 Neuro-Fuzzy Synergism	455	(4)
16.2.1 Weakly Coupled Neuro-Fuzzy Systems	455	(1)
16.2.2 Tightly Coupled Neuro-Fuzzy Systems	456	(3)
16.3 Fuzzy-GA Synergism	459	(1)
16.4 Neuro-GA Synergism	460	(2)
16.4.1 Adaptation of a Neural Learning Algorithm Using GA	460	(2)
16.5 GA-Belief Network Synergism	462	(1)
16.6 A Case Study of Synergism of 2 Neural Topology and GA	462	(6)
16.6.1 The Problem	462	(1)
16.6.2 Clustering by TASONN	463	(1)
16.6.3 Continuous Hopfield Net- A Review	463	(1)
16.6.4 Perception-to-Action Transformation	464	(1)
16.6.5 Optimization of the Energy Function Using GA	465	(1)
16.6.6 Experimental Details	466	(2)
16.7 Conclusions and Future Directions	468	(1)
Exercise	469	(6)
References	475	(2)
Chapter 17: Object Recognition from Gray Images Using Fuzzy ADALINE Neurons
17.1 Introduction	477	(3)
17.2 Proposed Model of ADALINE	480	(2)
17.3 Stability Analysis for Convergence of the ADALINE Model	482	(2)
17.4 Training of the Proposed Neural Net	484	(6)
17.4.1 Training Algorithm of ADALINES	484	(3)
17.4.2 Training of the Neural Net	487	(1)
17.4.3 Training with Multiple Input-Output Patterns	488	(2)
17.5 Translation Rotation and Size Invariant Gray Pattern Recognition	490	(3)
17.5.1 Design of Translational Invariance Network	491	(1)
17.5.2 Design of Rotational Invariance Network	492	(1)
17.5.3 Design of Size Invariance Network	493	(1)
17.6 Conclusions	493	(1)
Exercise	494	(1)
References	495	(2)
Chapter 18: Distributed Machine Learning Using Fuzzy Cognitive Maps
18.1 Introduction	497	(1)
18.2 Axelrod's Cognitive Maps	498	(2)
18.3 Kosko's Model	500	(3)
18.4 Kosko's Extended Model	503	(1)
18.5 Adaptive FCMs	504	(1)
18.6 Zhang, Chen and Bezdek's Model	505	(2)
18.7 Pal and Konar's Model	507	(6)
18.8 Conclusions	513	(1)
Exercise	513	(4)
References	517	(4)
Chapter 19: Machine Learning Using Fuzzy Petri Nets
19.1 Introduction	521	(2)
19.2 The Proposed Model for Cognitive Reasoning	523	(3)
19.2.1 Encoding of Weights	524	(1)
19.2.2 The recall Model	524	(2)
19.3 State-Space Formulation	526	(2)
19.3.1 State-Space Model for Belief Updating	527	(1)
19.3.2 State-Space Model for FTT Updating of Transitions	527	(1)
19.3.3 State-Space Model for Weights	528	(1)
19.4 Stability Analysis of the Cognitive Model	528	(4)
19.5 Computer Simulation	532	(3)
19.6 Implication of the Results	535	(1)
19.7 Knowledge Refinement by Hebbian Learning	535	(7)
19.7.1 The Encoding Model	535	(2)
19.7.2 The Recall/ Reasoning Model	537	(1)
19.7.3 Case Study by Computer Simulation	537	(5)
19.7.4 Implication of the Results	542	(1)
19.8 Conclusions	542	(2)
Exercise	544	(1)
References	545	(2)
Chapter 20: Computational Intelligence in Telecommunication Networks
20.1 Introduction	547	(2)
20.2 Network Routing Using Genetic Algorithms	549	(5)
20.2.1 Path-Genetic Operators	551	(2)
20.2.2 Fitness Evaluation	553	(1)
20.2.3 The Genetic Based Routing Algorithm	553	(1)
20.3 Computational Intelligence in Network Congestion Control	554	(6)
20.3.1 Fuzzy Congestion Controller	555	(5)
20.4 Computational Intelligence in Handling the Call Admission Control Problem	560	(5)
20.4.1 Call Admission Control Using Neural Networks	561	(1)
20.4.2 Input/Outputs of Neural Nets Used in Call Admission Control	562	(3)
20.5 Intelligent Traffic Control	565	(1)
20.6 Conclusions	566	(1)
Exercise	567	(3)
References	570	(1)
Chapter 21: Computational Intelligence in Mobile Robotics
21.1 Introduction	571	(2)
21.2 Path Planning of a Mobile Robot Using Neural Nets	573	(16)
21.2.1 Generation of Training Instances	574	(6)
21.2.2 Configuring the Neural Nets	580	(2)
21.2.3 Parameters Used for Performance Evaluation of the Neural Path-Planning Algorithms	582	(2)
21.2.4 Experimental Results- a Benchmark Analysis	584	(5)
21.2.5 Implication of the Results	589	(1)
21.3 Object Localization	589	(3)
21.4 Target Tracking and Interception by Mobile Robots Using Kalman Filtering	592	(9)
21.4.1 Measurements of the Input to Kalman Filter	595	(1)
21.4.2 Extended Kalman Filter- an Overview	596	(1)
21.4.3 Predicting Target Position Using Extended Kalman Filter	597	(3)
21.4.4 Use of the Back-propagation Neural Net	600	(1)
21.4.5 Experimental Results	600	(1)
21.5 Conclusions and Future Directions	601	(10)
Exercise
References
Chapter 22: Emerging Areas of Computational Intelligence
22.1 Introduction	611	(2)
22.2 Artificial Life	613	(4)
22.2.1 The Model of Artificial Fish Positions, Speed and Heading	613	(1)
22.2.2 Repulsion, Attraction and Aligning	614	(2)
22.2.3 Body Size and Form	616	(1)
22.3 Particle Swarm Optimization	617	(3)
22.3.1 Particle Swarm Size	619	(1)
22.3.2 The Split Swarm Algorithm	619	(1)
22.4 Artificial Immune Systems	620	(7)
22.4.1 Biological Communication among Different Species of Antibodies	621	(1)
22.4.2 A Simple Model of the Immune System	622	(5)
22.5 Fuzzy Chaos Theory 624 22.6 Rough Sets	627	(6)
22.6.1 Rough-Fuzzy Sets 629 22.7 Granular Computing	630	(2)
22.7.1 Rough Inclusion and Indiscernibility	632	(1)
22.7.2 Classifier Design	632	(1)
22.8 Redefining Computational Intelligence	633	(1)
22.9 Summary	634	(1)
Exercise	635	(5)
References	640	(3)
Chapter 23: Research Problems for Graduate Thesis and Pre-Ph D Preparatory Courses
23.1 Problem 1: Computing Max-Min Inverse Fuzzy Relation	643	(1)
23.2 An Outline to the Solution of Problem 1	644	(1)
23.3 Tutorial Assignment for Graduate Students	645	(1)
23.4 Problem 2: Reasoning with Fuzzy Petri Nets	646	(3)
23.4.1 Tutorial Problems on Fuzzy Petri Nets	646	(1)
23.4.2 Research Problems on Fuzzy Petri Nets	647	(2)
23.5 Problem 3: Spoken Word Reconstruction from Lip Stylus Sequences	649	(1)
23.6 Problem 4: Fuzzy State Equations and stability Analysis	650	(1)
23.7 An Outline to the Solution of Problem 4	651	(1)
23.8 Graduate Students' Assignments on Stability of Fuzzy State Equations	652	(1)
23.9 Problem 5: Fuzzy Data Mining	652	(1)
23.10 An approach to the Solution of Problem 5	653	(2)
23.11 Graduate Assignments on Fuzzy Data Mining	655	(1)
23.12 Problem 6: Selected Items in Signal Processing and Communication Engineering	655	(5)
23.12.1 System Modeling	656	(1)
23.12.2 Non-linear Prediction Problem	657	(1)
23.12.3 Inverse Modeling	658	(1)
23.12.4 Adaptive Channel Equalization	659	(1)
23.13 Problem 7: Handling Large Training Instances Using Back-Propagation-RBF Synergism	660	(2)
23.14 Problem 8: Macro Cell Placement and Routing in VLSI Design Using Genetic Algorithms	662	(1)
23.15 A Possible Solution to Macro Cell Placement and Routing	663	(1)
23.16 Problem 9: Drug Classification Problem Using Artificial Neural Networks	664	(3)
References	667	(2)
Appendix A: Sample Run of Programs Included in the CD
A.1 Detection of Mouth Opening of a Person from his/her Facial Image	669	(5)
A.2 A Program for Image Matching Using Fuzzy Moment Descriptors	674	(2)
A.3 GA in Path Planning of a Mobile Robot	676	(1)
A.4 The FPNreas1 Program	677	(6)
A.5 The FPNreas2 Program	683	(1)
A.6 The FPNtreas3 Program	684	(2)
A.7 Fuzzy Chaos Program	686	(2)
A.8 The New_Petri3 Program to Study Controllability on FPN	688	(3)
Appendix B: Evolutionary Algorithms of Current Interest
B.1 Ant Colony Systems: an Overview	691	(5)
B.1.1 The ACO Algorithm	693	(1)
B.1.2 Solving the classical TSP Problem by ACO	693	(2)
B.1.3 Application Domain of ACO	695	(1)
B.2 Differential Evolution	696	(2)
B.2.1 Outline of the Algorithm	696	(2)
B.2.2 Application Domain of DE and Recent Research	698	(1)
References	698	(3)
Index	701	(6)
About the Author	707