Multiagent Robotic Systems

by Liu; Jiming

• ISBN13: 9780849322884
• ISBN10: 084932288X
• Format: Hardcover
• Copyright: 2001-05-30
• Publisher: CRC Press

Summary

Providing a guided tour of the pioneering work and major technical issues, Multiagent Robotic Systems addresses learning and adaptation in decentralized autonomous robots. Its systematic examination demonstrates the interrelationships between the autonomy of individual robots and the emerged global behavior properties of a group performing a cooperative task. The author also includes descriptions of the essential building blocks of the architecture of autonomous mobile robots with respect to their requirement on local behavioral conditioning and group behavioral evolution.After reading this book you will be able to fully appreciate the strengths and usefulness of various approaches in the development and application of multiagent robotic systems. It covers:· Why and how to develop and experimentally test the computational mechanisms for learning and evolving sensory-motor control behaviors in autonomous robots· How to design and develop evolutionary algorithm-based group behavioral learning mechanisms for the optimal emergence of group behaviors· How to enable group robots to converge to a finite number of desirable task states through group learning· What are the effects of the local learning mechanisms on the emergent global behaviors· How to use decentralized, self-organizing autonomous robots to perform cooperative tasks in an unknown environmentEarlier works have focused primarily on how to navigate in a spatially unknown environment, given certain predefined motion behaviors. What is missing, however, is an in-depth look at the important issues on how to effectively obtain such behaviors in group robots and how to enable behavioral learning and adaptation at the group level. Multiagent Robotic Systems examines the key methodological issues and gives you an understanding of the underlying computational models and techniques for multiagent systems.

Table of Contents

I Motivation, Approaches, and Outstanding Issues	1	(48)
Why Multiple Robots?	3	(8)
Advantages	4	(1)
Major Themes	4	(1)
Agents and Multi-Agent Systems	5	(1)
Multi-Agent Robotics	6	(5)
Toward Cooperative Control	11	(8)
Cooperation-Related Research	12	(1)
Distributed Artificial Intelligence	12	(1)
Distributed Systems	13	(1)
Biology	13	(1)
Learning, Evolution, and Adaptation	13	(2)
Design of Multi-Robot Control	15	(4)
Approaches	19	(8)
Behavior-Based Robotics	20	(1)
Collective Robotics	21	(1)
Evolutionary Robotics	21	(3)
Inspiration from Biology and Sociology	24	(1)
Summary	25	(2)
Models and Techniques	27	(12)
Reinforcement Learning	27	(5)
Markov Decision Process	29	(1)
Reinforcement Learning Algorithms	29	(1)
Temporal Differencing Techniques	29	(1)
Q-Learning	30	(1)
Multi-Agent Reinforcement Learning	30	(2)
Genetic Algorithms	32	(2)
Artificial Life	34	(1)
Artificial Immune System	35	(1)
Probabilistic Modeling	36	(2)
Related Work on Multi-Robot Planning and Coordination	38	(1)
Outstanding Issues	39	(10)
Self-Organization	40	(1)
Local vs. Global Performance	40	(1)
Planning	41	(1)
Multi-Robot Learning	42	(1)
Coevolution	42	(1)
Emergent Behavior	43	(1)
Reactive vs. Symbolic Systems	43	(1)
Heterogeneous vs. Homogeneous Systems	44	(1)
Simulated vs. Physical Robots	45	(1)
Dynamics of Multi-Agent Robotic Systems	46	(1)
Summary	47	(2)
II Case Studies in Learning	49	(78)
Multi-Agent Reinforcement Learning: Technique	51	(14)
Autonomous Group Robots	52	(8)
Overview	52	(1)
Sensing Capability	53	(1)
Long-Range Sensors	53	(1)
Short-Range Sensors	54	(1)
Stimulus Extraction	55	(1)
Primitive Behaviors	56	(3)
Motion Mechanism	59	(1)
Multi-Agent Reinforcement Learning	60	(4)
Formulation of Reinforcement Learning	60	(3)
Behavior Selection Mechanism	63	(1)
Summary	64	(1)
Multi-Agent Reinforcement Learning: Results	65	(28)
Measurements	66	(1)
Stimulus Frequency	66	(1)
Behavior Selection Frequency	66	(1)
Group Behaviors	67	(26)
Collective Surrounding	68	(2)
Cooperation among RANGER Robots	70	(1)
Moving away from Spatially Cluttered Locations	70	(1)
Changing a Target	71	(1)
Cooperatively Pushing Scattered Objects	71	(1)
Collective Manipulation of Scattered Objects	71	(1)
Concurrent Learning in Different Groups of Robots	72	(1)
Concurrent Learning in Predator and Prey	72	(13)
Chasing	85	(1)
Escaping from a Surrounding Crowd	86	(7)
Multi-Agent Reinforcement Learning: What Matters?	93	(20)
Collective Sensing	94	(2)
Initial Spatial Distribution	96	(1)
Inverted Sigmoid Function	97	(1)
Behavior Selection Mechanism	97	(1)
Motion Mechanism	98	(11)
Emerging a Periodic Motion	109	(1)
Macro-Stable but Micro-Unstable Properties	110	(1)
Dominant Behavior	111	(2)
Evolutionary Multi-Agent Reinforcement Learning	113	(14)
Robot Group Example	114	(2)
Target Spatial Distributions	114	(1)
Target Motion Characteristics	114	(1)
Behavior Learning Mechanism	115	(1)
Evolving Group Motion Strategies	116	(4)
Chromosome Representation	116	(1)
Fitness Functions	117	(2)
The Algorithm	119	(1)
Parameters in the Genetic Algorithm	120	(1)
Examples	120	(2)
Summary	122	(5)
III Case Studies in Adaptation	127	(56)
Coordinated Maneuvers in a Dual-Agent System	129	(14)
Issues	130	(1)
Dual-Agent Learning	130	(1)
Specialized Roles in a Dual-Agent System	131	(1)
The Basic Capabilities of the Robot Agent	131	(1)
The Rationale of the Advice-Giving Agent	132	(3)
The Basic Actions: Learning Prerequisites	133	(1)
Genetic Programming of General Maneuveres	133	(1)
Genetic Programming of Specialized Strategic Maneuvers	134	(1)
Acquiring Complex Maneuvers	135	(3)
Experimental Design	135	(1)
The Complexity of Robot Environments	135	(1)
Experimental Results	136	(1)
Lightweight or Heavyweight Flat Posture	137	(1)
Lightweight Curved Posture	137	(1)
Lightweight Corner Posture	138	(1)
Lightweight Point Posture	138	(1)
Summary	138	(5)
Collective Behavior	143	(40)
Group Behavior	144	(2)
What is Group Behavior?	145	(1)
Group Behavior Learning Revisited	145	(1)
The Approach	146	(3)
The Basic Ideas	146	(1)
Group Robots	147	(1)
Performance Criterion for Collective Box-Pushing	147	(1)
Evolving a Collective Box-Pushing Behavior	148	(1)
The Remote Evolutionary Computation Agent	149	(1)
Collective Box-Pushing by Applying Repulsive Forces	149	(6)
A Model of Artificial Repulsive Forces	149	(1)
Pushing Force and the Resulting Motion of a Box	149	(1)
Chromosome Representation	150	(1)
Fitness Function	151	(1)
Examples	152	(1)
Task Environment	152	(1)
Simulation Results	153	(1)
Generation of Collective Pushing Behavior	153	(1)
Adaptation to New Goals	154	(1)
Discussions	154	(1)
Collective Box-Pushing by Exerting External Contact Forces and Torques	155	(20)
Interaction between Three Group Robots and a Box	155	(1)
Pushing a Cylindrical Box	156	(1)
Pushing Position and Direction	156	(1)
Pushing Force and Torque	156	(1)
Pushing a Cubic Box	157	(1)
The Coordinate System	157	(1)
Pushing Force and Torque	157	(1)
Chromosome Representation	157	(1)
Fitness Functions	157	(1)
Examples	158	(1)
Task Environment	158	(1)
Adaptation to New Goals	158	(1)
Simulation Results	158	(14)
Adaptation to Dynamically Changing Goals	172	(2)
Discussions	174	(1)
Convergence Analysis for the Fittest-Preserved Evolution	175	(6)
The Transition Matrix of a Markov Chain	175	(3)
Characterizing the Transition Matrix using Eigenvalues	178	(3)
Summary	181	(2)
IV Case Studies in Self-Organization	183	(48)
Multi-Agent Self-Organization	185	(24)
Artificial Potential Field (APF)	186	(2)
Motion Planning Based on Artificial Potential Field	186	(1)
Collective Potential Field Map Building	187	(1)
Overview of Self-Organization	188	(1)
Self-Organization of a Potential Field Map	189	(5)
Coordinate Systems for a Robot	189	(1)
Proximity Measurements	190	(1)
Distance Association in a Neighboring Region	190	(2)
Incremental Self-Organization of a Potential Field Map	192	(1)
Robot Motion Selection	193	(1)
Directional 1	193	(1)
Directional 2	194	(1)
Random	194	(1)
Experiment 1	194	(1)
Experimental Design	194	(1)
Experimental Results	195	(1)
Experiment 2	195	(1)
Experimental Design	195	(1)
Experimental Results	196	(1)
Discussions	196	(13)
Evolutionary Multi-Agent Self-Organization	209	(22)
Evolution of Cooperative Motion Strategies	210	(5)
Representation of a Proximity Stimulus	212	(1)
Stimulus-Response Pairs	212	(1)
Chromosome Representation	213	(1)
Fitness Functions	214	(1)
The Algorithm	215	(1)
Experiments	215	(3)
Experimental Design	215	(1)
Comparison with a Non-Evolutionary Mode	216	(1)
Experimental Results	217	(1)
Discussions	218	(1)
Evolution of Group Behaviors	218	(1)
Cooperation among Robots	218	(1)
Summary	219	(12)
V An Exploration Tool	231	(48)
Toolboxes for Multi-Agent Robotics	233	(46)
Overview	233	(1)
Toolbox for Multi-Agent Reinforcement Learning	234	(2)
Architecture	234	(1)
File Structure	234	(1)
Function Description	235	(1)
User Configuration	235	(1)
Data Structure	236	(1)
Toolbox for Evolutionary Multi-Agent Reinforcement Learning	236	(1)
File Structure	236	(1)
Function Description	236	(1)
User Configuration	237	(1)
Toolboxes for Evolutionary Collective Behavior Implementation	237	(21)
Toolbox for Collective Box-Pushing by Artificial Repulsive Forces	237	(1)
File Structure	237	(1)
Function Description	237	(1)
User Configuration	237	(2)
Data Structure	239	(1)
Toolbox for Implementing Cylindrical/Cubic Box-Pushing Tasks	240	(1)
File Structure	240	(1)
Function Description	240	(15)
User Configuration	255	(2)
Data Structure	257	(1)
Toolbox for Multi-Agent Self-Organization	258	(2)
Architecture	258	(1)
File Structure	258	(1)
Function Description	258	(1)
User Configuration	258	(1)
Data Structure	258	(2)
Toolbox for Evolutionary Multi-Agent Self-Organization	260	(9)
Architecture	260	(1)
File Structure	260	(1)
Function Description	260	(4)
User Configuration	264	(1)
Data Structure	265	(4)
Example	269	(10)
True Map Calculation	269	(2)
Initialization	271	(2)
Start-Up	273	(1)
Result Display	274	(5)
References	279	(22)
Index	301