Layered Learning in Multiagent Systems : A Winning Approach to Robotic Soccer

by Peter Stone

• ISBN13: 9780262194389
• ISBN10: 0262194384
• Format: Hardcover
• Copyright: 2000-03-17
• Publisher: Bradford Books

Summary

This book looks at multiagent systems that consist of teams of autonomous agents acting in real-time, noisy, collaborative, and adversarial environments. The book makes four main contributions to the fields of machine learning and multiagent systems. First, it describes an architecture within which a flexible team structure allows member agents to decompose a task into flexible roles and to switch roles while acting. Second, it presents layered learning, a general-purpose machine-learning method for complex domains in which learning a mapping directly from agents' sensors to their actuators is intractable with existing machine-learning methods. Third, the book introduces a new multiagent reinforcement learning algorithm--team-partitioned, opaque-transition reinforcement learning (TPOT-RL)--designed for domains in which agents cannot necessarily observe the state-changes caused by other agents' actions. The final contribution is a fully functioning multiagent system that incorporates learning in a real-time, noisy domain with teammates and adversaries--a computer-simulated robotic soccer team. Peter Stone's work is the basis for the CMUnited Robotic Soccer Team, which has dominated recent RoboCup competitions. RoboCup not only helps roboticists to prove their theories in a realistic situation, but has drawn considerable public and professional attention to the field of intelligent robotics. The CMUnited team won the 1999 Stockholm simulator competition, outscoring its opponents by the rather impressive cumulative score of 110-0.

Table of Contents

Preface	ix
Introduction	1	(8)
Substrate Systems	9	(32)
Team Member Agent Architecture	41	(48)
Layered Learning	89	(16)
Learning an Individual Skill	105	(12)
Learning a Multiagent Behavior	117	(24)
Learning a Team Behavior	141	(40)
Competition Results	181	(16)
Related Work	197	(34)
Conclusions and Future Work	231	(8)
Appendix A: List of Acronyms	239	(2)
Appendix B: Robotic Soccer Agent Skills	241	(14)
Appendix C: CMUnited-98 Simulator Team Behavior Modes	255	(4)
Appendix D: CMUnited Simulator Team Source Code	259	(2)
References	261