Handbook of Virtual Humans

by Magnenat-Thalmann, Nadia; Thalmann, Daniel

• ISBN13: 9780470023167
• ISBN10: 0470023163
• Edition: 1st
• Format: Hardcover
• Copyright: 2004-10-29
• Publisher: WILEY

Summary

Virtual Humans are becoming more and more popular and used in many applications such as the entertainment industry (in both film and games) and medical applications. This comprehensive book covers all areas of this growing industry including face and body motion, body modelling, hair simulation, expressive speech simulation and facial communication, interaction with 3D objects, rendering skin and clothes and the standards for Virtual Humans. Written by a team of current and former researchers at MIRALab, University of Geneva or VRlab, EPFL, this book is the definitive guide to the area. Explains the concept of avatars and autonomous virtual actors and the main techniques to create and animate them (body and face). Presents the concepts of behavioural animation, crowd simulation, intercommunication between virtual humans, and interaction between real humans and autonomous virtual humans Addresses the advanced topics of hair representation and cloth animation with applications in fashion design Discusses the standards for Virtual Humans, such as MPEG-4 Face Animation and MPEG-4 Body Animation.

Author Biography

Professor Nadia Magnenat Thalmann is an eminent computer graphics scientist who is the founder and head of MIRALab at the University of Geneva. She presently serves as the Director of the Institute for Media Innovation in Singapore at Nanyang Technological University. Prof. Daniel Thalmann is a Swiss and Canadian Computer Scientist and a pioneer in Virtual Humans. He is currently Honorary Professor at EPFL, Switzerland, and Director of Research Development at MIRALab Sarl.

Table of Contents

Preface	xiv
List of Contributors	xv
List of Figures	xvi
List of Tables	xxiv
An Overview of Virtual Humans	1	(25)
Nadia Magnenat-Thalmann
Daniel Thalmann
Why Virtual Humans?	1	(2)
History of Virtual Humans	3	(5)
Early Models	3	(2)
Short Films and Demos	5	(2)
The Evolution towards Real-Time	7	(1)
The Applications of Virtual Humans	8	(12)
Numerous Applications	8	(1)
Virtual Presenters for TV and the Web	8	(1)
Virtual Assistants for Training in Case of Emergency	9	(2)
Virtual Ancient People in Inhabited Virtual Cultural Heritage	11	(1)
Virtual Audience for Treatment of Social Phobia	12	(1)
Virtual Mannequins for Clothing	13	(1)
Virtual Workers in Industrial Applications	14	(2)
Virtual Actors in Computer-Generated Movies	16	(2)
Virtual Characters in Video Games	18	(2)
The Challenges in Virtual Humans	20	(4)
A Good Representation of Faces and Bodies	20	(1)
A Flexible Motion Control	20	(1)
A High-Level Behavior	21	(1)
Emotional Behavior	22	(1)
A Realistic Appearance	22	(1)
Interacting with the Virtual World	23	(1)
Interacting with the Real World	24	(1)
Conclusion	24	(2)
Face Cloning and Face Motion Capture	26	(26)
Wonsook Lee
Taro Goto
Sumedha Kshirsagar
Tom Molet
Introduction	26	(1)
Feature-Based Facial Modeling	27	(19)
Facial Modeling Review and Analysis	27	(4)
Generic Human Face Structure	31	(1)
Photo-Cloning	32	(1)
Feature Location and Shape Extraction	33	(5)
Shape Modification	38	(1)
Texture Mapping	39	(2)
Face Cloning from Range Data	41	(1)
Validation of the Face Cloning Results	41	(5)
Facial Motion Capture	46	(6)
Motion Capture for Facial Animation	46	(1)
MPEG-4-Based Face Capture	47	(2)
Generation of Static Expressions or Key-Frames	49	(1)
Analysis of Facial Capture Data to Improve Facial Animation	50	(2)
Body Cloning and Body Motion Capture	52	(23)
Pascal Fua
Ralf Plaenkers
WonSook Lee
Tom Molet
Introduction	52	(1)
Body Models for Fitting Purposes	53	(2)
Stick Figure	53	(1)
Simple Volumetric Primitives	53	(1)
Multi-Layered Models	54	(1)
Anatomically Correct Models	55	(1)
Static Shape Reconstruction	55	(5)
3-D Scanners	55	(1)
Finding Structure in Scattered 3-D Data	56	(1)
Conforming Animatable Models to 3-D Scanned Data	57	(1)
Photo-Based Shape Reconstruction	58	(2)
Video-Based Shape Reconstruction	60	(1)
Dynamic Motion Capture	60	(7)
Early Motion Analysis	61	(1)
Electro-Magnetic and Optical Motion Capture Systems	62	(2)
Video-Based Motion Capture	64	(3)
Articulated Soft Objects for Shape and Motion Estimation	67	(4)
State Vector	67	(1)
Metaballs and Quadratic Distance Function	68	(1)
Optimization Framework	68	(2)
Implementation and Results	70	(1)
Conclusion	71	(4)
Anthropometric Body Modeling	75	(24)
Hyewon Seo
Introduction	75	(2)
Background	77	(9)
Anthropometry	77	(4)
Anthropometric Human Models in CG	81	(3)
Motivating Applications	84	(1)
Challenging Problems	85	(1)
Our Approaches to Anthropometric Models	86	(12)
Overview	87	(1)
Data Acquisition	87	(2)
Pre-Processing	89	(5)
Interpolator Construction	94	(2)
Results and Implementation	96	(2)
Conclusion	98	(1)
Body Motion Control	99	(20)
Ronan Boulic
Paolo Baerlocher
Introduction	99	(2)
State of the Art in 3-D Character Animation	101	(12)
The Levels of Abstraction of the Musculo-Skeletal System	101	(2)
Techniques for the Animation and the Control of the Multi-Body System	103	(1)
What Is Motion?	104	(3)
Background to Inverse Kinematics	107	(2)
Review of Inverse Kinematics Resolution Methods	109	(3)
Other Issues in the Production of 3-D Character Animation	112	(1)
The Multiple Priority Levels IK (MPL-IK)	113	(3)
Background to Numeric IK	113	(2)
Handling Two Conflicting Constraints	115	(1)
Generalizing to p Priority Levels	116	(1)
MPL-IK Results	116	(1)
Conclusion	117	(2)
Facial Deformation Models	119	(21)
Prem Kalra
Stephane Garchery
Sumedha Kshirsagar
Introduction	119	(1)
Some Preliminaries about the Anatomy of a Face	120	(2)
Skin	120	(1)
Muscles	120	(1)
Bone	121	(1)
Control Parameterization	122	(1)
Interpolation	122	(1)
FACS (Facial Action Coding System)	122	(1)
FAP (Facial Animation Parameters)	123	(1)
Facial Deformation Models	123	(13)
Shape Interpolation	123	(1)
Parametric Model	123	(1)
Muscle-Based Models	124	(4)
Finite Element Method	128	(1)
Other Models	129	(1)
MPEG-4-Based Facial Animation	129	(7)
Tongue, Wrinkles and Other Features	136	(1)
Summary	137	(1)
Conclusion	137	(3)
Body Deformations	140	(21)
Amaury Aubel
Surface Models	140	(5)
Rigid Deformations	140	(1)
Local Surface Operators	141	(1)
Skinning	141	(1)
Contour Deformation	142	(2)
Deformations by Example	144	(1)
Volumetric Models	145	(3)
Implicit Surfaces	146	(1)
Collision Models	147	(1)
Multi-Layered Models	148	(10)
Skeleton Layer	148	(1)
Muscle Layer	148	(7)
Fat Layer	155	(1)
Skin Layer	155	(3)
Conclusion	158	(3)
Comparative Analysis	159	(1)
Depth of Simulation	159	(1)
Future Research Directions	160	(1)
Hair Simulation	161	(31)
Sunil Hadap
Introduction	161	(1)
Hair Shape Modeling	162	(2)
Hair Dynamics	164	(1)
Hair Rendering	165	(3)
Summary	168	(1)
Static Hair Shape Modeling Based on Fluid Flow	169	(9)
Hair Shape Model	170	(3)
Interactive Hair-Styler	173	(3)
Enhancing Realism	176	(2)
Modeling Hair Dynamics Based on Continuum Mechanics	178	(12)
Hair as Continuum	179	(3)
Single Hair Dynamics	182	(3)
Fluid Hair Model	185	(4)
Results	189	(1)
Conclusion	190	(2)
Cloth Simulation	192	(38)
Pascal Volino
Frederic Cordier
Introduction	192	(1)
Technology Summary	193	(5)
Historical Review	193	(1)
Cloth Mechanics	193	(5)
Mechanical Simulation of Cloth	198	(13)
Mechanical Modeling Schemes	199	(5)
A Precise Particle-Based Surface Representation	204	(3)
Numerical Integration	207	(4)
Collision Techniques	211	(9)
Principles of Collision Detection	212	(1)
Collision Detection for Cloth Simulation	213	(2)
Collisions on Polygonal Meshes	215	(1)
Collision Response Schemes	216	(4)
Enhancing Garments	220	(6)
Mesh Smoothing	220	(2)
Geometrical Wrinkles	222	(1)
Advanced Fabric Lighting	223	(1)
Toward Real-Time Garment Animation	224	(2)
Designing Garments	226	(3)
Garment Design Tools	227	(1)
Applications	228	(1)
Conclusion	229	(1)
Expressive Speech Animation and Facial Communication	230	(30)
Sumedha Kshirsagar
Arjan Egges
Stephane Garchery
Introduction	230	(1)
Background and Review	231	(1)
Facial Animation Design	231	(2)
Parameterization	233	(1)
High Level Control of Animation	234	(4)
Speech Animation	238	(7)
Using Text-to-Speech	238	(1)
Phoneme Extraction from Natural Speech	239	(3)
Co-articulation	242	(1)
Expression Blending	243	(1)
Enhancing Realism	244	(1)
Facial Motion Capture and Analysis	245	(7)
Data Analysis	245	(1)
Principal Component Analysis	246	(1)
Contribution of Principal Components	247	(3)
Nature of Analysis Data and the PCs	250	(1)
Expression Blending Using PCs	250	(2)
Facial Communication	252	(6)
Dialogue Generation	253	(2)
Natural Language Processing and Generation	255	(1)
Emotions	255	(1)
Personality and Mood	256	(2)
Putting It All Together	258	(1)
Conclusion	259	(1)
Behavioral Animation	260	(27)
Jean-Sebastien Monzani
Anthony Guye-Vuilleme
Etienne de Sevin
What Is Behavioral Animation?	260	(2)
Behavior	261	(1)
Autonomy	262	(1)
State-of-the-Art	262	(7)
Perception and Memory	262	(2)
Defining Behaviors	264	(3)
Interactions	267	(1)
Animation	268	(1)
Applications	268	(1)
An Architecture for Behavioral Animation	269	(8)
Separate the Body and the Brain	269	(1)
System Design	270	(3)
Animation: A Layered Approach	273	(3)
Intelligent Virtual Agent: Simulating Autonomous Behavior	276	(1)
Behavioral Animation and Social Agents	277	(2)
Case Study	279	(6)
Storytelling	279	(2)
A Mechanism of Motivated Action Selection	281	(4)
Conclusion	285	(2)
Body Gesture Recognition and Action Response	287	(16)
Luc Emering
Bruno Herbelin
Introduction: Reality vs Virtuality	287	(2)
State-of-the-Art	289	(1)
Involved Technology	289	(1)
Action Recognition	289	(9)
Recognition Methods	289	(2)
Model vs Data-Oriented	291	(2)
Recognizable Actions	293	(1)
Action-Analysis Levels	294	(1)
Action Specification	295	(2)
Action Recognition Algorithm	297	(1)
Case Studies	298	(2)
Interactive Fighting	300	(1)
Discussion	300	(3)
Sense of Touching	300	(1)
Reactivity	301	(1)
Objective and Subjective Views	301	(1)
Embodiment	301	(1)
System Performance	301	(1)
Delegate Sub-Tasks	301	(1)
Semantic Modulation	302	(1)
Event/Action Associations	302	(1)
Realistic Behavior	302	(1)
Verbal Response	302	(1)
Interaction with 3-D Objects	303	(20)
Marcello Kallmann
Introduction	303	(1)
Related Work	304	(3)
Object Functionality	305	(1)
Actor Animation	305	(2)
Smart Objects	307	(3)
Interaction Features	307	(2)
Interpreting Interaction Features	309	(1)
SOMOD	310	(4)
Object Properties	310	(1)
Interaction Information	310	(2)
Behaviors	312	(2)
Interacting with Smart Objects	314	(3)
Interpretation of Plans	314	(1)
Manipulation Actions	315	(2)
Case Studies	317	(4)
Opening a Drawer	318	(1)
Interaction of Multiple Actors	319	(1)
Complex Behaviors	320	(1)
Remaining Problems	321	(2)
Groups and Crowd Simulation	323	(30)
Soraia Raupp Musse
Branislav Ulicny
Amaury Aubel
Introduction	323	(1)
Structure of this Chapter	324	(1)
Related Work	324	(9)
Crowd Evacuation Simulators	325	(2)
Crowd Management Training Systems	327	(1)
Sociological Models	328	(1)
Computer Graphics	329	(1)
Classification of Crowd Methods	330	(3)
A Hierarchical Approach to Model Crowds	333	(5)
Hierarchic Model	333	(2)
Emergent Crowds	335	(3)
Crowd Visualization	338	(8)
Virtual Human Model	338	(1)
Crowd Creation	338	(1)
Level of Detail	339	(1)
Animated Impostors	339	(1)
Impostor Rendering and Texture Generation	340	(2)
Texture Refreshment Approach	342	(1)
Visibility Issue	343	(1)
Factored Impostor	344	(1)
Z-Buffer Corrected Impostor	345	(1)
Results	346	(1)
Case Studies	346	(5)
Programmed Crowds	346	(2)
Guided Crowds	348	(1)
Autonomous Crowds	349	(1)
Interactive Crowd in Panic Situation	350	(1)
Conclusion	351	(2)
Rendering of Skin and Clothes	353	(20)
Neeharika Adabala
Introduction	353	(1)
Rendering of Skin	354	(7)
Texturing	354	(2)
Illumination Models	356	(2)
Interaction with the Environment	358	(1)
Temporal Features	358	(2)
Summary of Skin Rendering Techniques	360	(1)
Rendering of Clothes	361	(11)
Modeling Color Variation from Images	362	(1)
Illumination Models	362	(2)
Representation of Milli-Geometry	364	(1)
Rendering with Micro/Milli Geometric Detail and BRDF for Woven Clothes	365	(5)
Macro-geometry and Interaction with the Environment	370	(2)
Conclusion	372	(1)
Standards for Virtual Humans	373	(19)
Stephane Garchery
Ronan Boulic
Tolga Capin
Prem Kalra
Introduction	373	(1)
The H-Anim Specification Scheme	373	(6)
The Need for a Standard Human Skeleton	373	(1)
H-Anim Skeleton Convention	374	(1)
Benefits and Limitations of the Standard Skeleton	375	(1)
Enforcing Motion Re-use	376	(3)
The MPEG-4 FBA Standard	379	(12)
MPEG-4 Body Animation Standard	380	(3)
MPEG-4 Face Animation Standard	383	(8)
What's Next?	391	(1)
Appendix A: Damped Least Square Pseudo-Inverse J+λ	392	(1)
Appendix B: H-Anim Joint and Segment Topology	393	(3)
Appendix C: Facial Animation Parameter Set	396	(4)
References	400	(37)
Index	437