9780471988632

Avatars in Networked Virtual Environments Tolga K. ?apin Computer Graphics Lab, EPFL, Lausanne, Switzerland Igor S. Pandzic MIRALab, University of Geneva, Switzerland Nadia Magnenat-Thalmann MIRALab, University of Geneva, Switzerland Daniel Thalmann Computer Graphics Lab, EPFL, Lausanne, Switzerland Telepresence is the future of multimedia systems and will allow participants to share professional and private experiences, meetings, games and parties. Networked Virtual Environments (NVEs) are systems that allow distant users to interact in a common Virtual Environment. One of the areas of research in NVEs is the user representation or 'avatar', the way that participants are geographically represented in Virtual Environments. * Examines solutions to the Virtual Life Network (VLNET) system, a flexible framework for Virtual Humans in NVEs * Presents different means of facial communication, gesture and non-verbal communication * Provides details for solving all problems of handling Virtual Human data across the network * Analyses the potential relation of the future MPEG-4 standard to NVEs * Discusses the standardisation of Virtual Humans in VRML * Includes applications such as a Virtual tennis game and studies the experimental results and achievements. Essential reading for a wide-ranging audience including computer scientists, engineers in communication systems, undergraduate and postgraduate students of computer science, and researchers in computer animation, visualisation and communications technology.

PREFACE

1 CONCEPTS IN NETWORKED VIRTUAL ENVIRONMENTS

(14)

1.1 Virtual Environments

(1)

1.2 Networked Virtual Environments

(2)

1.3 Virtual Environments for Interaction

(1)

1.4 Virtual Humans in NVEs

(3)

1.5 Problem Domain

(1)

1.6 Immersion and Presence

(4)

1.7 Fidelities of NVEs

(1)

1.8 Human Communication

(1)

2 A TAXONOMY OF NETWORKED VIRTUAL ENVIRONMENTS

(44)

2.1 Preconditions of NVEs

(2)

2.1.1 Target Applications

(1)

2.1.2 Underlying Network

(1)

2.1.3 Projected Number and Properties of Connecting Hosts

(1)

2.1.4 Input Devices and Rendering Systems

(1)

2.2 Design Decisions for NVEs

(39)

2.2.1 Host Program Architecture

(16)

2.2.2 Data and Task Distribution Scheme

(9)

2.2.3 Network Topology

(5)

2.2.4 Avatar Representation

(5)

2.2.5 Protocol

(3)

2.2.6 Further Improvements of NVE Fidelities

(1)

2.3 Summary of Existing NVE Systems

(2)

3 THE VLNET SYSTEM

(40)

3.1 Overview and Preconditions

(4)

3.1.1 Precondition 1: Target Applications

(1)

3.1.2 Precondition 2: Underlying Network

(1)

3.1.3 Precondition 3: Projected Number of Connecting Hosts

(1)

3.1.4 Input Devices and Rendering Systems

(1)

3.2 Client Architecture: Multiprocess Integration

(27)

3.2.1 Overview of Heterogeneous Parallelism in VLNET

(2)

3.2.2 VLNET Client Architecture

(7)

3.2.3 Driver Simulation Computations

(2)

3.2.4 Driver Update to the External Interfaces

(5)

3.2.5 Engine Detection of Update in External Interfaces

(1)

3.2.6 Engine Internal Computations

(1)

3.2.7 Engine Update of the VLNET Output Queue

(2)

3.2.8 Communication Computations to Send VLNET Messages

(1)

3.2.9 Communication Receipt of VLNET Messages

(1)

3.2.10 Communication Update of Messages to VLNET Input Queue

(1)

3.2.11 Engine Reading of Related Messages from VLNET Input Queue

(1)

3.2.12 Engine Update of Internal Shared Memory

(1)

3.2.13 Cull and Display Processes

(1)

3.2.14 Dbase Process

(1)

3.2.15 Complexity of Phases within a Frame

(4)

3.2.16 Dependencies among Components

(1)

3.3 Other Design Decisions

(7)

3.3.1 Partitioning of the Virtual World: Rooms Connected by Portals

(1)

3.3.2 Data Distribution Scheme: Sharing of Data

(1)

3.3.3 Network Topology: Client-Server, Future Work on Multiple Servers

(3)

3.3.4 Participant Embodiments as Realistic Virtual Humans

(1)

3.3.5 VLNET Session Management and Communication Protocol

(4)

4 REPRESENTATION OF VIRTUAL HUMANS

(42)

4.1 Virtual Humans

(1)

4.2 Why Virtual Humans in NVE?

100

(1)

4.3 Architecture for Virtual Humans in NVEs

101

(1)

4.4 Design Choices for Representation

102

(7)

4.4.1 Articulated Structure for Virtual Human Modelling

102

(3)

4.4.2 Scalability for Virtual Human Representation

105

(1)

4.4.3 Computational Scalability

106

(1)

4.4.4 Graphics Scalability

107

(2)

4.5 Design Choices for Control

109

(4)

4.5.1 Types of Virtual Human Control

109

(4)

4.6 Motion Control Implementation

113

(17)

4.6.1 Implementation of Human Animation Drivers

115

(11)

4.6.2 Combination of Drivers

126

(4)

4.7 Autonomous and Interactive Perceptive Actors

130

(11)

4.7.1 Perception and Action

130

(1)

4.7.2 Virtual Vision

131

(1)

4.7.3 Virtual Audition

132

(1)

4.7.4 Virtual Haptic

132

(1)

4.7.5 Interactive Perceptive Actors

133

(1)

4.7.6 Intercommunication between Synthetic Actors

133

(1)

4.7.7 Sensing the Real World for a Synthetic Actor

134

(2)

4.7.8 Towards `Intelligent' Actors

136

(1)

4.7.9 Interfacing Autonomous Virtual Humans with the NVE

137

(4)

5 FACIAL AND GESTURAL COMMUNICATION

141

(28)

5.1 Introduction

141

(1)

5.2 Facial Animation Using MPAs

142

(3)

5.3 Video Texturing of the Face

145

(2)

5.4 Model-Based Coding of Facial Expressions

147

(8)

5.4.1 Initialization

148

(1)

5.4.2 Head Tracking

149

(2)

5.4.3 Jaw Rotation

151

(1)

5.4.4 Data Verification

151

(1)

5.4.5 Eyebrows

152

(1)

5.4.6 Eyes

152

(1)

5.4.7 Nose and Mouth

153

(2)

5.5 Lip Movement Synthesis from Speech

155

(1)

5.6 Predefined Expressions or Animations

156

(1)

5.7 Avatars and Non-Verbal Communication

156

(6)

5.7.1 The Field of Non-Verbal Communication in Social Sciences

156

(3)

5.7.2 Introducing Social Science Theories into NVE Systems

159

(3)

5.8 Non-Verbal Communication with 2D Interface

162

(4)

5.9 Non-Verbal Communication with Chat Interface

166

(1)

5.10 Autonomous Actor Control with Chat Interface

167

(2)

6 NETWORKING DATA FOR VIRTUAL HUMANS

169

(32)

6.1 Introduction

169

(3)

6.2 Elements of Networking Scalability

172

(1)

6.3 Loading Scalability

173

(3)

6.3.1 Asynchronous Loading

173

(2)

6.3.2 Progressive Downloading

175

(1)

6.4 Transformation Scalability

176

(2)

6.5 Comparison of Representation Types

178

(3)

6.6 Bandwidth Scalability

181

(1)

6.7 Lossless Compression of Posture Data

182

(5)

6.7.1 Joint Grouping

182

(2)

6.7.2 Downloadable Drivers

184

(3)

6.8 Lossy Compression of Posture Data

187

(2)

6.8.1 Multiple Levels of Resolution for Joint Angles

187

(1)

6.8.2 Transmission of a Subset of Joints

188

(1)

6.9 Data Compression Tools

189

(2)

6.10 Dead Reckoning

191

(6)

6.10.1 Dead Reckoning for Virtual Human Figures

192

(1)

6.10.2 Kalman Filtering

193

(4)

6.10.3 Dead-Reckoning Algorithm for Virtual Body

197

(1)

6.11 Filtering

197

(4)

7 STANDARDS FOR NVEs AND VIRTUAL HUMANS

201

(16)

7.1 MPEG-4 Face and Body Animation Specification

201

(6)

7.1.1 Introduction to MPEG-4

201

(2)

7.1.2 Face and Body Animation (FBA)

203

(4)

7.2 VRML 2.0 HAnim Specification

207

(5)

7.3 Bitstream Contents in NVE Applications

212

(2)

7.3.1 Download

212

(1)

7.3.2 State Updates

213

(1)

7.3.3 Events

213

(1)

7.3.4 System Messages

213

(1)

7.3.5 Video

213

(1)

7.3.6 Audio

213

(1)

7.3.7 Text

214

(1)

7.4 How MPEG-4 Can Meet NVE Requirements

214

(2)

7.4.1 Download

214

(1)

7.4.2 State Updates

214

(1)

7.4.3 Events and System Messages

215

(1)

7.4.4 Video

215

(1)

7.4.5 Audio

215

(1)

7.4.6 Text

216

(1)

7.4.7 Integration

216

(1)

7.5 Concluding Remarks

216

(1)

8 APPLICATIONS AND EXPERIMENTS

217

(34)

8.1 Potential Applications

217

(1)

8.2 Example Applications and Programming Results

217

(9)

8.3 Evaluation of Virtual Human Embodiment in NVEs

226

(5)

8.4 Performance of the System

231

(7)

8.4.1 Display Rate for Different Resolutions

231

(2)

8.4.2 Computations at the Body Representation Engine

233

(2)

8.4.3 Computations at the Body Driver

235

(2)

8.4.4 Performance of the Architecture

237

(1)

8.5 Networking Results

238

(13)

8.5.1 Loading Results

238

(1)

8.5.2 Traffic Measurement during Interaction

239

(3)

8.5.3 Analysis of the Network Results

242

(1)

8.5.4 Downloadable Drivers

243

(1)

8.5.5 Compression Results

244

(1)

8.5.6 Dead-Reckoning Results

245

(6)

9. CONCLUSION

251

(2)

BIBLIOGRAPHY

253

(12)

INDEX

265

Amazon no longer offers textbook rentals. We do!

Amazon no longer offers textbook rentals. We do!

We're the #1 textbook rental company. Let us show you why.

Avatars in Networked Virtual Environments

0471988634

Summary

Table of Contents

Supplemental Materials

Rewards Program