Building Tablet PC Applications

Many pen-based software platforms have been introduced in the last decade, but only recently has the introduction of faster processors and hardware enabled mainstream introduction of tablet-sized pen-based PCs. This book tackles the challenge of writing compelling Tablet PC software on two fronts. First, it distills decades of usability research into pen-based computing to present an authoritative discussion of the optimal design of pen-based user interfaces. Second, it provides an in-depth exposition of the Tablet PC Ink SDK, complete with tips and tricks on how developers can produce the most powerful and natural "ink-aware" applications.

Philip Su is a lead developer at Microsoft in the Tablet PC group, working on integration of the Tablet PC platform into Microsoft Journal software. Previously, he worked as a lead developer on Microsoft Money and as a software design engineer on Microsoft Word. Philip is also the founder and CEO of Sonetics Software, LLC, a company that specializes in digital audio software. Philip has won numerous awards, including the prestigious National Science Foundation Graduate Fellowship as well as the Maryland Technical Invention of the Year Award. His areas of interest include discrete digital audio processing, multimodal user interfaces, and digital media. He holds dual B.S. degrees in computer science and neurobiology, graduating summa cum laude from University of Maryland.

Foreword

xi

Acknowledgments

xv

Introduction

xvii

Part I The Tablet PC and Its Applications

Tablet Computing Comes of Age

3

(22)

What Makes a Tablet Computer

4

(5)

Form Factor

4

(1)

Pen Input

5

(1)

Stand-Alone and General-Purpose

5

(1)

What Isn't a Tablet Computer

6

(1)

The Role of Tablet Computers

7

(2)

A Brief History of Tablet Computing

9

(9)

GRiD GRiDPad

10

(1)

GO PenPoint

11

(1)

Microsoft Windows for Pen Computing

12

(1)

Apple Newton

13

(2)

Palm Computing PalmPilot

15

(1)

Microsoft Pocket PC

16

(1)

In a Little While

17

(1)

The Microsoft Tablet PC

18

(7)

The Birth of Microsoft Tablet PC

19

(1)

Microsoft Tablet PC Hardware Guidelines

20

(1)

Microsoft Tablet PC Software

20

(2)

Could This Be the One?

22

(3)

Designing Tablet PC Applications

25

(40)

User Research

25

(7)

Tablet Usability

26

(1)

Application Usability

27

(4)

Pen Usability

31

(1)

Of Mice and Pen

32

(1)

Tablet Displays

32

(1)

Digitizer Technology

33

(13)

Digitizer Distortion

37

(2)

Parallax

39

(2)

Still Motion

41

(3)

Handedness

44

(2)

Thinking in Ink

46

(19)

Ink Modeling

47

(3)

Ink Interaction

50

(7)

Ink Realism

57

(8)

Part II The Tablet PC Platform SDK

Introduction to the Tablet PC Platform SDK

65

(30)

A Sort of Homecoming

65

(7)

Finding the Right Operating System for the Job

68

(1)

Managed APIs

69

(1)

Ink Controls

70

(1)

COM Automation APIs

70

(1)

Sample Applications

71

(1)

Installing the Tablet PC Platform SDK

72

(7)

System Requirements

72

(1)

Getting the SDK

73

(1)

Setting Up Your Environment

73

(5)

Anatomy of the Tablet PC Platform SDK

78

(1)

Overview of the Managed API

79

(13)

All That You Can't Leave Behind

81

(1)

Design Goals of the API

82

(5)

Managed API Object Survey

87

(3)

Ink Data Management API

90

(1)

Ink Recognition API

91

(1)

Ink Control Comparison with Managed API

92

(2)

Welcome to the Great Adventure

94

(1)

Tablet PC Platform SDK: Tablet Input

95

(80)

Sample Applications

95

(1)

Capturing Input from the Pen

96

(2)

Requirement #1---Mouse Emulation

96

(1)

Requirement #2---Digital Ink

97

(1)

Requirement #3---Pen-Based Actions

98

(1)

Summing Up the Requirements

98

(1)

Anatomy of the Tablet PC's Tablet Input Subsystem

98

(11)

Tablet Hardware

100

(1)

Chock-full of HID-y Goodness

101

(1)

The Center of the TIS Universe: Wisptis.exe

102

(6)

Winlogon Desktop Support

108

(1)

What About Ink?

109

(1)

Platform SDK Support for Tablet Input

109

(54)

Getting Ink from a Tablet

109

(3)

When Ink Is Not Enough

112

(9)

InkCollector Events

121

(21)

InkOverlay Events

142

(1)

Specifying the Tablet Data to Capture---Packet Properties

142

(13)

Extending InkOverlay Behaviors

155

(1)

Sample Application: TopOfPenErase

155

(5)

Sample Application: ModelessSwitch

160

(3)

Getting Introspective

163

(9)

Tablets Collection

163

(1)

Tablet Class

163

(7)

Common Properties on InkCollector and InkOverlay

170

(2)

Best Practices for InkCollector and InkOverlay

172

(3)

Tablet PC Platform SDK: Ink Data Management, Part I

175

(70)

Ink and Stroke Objects

176

(32)

Introduction to the Ink, Stroke, and Strokes Classes

178

(8)

Using Strokes Collections

186

(7)

Creation, Deletion, and Ownership of Stroke Objects

193

(15)

Rendering Digital Ink

208

(32)

Renderer Class

208

(17)

Adding Style---The DrawingAttributes Class

225

(15)

Special Rendering Effects

240

(5)

Tablet PC Platform SDK: Ink Data Management, Part II

245

(102)

Stroke Geometry

245

(36)

Computing the Bounding Box of a Stroke

246

(2)

Retrieving the Points of a Stroke

248

(3)

Computing Intersections of a Stroke

251

(7)

Retrieving and Setting the Packet Data of a Stroke

258

(2)

Retrieving the Cusps of a Stroke

260

(3)

Putting It Together---the StrokeDataViewer Example

263

(13)

Transforming Strokes

276

(5)

Targeting and Hit-Testing Ink Strokes

281

(30)

Different Types of Hit-Testing

281

(3)

Hit-Testing Functions

284

(27)

Splitting and Trimming Ink

311

(14)

Splitting Strokes

311

(1)

Clipping/Trimming Strokes

312

(13)

Serialization, the Clipboard, and Drag and Drop

325

(16)

Serialization

325

(9)

Using the Clipboard

334

(7)

Implementing Drag and Drop

341

(6)

Tablet PC Platform SDK: Ink Recognition

347

(74)

Recognizer Architecture

348

(3)

Text vs. Object Recognition

348

(1)

Synchronous vs. Asynchronous Recognition

349

(1)

Recognition Results

350

(1)

Performing Simple Recognition

351

(16)

Recognizing Text

351

(3)

Recognizing Application Gestures

354

(13)

Using the Recognition Classes

367

(54)

Obtaining a Recognizer to Use

367

(1)

Initiating a Recognition Session

368

(3)

Supplying Strokes to the Recognizer

371

(1)

Getting Results I: Easy Synchronous Recognition

372

(4)

Getting Results II: Electric Boogaloo (a.k.a. Harder Synchronous Recognition)

376

(8)

Getting Results III: The Final Chapter (a.k.a. Asynchronous Recognition)

384

(7)

Working with Recognition Results

391

(21)

Storing Recognition Results

412

(1)

Recognition Properties

413

(2)

Improving Recognition Results

415

(6)

Ink Controls

421

(38)

Achtung Baby

421

(1)

InkEdit

422

(24)

InkEdit Basics

423

(10)

Working with Ink

433

(2)

Recognizing Ink and Gestures

435

(9)

InkEdit Parting Thoughts

444

(2)

InkPicture

446

(13)

Part III Advanced Tablet PC Topics

Updating Existing Applications

459

(56)

Even Better than the Real Thing

459

(4)

User Benefits of Integration

460

(2)

Business Benefits of Integration

462

(1)

Technical Considerations

463

(28)

Application Design

463

(6)

Performance

469

(22)

InkPadJunior

491

(10)

Part IV Appendixes

A BuildingTabletApps Library Reference

501

(10)

B Tablet PC Hardware Guidelines

511

(4)

Index

515

What is included with this book?

The New copy of this book will include any supplemental materials advertised. Please check the title of the book to determine if it should include any access cards, study guides, lab manuals, CDs, etc.

The Used, Rental and eBook copies of this book are not guaranteed to include any supplemental materials. Typically, only the book itself is included. This is true even if the title states it includes any access cards, study guides, lab manuals, CDs, etc.