In-Memory Data Management : An Inflection Point for Enterprise Applications

by Plattner, Hasso; Zeier, Alexander

• ISBN13: 9783642193620
• ISBN10: 3642193625
• Format: Hardcover
• Copyright: 2011-06-23
• Publisher: Springer-Verlag New York Inc

Summary

In the last fifty years the world has been completely transformed through the use of IT. We have now reached a new inflection point. Here we present, for the first time, how in-memory computing is changing the way businesses are run. Today, enterprise data is split into separate databases for performance reasons. Analytical data resides in warehouses, synchronized periodically with transactional systems. This separation makes flexible, real-time reporting on current data impossible. Multi-core CPUs, large main memories, cloud computing and powerful mobile devices are serving as the foundation for the transition of enterprises away from this restrictive model. We describe techniques that allow analytical and transactional processing at the speed of thought and enable new ways of doing business. The book is intended for university students, IT-professionals and IT-managers, but also for senior management who wish to create new business processes by leveraging in-memory computing.

Table of Contents

Foreword	p. XI
Preface	p. XIII
Introduction	p. 1
An Inflection Point for Enterprise Applications	p. 5
Desirability, Feasibility, Viability - The Impact of In-Memory	p. 7
Information in Real Time - Anything, Anytime, Anywhere	p. 7
Response Time at the Speed of Thought	p. 9
Real-Time Analytics and Computation on the Fly	p. 10
The Impact of Recent Hardware Trends	p. 11
Database Management Systems for Enterprise Applications	p. 11
Main Memory Is the New Disk	p. 14
From Maximizing CPU Speed to Multi-Core Processors	p. 15
Increased Bandwidth between CPU and Main Memory	p. 17
Reducing Cost through In-Memory Data Management	p. 20
Total Cost of Ownership	p. 20
Cost Factors in Enterprise Systems	p. 21
In-Memory Performance Boosts Cost Reduction	p. 22
Conclusion	p. 23
Why Are Enterprise Applications So Diverse?	p. 25
Current Enterprise Applications	p. 25
Examples of Enterprise Applications	p. 27
Enterprise Application Architecture	p. 29
Data Processing in Enterprise Applications	p. 30
Data Access Patterns in Enterprise Applications	p. 31
Conclusion	p. 31
SanssouciDB - Blueprint for an In-Memory Enterprise Database System	p. 33
Targeting Multi-Core and Main Memory	p. 34
Designing an In-Memory Database System	p. 36
Organizing and Accessing Data in Main Memory	p. 37
Conclusion	p. 40
SanssouciDB - A Single Source of Truth through In-Memory	p. 41
The Technical Foundations of SanssouciDB	p. 43
Understanding Memory Hierarchies	p. 43
Introduction to Main Memory	p. 44
Organization of the Memory Hierarchy	p. 47
Trends in Memory Hierarchies	p. 49
Memory from a Programmer's Point of View	p. 50
Parallel Data Processing Using Multi-Core and Across Servers	p. 57
Increasing Capacity by Adding Resources	p. 57
Parallel System Architectures	p. 59
Parallelization in Databases for Enterprise Applications	p. 61
Parallel Data Processing in SanssouciDB	p. 64
Compression for Speed and Memory Consumption	p. 68
Light-Weight Compression	p. 69
Heavy-Weight Compression	p. 73
Data-Dependent Optimization	p. 73
Compression-Aware Query Execution	p. 73
Compression Analysis on Real Data	p. 74
Column, Row, Hybrid - Optimizing the Data Layout	p. 75
Vertical Partitioning	p. 75
Finding the Best Layout	p. 78
Challenges for Hybrid Databases	p. 81
The Impact of Virtualization	p. 81
Virtualizing Analytical Workloads	p. 82
Data Model and Benchmarking Environment	p. 82
Virtual versus Native Execution	p. 83
Response Time Degradation with Concurrent VMs	p. 84
Conclusion	p. 86
Organizing and Accessing Data in SanssouciDB	p. 89
SQL for Accessing In-Memory Data	p. 90
The Role of SQL	p. 90
The Lifecycle of a Query	p. 91
Stored Procedures	p. 91
Data Organization and Indices	p. 91
Increasing Performance with Data Aging	p. 92
Active and Passive Data	p. 93
Implementation Considerations for an Aging Process	p. 95
The Use Case for Horizontal Partitioning of Leads	p. 95
Efficient Retrieval of Business Objects	p. 98
Retrieving Business Data from a Database	p. 98
Object Data Guide	p. 99
Handling Data Changes in Read-Optimized Databases	p. 100
The Impact on SanssouciDB	p. 101
The Merge Process	p. 103
Improving Performance with Single Column Merge	p. 107
Append, Never Delete, to Keep the History Complete	p. 109
Insert-Only Implementation Strategies	p. 110
Minimizing Locking through Insert-Only	p. 111
The Impact on Enterprise Applications	p. 114
Feasibility of the Insert-Only Approach	p. 117
Enabling Analytics on Transactional Data	p. 118
Aggregation on the Fly	p. 119
Analytical Queries without a Star Schema	p. 128
Extending Data Layout without Downtime	p. 135
Reorganization in a Row Store	p. 135
On-The-Fly Addition in a Column Store	p. 136
Business Resilience through Advanced Logging Techniques	p. 137
Recovery in Column Stores	p. 138
Differential Logging for Row-Oriented Databases	p. 140
Providing High Availability	p. 141
The Importance of Optimal Scheduling for Mixed Workloads	p. 142
Introduction to Scheduling	p. 142
Characteristics of a Mixed Workload	p. 145
Scheduling Short and Long Running Tasks	p. 146
Conclusion	p. 148
How In-Memory Changes the Game	p. 151
Application Development	p. 153
Optimizing Application Development for SanssouciDB	p. 153
Application Architecture	p. 154
Moving Business Logic into the Database	p. 155
Best Practices	p. 157
Innovative Enterprise Applications	p. 158
New Analytical Applications	p. 158
Operational Processing to Simplify Daily Business	p. 162
Information at Your Fingertips with Innovative User-Interfaces	p. 164
Conclusion	p. 169
Finally, a Real Business Intelligence System Is at Hand	p. 171
Analytics on Operational Data	p. 171
Yesterday's Business Intelligence	p. 171
Today's Business Intelligence	p. 174
Drawbacks of Separating Analytics from Daily Operations	p. 176
Dedicated Database Designs for Analytical Systems	p. 178
Analytics and Query Languages	p. 180
Enablers for Changing Business Intelligence	p. 182
Tomorrow's Business Intelligence	p. 183
How to Evaluate Databases after the Game Has Changed	p. 185
Benchmarks in Enterprise Computing	p. 185
Changed Benchmark Requirements for a Mixed Workload	p. 187
A New Benchmark for Daily Operations and Analytics	p. 188
Conclusion	p. 192
Scaling SanssouciDB in the Cloud	p. 193
What Is Cloud Computing?	p. 194
Types of Cloud Applications	p. 195
Cloud Computing from the Provider Perspective	p. 197
Multi-Tenancy	p. 197
Low-End versus High-End Hardware	p. 201
Replication	p. 201
Energy Efficiency by Employing In-Memory Technology	p. 202
Conclusion	p. 204
The In-Memory Revolution Has Begun	p. 205
Risk-Free Transition to In-Memory Data Management	p. 205
Operating In-Memory and Traditional Systems Side by Side	p. 206
System Consolidation and Extensibility	p. 207
Customer Proof Points	p. 208
Conclusion	p. 209
References	p. 211
About the Authors	p. 221
Glossary	p. 223
Abbreviations	p. 231
Index	p. 233
Table of Contents provided by Ingram. All Rights Reserved.