Data Strategy

by Adelman, Sid; Moss, Larissa; Abai, Majid

• ISBN13: 9780321240996
• ISBN10: 0321240995
• Edition: 1st
• Format: Paperback
• Copyright: 2005-06-15
• Publisher: Addison-Wesley Professional

Summary

Delivers the most current thinking and best practices in developing data management systems with less risk and a higher success rate.

Author Biography

Sid Adelman is a principal in Sid Adelman & Associates, an organization specializing in planning and implementing data warehouses, in data warehouse and BI assessments, and in establishing effective data architectures and strategies. He is a regular speaker at the Data Warehouse Institute and IBM's DB2 and Data Warehouse Conference. Sid also speaks often at DAMA conferences. He chairs the "Ask the Experts" column on http://www.dmreview.com.

Sid is a founding member of the Business Intelligence Alliance. Its members include Colin White, Herb Edelstein, Larry English, David Foote, Douglas Hackney, Pieter Mimno, Neil Raden, and David Marco. Sid is also a frequent contributor to journals that focus on data warehouse and data-related topics. He co-authored Data Warehouse Project Management with Larissa Moss. He is the primary author of Impossible Data Warehouse Situations with Solutions from the Experts.

Sid can be reached at [e-mail]. His web site is http://www.sidadelman.com.

Larissa Moss is president of Method Focus Inc., a corporation specializing in enterprise information management. She frequently lectures at data warehouse and data quality conferences worldwide on the topics of data warehousing, business intelligence, and other enterprise architecture and data strategy topics, such as data integration, data modeling, data quality, and metadata. Larissa is a senior consultant of the Cutter Consortium and a member of Friends of NCR-Teradata and the IBM Gold Group. Her present and past associations also include membership in DAMA, part-time faculty member at the Extended University of California Polytechnic University Pomona, associate of the Relational Institute and the Codd & Date Consulting Group, and lecturer for TDWI, DCI, MIS Training Institute, and PESG.

Larissa has authored and co-authored numerous books, white papers, and articles on business intelligence, project management, information asset management, development methodologies, data quality, and organizational realignments. She can be reached at [e-mail]. Her web-site is http://www.methodfocus.com.

Majid Abai is President of Seena Technologies, a Santa Monica, California consultancy dedicated to delivery of holistic data and enterprise solutions to various organizations. Majid's two decades of IT experience have been primarily focused on solution architecture, data strategies, and business intelligence systems for organizations facing challenges with the management of massive amounts of data. Majid has developed and teaches a class in Business Intelligence at the University of California, Los Angeles (UCLA) and several other seminars and lectures for national and international corporations. He can be reached at [e-mail]. Seena Technologies website is http://www.seenatech.com.

© Copyright Pearson Education. All rights reserved.

Table of Contents

Acknowledgments	xix
About the Authors	xxi
Foreword	xxiii
Introduction	1	(22)
Current Status in Contemporary Organizations	1	(2)
Why a Data strategy Is Needed	3	(1)
Value of Data as an Organizational Asset	4	(1)
Vision and Goals of the Enterprise	4	(2)
Support of the IT Strategy	5	(1)
Components of a Data Strategy	6	(9)
Data Integration	6	(1)
Data Quality	7	(1)
Metadata	8	(1)
Data Modeling	9	(1)
Organizational Roles and Responsibilities	10	(1)
Performance and Measurement	10	(1)
Security and Privacy	11	(1)
DBMS Selection	12	(1)
Business Intelligence	13	(1)
Unstructured Data	14	(1)
Business Value of Data and ROI	14	(1)
How Will You Develop and Implement a Data Strategy?	15	(6)
Data Environment Assessment	16	(5)
References	21	(2)
Data Integration	23	(24)
Ineffective ``Silver-Bullet'' Technology Solutions	24	(5)
Enterprise Resource Planning (ERP)	24	(2)
Data Warehousing (DW)	26	(1)
Customer Relationship Management (CRM)	27	(1)
Enterprise Application Integration (EAI)	28	(1)
Gaining Management Support	29	(6)
Business Case for Data Integration	31	(4)
Integrating Business Data	35	(3)
Know Your Business Entities	35	(1)
Mergers and Acquisitions	36	(1)
Data Redundancy	37	(1)
Data Lineage	37	(1)
Multiple DBMSs and Their Impact	38	(1)
Deciding What Data Should Be Integrated	38	(4)
Data Integration Prioritization	39	(1)
Risks of Data Integration	40	(2)
Consolidation and Federation	42	(2)
Data Consolidation	42	(1)
Data Federation	42	(1)
Data Integration Strategy Capability Maturity Model	43	(1)
Getting Started	44	(1)
Conclusion	45	(1)
References	46	(1)
Data Quality	47	(26)
Current State of Data Quality	48	(1)
Recognizing Dirty Data	49	(2)
Data Quality Rules	51	(6)
Business Entity Rules	51	(2)
Business Attribute Rules	53	(1)
Data Dependency Rules	54	(1)
Data Validity Rules	55	(2)
Data Quality Improvement Practices	57	(3)
Data Profiling	57	(1)
Data Cleansing	58	(1)
Data Defect Prevention	59	(1)
Enterprise-Wide Data Quality Disciplines	60	(6)
Data Quality Maturity Levels	61	(1)
Standards and Guidelines	62	(1)
Development Methodology	63	(1)
Data Naming and Abbreviations	63	(1)
Metadata	63	(1)
Data Modeling	64	(1)
Data Quality	65	(1)
Testing	65	(1)
Reconciliation	65	(1)
Security	66	(1)
Data Quality Metrics	66	(1)
Enterprise Architecture	66	(3)
Data Quality Improvement Process	68	(1)
Business Sponsorship	69	(2)
Business Responsibility for Data Quality	70	(1)
Conclusion	71	(1)
References	71	(2)
Metadata	73	(26)
Why Metadata Is Critical to the Business	74	(4)
Metadata as the Keystone	74	(1)
Management Support for Metadata	75	(1)
Starting a Metadata Management Initiative	76	(2)
Metadata Categories	78	(4)
Business Metadata	79	(2)
Technical Metadata	81	(1)
Process Metadata	81	(1)
Usage Metadata	82	(1)
Metadata Sources	82	(2)
Metadata Repository	84	(4)
Buying a Metadata Repository Product	84	(1)
Building a Metadata Repository	85	(1)
Centralized Metadata Repository	85	(1)
Distributed Metadata Repository	86	(1)
XML-Enabled Metadata Repository	87	(1)
Developing a Metadata Repository	88	(5)
Justification	88	(1)
Planning	88	(1)
Analysis	89	(1)
Design	90	(1)
Construction	91	(1)
Deployment	92	(1)
Managed Metadata Environment	93	(4)
Metadata Sourcing	94	(1)
Metadata Integration	95	(1)
Metadata Management	95	(1)
Metadata Marts	96	(1)
Metadata Delivery	97	(1)
Communicating and Selling Metadata	97	(1)
Conclusion	97	(1)
References	98	(1)
Data Modeling	99	(34)
Origins of Data Modeling	100	(2)
Significance of Data Modeling	102	(3)
Logical Data Modeling Concepts	105	(4)
Process-Independence	105	(1)
Business-Focused Data Analysis	106	(2)
Data Integration (Single Version of Truth)	108	(1)
Data Quality	109	(1)
Enterprise Logical Data Model	109	(6)
Big-Bang Versus Incremental	109	(3)
Top-Down versus Bottom-Up	112	(3)
Physical Data Modeling Concepts	115	(2)
Process-Dependence	116	(1)
Database Design	117	(1)
Physical Data Modeling Techniques	117	(5)
Denormalization	117	(3)
Surrogate Keys	120	(1)
Indexing	121	(1)
Partitioning	121	(1)
Database Views	122	(1)
Dimensionality	122	(4)
Star Schema	124	(1)
Snowflake	125	(1)
Starflake	126	(1)
Factors that Influence the Physical Data Model	126	(4)
Guideline 1: High Degree of Normalization for Robustness	126	(1)
Guideline 2: Denormalization for Short-Term Solutions	127	(1)
Guideline 3: Usage of Views on Powerful Servers	127	(1)
Guideline 4: Usage of Views on Powerful RDBMS Software	127	(1)
Guideline 5: Cultural Influence on Database Design	128	(1)
Guideline 6: Modeling Expertise Affects Database Design	128	(1)
Guideline 7: User-Friendly Structures	129	(1)
Guideline 8: Metric Facts Determine Database Design	129	(1)
Guideline 9: When to Mimic Source Database Design	130	(1)
Conclusion	130	(1)
References	131	(2)
Organizational Roles and Responsibilities	133	(28)
Building the Teams Who Create and Maintain the Strategy	134	(1)
Resistance to Change	134	(1)
Existing Organization	134	(1)
Resistance to Standards	135	(1)
``Reasons'' for Resistance	135	(1)
Optimal Organizational Structures	135	(3)
Distributed Organizations	137	(1)
Outsourced Personnel	137	(1)
Training	138	(2)
Who Should Attend	138	(1)
Mindset	139	(1)
Choice of Class	139	(1)
Timing	140	(1)
Roles and Responsibilities	140	(8)
Data Strategist	140	(1)
Database Administrator	141	(1)
Data Administrator	142	(1)
Metadata Administrator	142	(1)
Data Quality Steward	143	(2)
Consultants and Contractors	145	(1)
Security Officer	145	(1)
Sharing Data	146	(1)
Strategic Data Architect	147	(1)
Technical Services	147	(1)
Data Ownership	148	(3)
Domains	148	(1)
Security and Privacy	148	(1)
Availability Requirements	149	(1)
Timeliness and Periodicity Requirements	149	(1)
Performance Requirements	150	(1)
Data Quality Requirements	150	(1)
Business Rules	151	(1)
Information Stewardship	151	(5)
Steward Deliverables	155	(1)
Key Skills and Competencies	155	(1)
Worst Practices	156	(2)
Agenda for Weekly Data Strategy Team Meeting	158	(1)
Conclusion	159	(2)
Performance	161	(44)
Performance Requirements	163	(1)
Service Level Agreements	164	(2)
Response Time	165	(1)
Capacity Planning: Performance Modeling	166	(2)
Capacity Planning: Benchmarks	168	(8)
Why Pursue a Benchmark?	168	(1)
Benchmark Team	169	(1)
Benefits of a Good Benchmark: Goals and Objectives	170	(1)
Problems with ``Standard'' Benchmarks	170	(1)
The Cost of Running a Benchmark	171	(1)
Identifying and Securing Data	171	(1)
Establishing Benchmark Criteria and Methodology	171	(3)
Evaluating and Measuring Results	174	(1)
Verifying and Reconciling Results	175	(1)
Communicating Results Effectively	175	(1)
Application Packages: Enterprise Resource Planning (ERPs)	176	(1)
Designing, Coding, and Implementing	177	(12)
Designing	178	(1)
Coding	179	(1)
Implementation	180	(1)
Design Reviews	180	(9)
Setting User Expectations	189	(1)
Monitoring (Measurement)	190	(5)
Conformance to Measures of Success	191	(1)
Types of Metrics	191	(2)
Responsibility for Measurement	193	(1)
Means to Measure	193	(1)
Use of Measurements	193	(1)
Return on Investment (ROI)	194	(1)
Reporting Results to Management	194	(1)
Tuning	195	(3)
Tuning Options	196	(1)
Reporting Performance Results	197	(1)
Selling Management on Performance	198	(1)
Case Studies	198	(3)
Performance Tasks	201	(1)
Conclusion	202	(1)
References	202	(3)
Security and Privacy of Data	205	(18)
Data Identification for Security and Privacy	206	(2)
User Role	207	(1)
Roles and Responsibilities	208	(2)
Security Officer	208	(1)
Data Owner	209	(1)
System Administrator	209	(1)
Regulatory Compliance	210	(1)
Auditing Procedures	211	(2)
Security Audits	211	(1)
External Users of Your Data	212	(1)
Design Solutions	213	(2)
Database Controls	213	(1)
Security Databases	213	(1)
Test and Production Data	213	(1)
Data Encryption	214	(1)
Standards for Data Usage	214	(1)
Impact of the Data Warehouse	215	(1)
Vendor Issues	215	(2)
Software	215	(1)
External Data	216	(1)
Communicating and Selling Security	217	(1)
Security and Privacy Indoctrination	217	(1)
Monitoring Employees	217	(1)
Training	217	(1)
Communication	218	(1)
Best Practices and Worst Practices	218	(1)
Identify Your Own Sensitive Data Exercise	219	(1)
Conclusion	220	(3)
DBMS Selection	223	(36)
Existing Environment	223	(3)
Capabilities and Functions	224	(2)
DBMS Choices	226	(1)
Why Standardize the DBMS?	227	(1)
Integration Problems	227	(1)
Greater Staff Expense	228	(1)
Software Expense	228	(1)
Total Cost of Ownership	228	(4)
Hardware	230	(1)
Network Usage	230	(1)
DBMS	230	(1)
Consultants and Contractors	231	(1)
Internal Staff	231	(1)
Help Desk Support	231	(1)
Operations and System Administration	232	(1)
IT Training	232	(1)
Application Packages and ERPs	232	(1)
Criteria for Selection	233	(1)
Selection Process	234	(2)
Reference Checking	236	(6)
Alternatives to Reference Checking	236	(1)
Selecting and Gathering References	237	(1)
Desired Types of References	237	(1)
The Process of Reference Checking	238	(1)
Questions to Ask	239	(3)
RFPs for DBMSs	242	(4)
RFP Best Practices	242	(4)
Response Format	246	(1)
Evaluating Vendors	246	(1)
Dealing with the Vendor	247	(8)
Performance	249	(1)
Vendor's Level of Service	250	(1)
Early Code	250	(1)
Rules of Engagement	250	(2)
Set the Agenda for Meetings and Presentations	252	(1)
Professional Employee Information	253	(1)
Financial Information	254	(1)
Selection Matrix-----Categorize Capabilities and Functions	254	(1)
Exercise---How Well Are You Using Your DBMS?	255	(2)
Conclusion	257	(1)
References	257	(2)
Business Intelligence	259	(18)
What Is Business Intelligence?	260	(4)
A Brief History	261	(1)
Importance of BI	262	(2)
BI Components	264	(3)
Data Warehouse	264	(1)
Metadata Repository	265	(1)
Data Transformation and Cleansing	265	(1)
OLAP and Analytics	266	(1)
Data Presentation and Visualization	267	(1)
Important BI Tools and Processes	267	(2)
Data Mining	267	(1)
Rule-Based Analytics	268	(1)
Balanced Scorecard	268	(1)
Digital Dashboard	269	(1)
Emerging Trends and Technologies	269	(3)
Mining Structured and Unstructured Data	270	(1)
Radio Frequency Identification	271	(1)
BI Myths and Pitfalls	272	(2)
Conclusion	274	(1)
References	274	(3)
Strategies for Managing Unstructured Data	277	(14)
What Is Unstructured Data?	278	(4)
A Brief History	278	(2)
Why Now?	280	(2)
Current State of Unstructured Data in Organizations	282	(1)
A Unified Content Strategy for the Organization	282	(5)
Definition of a Unified Content Strategy	282	(1)
Storage and Administration	283	(3)
Content Reusability	286	(1)
Search and Delivery	286	(1)
Combining Structured and Unstructured Data	287	(1)
Emerging Technologies	287	(3)
Digital Asset Management Software	287	(1)
Digital Rights Management Software	288	(2)
Electronic Medical Records	290	(1)
Conclusion	290	(1)
References	290	(1)
Business Value of Data and ROI	291	(18)
The Business Value of Data	291	(4)
Companies that Sell Customer Data	292	(1)
Internal Information Gathered About Customers	292	(1)
Call Center Data	292	(1)
Click-Stream Data	293	(1)
Demographics	293	(1)
Channel Preferences	293	(1)
Direct Retailers	294	(1)
Loyalty Cards	294	(1)
Travel Data	294	(1)
Align Data with Strategic Goals	295	(1)
ROI Process	295	(1)
The Cost of Developing a Data Strategy	295	(5)
Data Warehouse	296	(1)
Hardware	297	(1)
Software	297	(1)
Personnel Costs	298	(1)
Training	299	(1)
Operations and System Administration	299	(1)
Total Cost of Ownership	299	(1)
Benefits of a Data Strategy	300	(8)
The Data Warehouse	301	(1)
Estimating Tangible Benefits	301	(4)
Estimating Intangible Benefits	305	(2)
Post-Implementation Benefits Measurement	307	(1)
Conclusion	308	(1)
Reference	308	(1)
Appendix A: ROI Calculation Process, Cost Template, and Intangible Benefits Template	309	(8)
Cost of Capital	309	(1)
Risk	310	(1)
ROI Example	310	(2)
Net Present Value	311	(1)
Internal Rate of Return	311	(1)
Payback Period	312	(1)
Cost Calculation Template	312	(3)
Intangible Benefits Calculation Template	315	(1)
Reference	315	(2)
Appendix B: Resources	317	(6)
Publications	317	(3)
Websites	320	(3)
Index	323

Excerpts

Foreword Data strategy is one of the most ubiquitous and misunderstood topics in the information technology (IT) industry. Most corporations' data strategy and IT infrastructure were not planned, but grew out of "stovepipe" applications over time with little to no regard for the goals and objectives of the enterprise. This stovepipe approach has produced the highly convoluted and inflexible IT architectures so prevalent in corporations today. This architecture--or rather the lack thereof--creates a significant stumbling block because it is exceedingly time consuming and costly to modify existing systems. In fact, I have seen situations in which a chief marketing officer could not initiate a desired marketing campaign because the opportunity to do so would have grown stale by the time the systems were modified to implement the new campaign. Besides inflexibility, the lack of enterprise IT planning has lead to epidemic levels of data redundancy. In my experience, most major corporations and large government organizations have three- to four-fold "needless data redundancy"--data that exists for no other reason than failure to properly plan and implement. This issue has become so pressing that it has entered into the chief executive officer (CEO)'s key corporate objectives. I have personally witnessed several CEOs declare that their organization must simplify its IT portfolio, so that redundant data and applications can be removed. Many organizations target enterprise data strategy as one of the key initiatives to reduce data redundancy, simplify IT portfolios, and ease the strain on the architectures of applications. Through metadata management, an enterprise data strategy identifies how data should be constructed, what data exists, and what the meaning of that data is. This helps organizations address data redundancy by showing when a proposed new system will replicate existing applications. This is a critical aspect of data strategy because many companies want to consolidate existing redundant applications, but processes are not in place to prevent new redundancy from entering the IT environment. Thus, an effective enterprise data strategy can save organizations that currently operate as the proverbial sinking ships whose crews are bailing water, but cannot plug the leaks. A sound enterprise data strategy not only "bails water" by affording IT staff the means and methods for reducing existing redundant data, but it can "plug the leaks" by ensuring that new redundancies stop flowing into the organization. Sid Adelman, Larissa Moss, and Majid Abai's book represents an outstanding achievement in defining the key activities for implementing a successful enterprise data strategy. Their real-world experience assisting companies shines throughout the book and makes it a must read for any IT professional. --David Marco, President of EWSolutions Copyright Pearson Education. All rights reserved.