Big Data Governance An Emerging Imperative

by Soares, Sunil

• ISBN13: 9781583473771
• ISBN10: 1583473777
• Format: Paperback
• Copyright: 2013-01-01
• Publisher: MC Press

Summary

Written by a leading expert in the field, this account focuses on the convergence of two major trends in information managementbig data and information governanceby taking a strategic approach oriented around business cases and industry imperatives. With the advent of new technologies, enterprises are expanding and handling very large volumes of data; this book, nontechnical in nature and geared toward business audiences, encourages the practice of establishing appropriate governance over big data initiatives and addresses how to manage and govern big data, highlighting the relevant processes, procedures, and policies. It teaches readers to understand how big data fits within an overall information governance program; quantify the business value of big data; apply information governance concepts such as stewardship, metadata, and organization structures to big data; appreciate the wide-ranging business benefits for various industries and job functions; sell the value of big data governance to businesses; and establish step-by-step processes to implement big data governance.

Author Biography

Sunil Soares is the director of information governance for IBM Software Group and a former director of Worldwide Channels and Alliances for InfoSphere, IBM Software Group. He is the author of The IBM Governance Unified Process and Selling Information Governance to the Business. He lives in Harrington Park, New Jersey.

Table of Contents

Foreword	p. xvii
Foreword	p. xix
Preface	p. xxi
Getting Started	p. 1
An Introduction to Big Data Governance	p. 3
The Big Data Governance Framework	p. 9
Big Data Types	p. 10
Information Governance Disciplines	p. 12
Industry and Functional Scenarios for Big Data Governance	p. 15
Summary	p. 27
The Maturity Assessment	p. 29
The IBM Information Governance Council Maturity Model	p. 29
Sample Questions to Assess Maturity	p. 31
Summary	p. 36
The Business Case	p. 37
Improve On-Time Performance and Passenger Safety Through Big Data Governance	p. 37
Quantify the Financial Impact of Big Data Governance on Customer Privacy	p. 39
Reduce IT Costs by Governing the Lifecycle of Big Data	p. 40
Estimate the Impact of Data Quality and Master Data on Big Data Initiatives	p. 41
Summary	p. 42
The Roadmap	p. 43
The Roadmap Case Studies	p. 43
Summary	p. 46
Big Data Governance Disciplines	p. 47
Organizing for Big Data Governance	p. 49
Map Key Processes and Establish a RACI Matrix to Identify Stakeholders in Big Data Governance	p. 49
Determine the Appropriate Mix of New and Existing Roles for Information Governance	p. 54
Appoint Big Data Stewards as Appropriate	p. 55
Add Big Data Responsibilities to Traditional Information Governance Roles as Appropriate	p. 60
Establish a Merged Information Governance Organization with Responsibilities That Include Big Data	p. 63
Summary	p. 65
Metadata	p. 67
Establish a Glossary That Represents the Business Definitions for Key Big Data Terms	p. 68
Understand the Ongoing Support for Metadata Within Apache Hadoop	p. 71
Tag Sensitive Big Data Within the Business Glossary	p. 73
Import Technical Metadata from the Relevant Big Data Stores	p. 74
Link the Relevant Data Sources to the Terms in the Business Glossary	p. 74
Leverage Operational Metadata to Monitor the Movement of Big Data	p. 75
Maintain Technical Metadata to Support Data Lineage and Impact Analysis	p. 75
Gather Metadata from Unstructured Documents to Support Enterprise Search	p. 77
Extend Existing Metadata Roles to Include Big Data	p. 77
Summary	p. 78
Big Data Privacy	p. 79
Identify Sensitive Big Data	p. 84
Flag Sensitive Big Data Within the Metadata Repository	p. 86
Address Privacy Laws and Regulations by Country, State, and Province	p. 86
Manage Situations Where Personal Data Crosses International Boundaries	p. 96
Monitor Access to Sensitive Big Data by Privileged Users	p. 98
Summary	p. 99
Big Data Quality	p. 101
Work with Business Stakeholders to Establish and Measure Confidence Intervals for the Quality of Big Data	p. 102
Leverage Semi-Structured and Unstructured Data to Improve the Quality of Sparsely Populated Structured Data	p. 107
Use Streaming Analytics to Address Data Quality Issues In-Memory Without Landing Interim Results to Disk	p. 107
Appoint Data Stewards Accountable to the Information Governance Council for Improving the Metrics Over Time	p. 111
Summary	p. 112
Business Process Integration	p. 113
Identify the Key Processes That Will Be Impacted by Big Data Governance	p. 114
Build a Process Map with Key Activities	p. 115
Map Big Data Governance Policies to the Key Steps in the Process	p. 116
Summary	p. 116
Master Data Integration	p. 117
Improve the Quality of Master Data to Support Big Data Analytics	p. 119
Leverage Big Data to Improve the Quality of Master Data	p. 121
Improve the Quality and Consistency of Key Reference Data to Support the Big Data Governance Program	p. 124
Consider Social Media Platform Policies to Determine the Level of Integration with Master Data Management	p. 125
Extract Meaning from Unstructured Text to Enrich Master Data	p. 126
Summary	p. 131
Managing the Lifecycle of Big Data	p. 133
Expand the Retention Schedule to Include Big Data Based on Local Regulations and Business Needs	p. 134
Document Legal Holds and Support eDiscovery Requests	p. 136
Compress and Archive Big Data to Reduce IT Costs and Improve Application Performance	p. 137
Manage the Lifecycle of Real-Time, Streaming Data	p. 138
Retain Social Media Records to Comply with Regulations and Support eDiscovery Requests	p. 139
Defensibly Dispose of Big Data No Longer Required Based on Regulations and Business Needs	p. 140
Summary	p. 140
The Governance of Big Data Types	p. 141
Web and Social Media	p. 143
Consider Evolving Regulations and Customs When Establishing Policies Regarding the Acceptable Use of Social Media Data About Customers	p. 145
Set Up Policies Regarding the Acceptable Use of Social Media Data About Employees and Job Candidates	p. 150
Leverage Confidence Intervals to Assess the Quality of Social Media Data	p. 152
Establish Policies Regarding the Acceptable Use of Cookies and Other Web Tracking Devices	p. 154
Define Policies to Link Online and Offline Data in a Way That Does Not Violate Privacy Concerns and Regulations	p. 162
Ensure the Consistency of Web Metrics	p. 165
Summary	p. 167
Machine-to-Machine Data	p. 169
Assess the Types of Geolocation Data Currently Available	p. 170
Establish Policies Regarding the Acceptable Use of Geolocation Data Pertaining to Customers	p. 172
Establish Policies Regarding the Acceptable Use of Geolocation Data Pertaining to Employees	p. 175
Ensure the Privacy of RFID Data	p. 176
Define Policies Relating to the Privacy of Other Types of M2M Data	p. 179
Address the Metadata and Quality of M2M Data	p. 181
Establish Policies Regarding the Retention Period for M2M Data	p. 184
Improve the Quality of Master Data to Support M2M Initiatives	p. 184
Secure the SCADA Infrastructure from Vulnerability to Cyber Attacks	p. 187
Summary	p. 192
Big Transaction Data	p. 193
Summary	p. 198
Biometrics	p. 199
Assess the Privacy Implications Relating to the Acceptable Use of Biometric Data	p. 200
Work with Legal Counsel to Determine the Impact of Evolving Regulations on the Use of Biometric Data for Customers and Employees	p. 202
Summary	p. 204
Human-Generated Data	p. 205
Establish Policies to Mask Sensitive Human-Generated Data	p. 206
Use Unstructured Human-Generated Data to Improve the Quality of Structured Data	p. 207
Manage the Lifecycle of Human-Generated Data to Reduce Costs and Comply with Regulations	p. 208
Extract Insights from Unstructured Human-Generated Data to Enrich MDM	p. 208
Summary	p. 209
Industry Perspectives	p. 211
Healthcare	p. 213
Leverage Unstructured Data to Improve the Quality of Sparsely Populated Structured Data	p. 214
Extract Additional Relevant Clinical Factors Not Available Within Structured Data	p. 215
Define Consistent Definitions for Key Business Terms	p. 216
Ensure Consistency in Patient Master Data Across Facilities	p. 216
Adhere to Privacy Requirements for Protected Health Information in Accordance with HIPAA	p. 216
Creatively Manage Reference Data to Yield Effective Clinical Insights	p. 217
Summary	p. 217
Utilities	p. 219
Duplicate Meter Readings	p. 222
Referential Integrity of the Primary Key	p. 222
Anomalous Meter Readings	p. 222
Data Quality for Customer Addresses	p. 223
Information Lifecycle Management	p. 223
Database Monitoring	p. 224
Technical Architecture	p. 224
Summary	p. 226
Communications Service Providers	p. 227
Big Data Types	p. 228
Integrating Big Data with Master Data	p. 229
Big Data Privacy	p. 231
Big Data Quality	p. 232
Big Data Lifecycle Management	p. 233
Summary	p. 233
Big Data Technology	p. 235
Big Data Reference Architecture	p. 237
Big Data Sources	p. 239
Open Source Foundational Components	p. 239
Hadoop Distributions	p. 241
Streaming Analytics	p. 242
Databases	p. 243
Big Data Integration	p. 244
Text Analytics	p. 246
Big Data Discovery	p. 247
Big Data Quality	p. 248
Metadata for Big Data	p. 249
Information Policy Management	p. 249
Master Data Management	p. 250
Data Warehouses and Data Marts	p. 251
Big Data Analytics and Reporting	p. 252
Big Data Security and Policy	p. 254
Big Data Lifecycle Management	p. 255
The Cloud	p. 258
Summary	p. 259
Big Data Platforms	p. 261
IBM	p. 262
Oracle	p. 268
SAP	p. 272
The Microsoft Big Data Platform	p. 276
HP	p. 278
Informatica	p. 279
SAS	p. 282
Teradata	p. 283
EMC	p. 284
Amazon	p. 284
Google	p. 285
Pentaho	p. 285
Talend	p. 286
Summary	p. 286
List of Acronyms	p. 287
Glossary	p. 291
Reviewer Profiles	p. 313
Contributor Profiles	p. 317
Index	p. 333
Table of Contents provided by Ingram. All Rights Reserved.