Enterprise Application Integration Using .NET

by Clark, Butch

• ISBN13: 9780321202956
• ISBN10: 0321202953
• Format: Paperback
• Copyright: 2005-01-01
• Publisher: Addison-Wesley Professional

Summary

The ultimate guide to attacking one of the biggest problems in IT today - application integration - using the .NET Framework.

Author Biography

Butch Clark has more than twenty years of experience in software analysis, design, development, and enterprise back-end process integration

Table of Contents

Preface	xix
The Great Integration Problem	1	(18)
Introduction	1	(1)
The Data Spread Issue	2	(2)
Opportunity Costs	2	(1)
People Costs	2	(1)
The Cost of Making Poor Business Decisions	3	(1)
Costs Resulting from Poor Customer and Partner Perceptions	4	(1)
Solving the EAI Problem	4	(8)
Integration as a Tool	5	(6)
Integration Diagrams	11	(1)
Defining Business Integration and EAI	12	(3)
Data View Versus Process View	13	(2)
Approaching an Integration Project	15	(1)
Get Management Support	15	(1)
Understand What's Happening Now	15	(1)
Understand What Should be Happening	16	(1)
The .NET Framework as a Solution	16	(1)
Summary	17	(2)
What's Coming Up	18	(1)
What Do You Really Need?	19	(24)
Introduction	19	(1)
A First Word on People	20	(1)
Common Trouble Spots	20	(1)
Corporate History and Culture	21	(2)
Monkey See, Monkey Do	22	(1)
Combating Trouble	23	(1)
The Information-Collection Phase	23	(13)
First Steps	25	(1)
Assembling a Team	25	(1)
Conducting Team Kick-Off Meetings	26	(3)
Gathering Information from End Users	29	(1)
Determining Scope	30	(1)
Determining What Systems Are in Place	31	(1)
Determining What Data Is Available	32	(1)
Determining What Manual Processes Are Available	33	(2)
Determining What Recurring Processes Are Being Performed	35	(1)
Determining What Business Tasks Need to Be Performed	35	(1)
The Analysis Phase	36	(5)
Begin the First Analysis	37	(3)
Gather System Interface Information	40	(1)
Summary	41	(2)
What's Coming Up	42	(1)
Requirements	43	(16)
Introduction	43	(1)
Requirements Backgrounder	43	(1)
Our (Fictional) Company	44	(3)
The Way Things Are Now	45	(1)
Our Goals	46	(1)
The Major Features We Will Address	46	(1)
Systems Currently in Use	47	(1)
A More Detailed Look at Our Requirements	47	(10)
R001: Generally Acceptable/Open Interface	47	(1)
R002: Requests Accepted Regardless of Originating OS	48	(1)
R003: Open, Easy-to-Use ``Request Language''	48	(1)
R004: Capability to Track Every Request Sent into the System	48	(1)
R005: Must Be Secure	49	(1)
R006: Capability to Add Tasks Without Changing the System	49	(1)
R007: Capability to Talk to Virtually Any Other System, OS, or Software Package	50	(1)
R008: Capability to Define a Logical Request and Perform Tasks on Its Behalf	50	(1)
R009: Capability to Validate the Input Data	51	(1)
R010: Capability to Define Business Rules	52	(1)
R011: Capability to Identify Portions of Requests That Can Cause a Rollback	53	(1)
R012: Capability to Provide Management Reports	54	(1)
R013: Capability to Provide Administrative Functions	54	(1)
R014: Provide a Flexible User Authorization Mechanism for Various Parts (Per User)	55	(1)
R015: Capability to Lock Out a User upon a Configurable Number of Password Failures	55	(1)
R016: Capability to Provide a Mechanism to Alert Support Staff of Errors in Real Time	56	(1)
R017: Capability to Add New Functionality to the System Quickly	56	(1)
R018: Capability to Continue Processing After Catastrophic System Failure	56	(1)
R019: Capability to Define Users and Their Roles, and Assign Administrator Privileges to Them	57	(1)
Summary	57	(2)
What's Coming Up	57	(2)
Our Solution	59	(18)
Introduction	59	(1)
Platform Requirements	59	(2)
Solution Overview	61	(8)
The Input Requests Block	62	(3)
The Web Service Block	65	(1)
The EAI Framework Back-End Components Block	66	(2)
The Database Block	68	(1)
Integration Endpoints	68	(1)
High-Level Activity Diagram	69	(6)
Summary	75	(2)
What's Coming Up	75	(2)
The Web Service	77	(46)
Introduction	77	(1)
Web Service Concepts	77	(11)
.NET XML Web Services	78	(1)
Communication Protocol	79	(6)
The Login Process	85	(3)
Web Service Tools	88	(7)
Creating a Vanilla Web Service	89	(6)
The EAI Framework Web Service	95	(26)
Setting the Default Namespace	96	(6)
The ConfigData Class	102	(1)
The EAIConfig.xml File	103	(4)
The EAIUser Class	107	(6)
The Authenticator Class	113	(6)
The UtilityTester Program	119	(2)
Summary	121	(2)
What's Coming Up	121	(2)
Logging	123	(32)
Introduction	123	(1)
System.Diagnostics Namespace	123	(10)
Trace and Debug Classes	124	(4)
Changeable Logging Levels	128	(4)
The TraceSwitch Class	132	(1)
Writing the Code	133	(21)
The DBLogProxy Class	133	(11)
The Logger Class	144	(10)
Summary	154	(1)
What's Coming Up	154	(1)
The Controller	155	(44)
Introduction	155	(1)
Getting to the Controller	155	(16)
The Controller Structure	156	(15)
System.Messaging Namespace	171	(9)
Message Queue Types	172	(2)
Recoverable Messages	174	(1)
Sending a Message	174	(1)
Examining Message Queues	175	(3)
Reading Messages	178	(2)
RequestQueMonitor Project	180	(4)
RequestQueMonitor startListening() Method	181	(3)
RequestsProcessor Discussion	184	(13)
System.Reflection Namespace	186	(1)
RequestHandlerFactory.cs	187	(4)
RequestsProcessor Code	191	(6)
Summary	197	(2)
What's Coming Up	198	(1)
RequestHandlers	199	(32)
Introduction	199	(1)
RequestHandlerBase Class	199	(4)
Extending RequestHandlerBase	203	(23)
Our First RequestHandler	203	(4)
Steps to Add a New Request Type	207	(18)
Checking Request-Specific Business Rules	225	(1)
Steps and Agents	226	(1)
Testing a New RequestHandler	227	(2)
Summary	229	(2)
What's Coming Up	230	(1)
Steps	231	(22)
Introduction	231	(1)
Steps Overview	232	(18)
Error Handling	233	(1)
Step Database Tables	234	(5)
The StepFactory Class	239	(3)
The StepBase Class	242	(6)
Creating a RequestHandler to Call Steps	248	(2)
Summary	250	(3)
What's Coming Up	251	(2)
Agents	253	(10)
Introduction	253	(1)
Structure of Agents	254	(8)
Calling an Agent	258	(2)
Expanding on an Agent	260	(2)
Summary	262	(1)
What's Coming Up	262	(1)
Business Rules	263	(22)
Introduction	263	(1)
Verification and Validation Tasks	263	(3)
Checking for Tags	264	(1)
Checking for Complex Tags	265	(1)
Validating Tag Values	265	(1)
Performing Other Types of Validation	266	(1)
Pre-/Post-Filtering	266	(11)
The PreProcBusinessRules Class	267	(2)
The PostProcBusinessRules Class	269	(3)
The BusinessRulesBase Class	272	(5)
Calling BusinessRules from a RequestHandler	277	(5)
Summary	282	(3)
What's Coming Up	283	(2)
Extending the System	285	(8)
Introduction	285	(1)
Some Basic System-Extension Rules	285	(2)
Examples of Extension, Customization, and Utilities	287	(2)
Security	287	(1)
Data-Validation Utility	287	(1)
External Request Validation	288	(1)
Utility to Report and Escalate Problems	288	(1)
Performance Metrics	288	(1)
Personalized Portal Interfaces	288	(1)
Configurable Processing Using a Bill of Material or Pipeline Strategy	288	(1)
Special Logging	289	(1)
Adding IP Checking	289	(2)
Summary	291	(2)
What's Coming Up	291	(2)
Front Ends	293	(22)
Introduction	293	(1)
The Client Function	293	(12)
Running EAIClient	293	(4)
Creating a Front-End Application	297	(4)
Examining the EAIClient Code	301	(4)
Writing Your Own Front End	305	(1)
Web Application Front End	306	(1)
A Quick Java Client	307	(4)
The Java Code	309	(2)
A Quick Perl Submitter	311	(1)
Other Front-End Utilities	312	(1)
Summary	313	(2)
What's Coming Up	314	(1)
The Database	315	(38)
Introduction	315	(1)
EAIFramework Database and Tables	316	(14)
Creating Your Database	322	(3)
Creating Your Tables	325	(5)
Stored Procedures and the Issue of Security	330	(17)
Stored Procedures	331	(6)
Proxy Classes	337	(10)
Scripts	347	(5)
Scripts to Initialize and Load Sample Data	347	(1)
Scripts to Pull Out Reporting Data	348	(3)
Script to Remove Sample Data	351	(1)
Summary	352	(1)
What's Coming Up	352	(1)
UML	353	(18)
Introduction	353	(1)
UML Overview	354	(14)
Use-Case Diagrams	355	(2)
Statechart Diagrams	357	(3)
Class Diagrams	360	(3)
Sequence Diagrams	363	(2)
Activity Diagrams	365	(2)
Component Diagrams	367	(1)
Summary	368	(3)
What's Coming Up	369	(2)
XML	371	(10)
Introduction	371	(1)
XML Overview	371	(1)
An XML Primer	372	(8)
Defining XML Documents	373	(1)
Defining XML Namespaces	373	(1)
Defining the System.XML Namespace	374	(1)
Reading XML with a TextReader	375	(1)
Reading XML with a DOM	376	(2)
Your First Foray into XPath	378	(2)
Summary	380	(1)
What's Coming Up	380	(1)
SOAP Requests	381	(26)
Introduction	381	(1)
What Is SOAP?	381	(2)
Your SOAP Headers	383	(7)
Using Custom Headers	384	(3)
Web Service Changes	387	(1)
Unknown SOAP Headers	388	(2)
The SoapFramework Class	390	(11)
Running This Web Service	399	(1)
Testing This Web Service	400	(1)
SOAP Extensions	401	(4)
Inserting Code into the Hooks	403	(2)
Summary	405	(2)
What's Coming Up	405	(2)
Testing	407	(14)
Introduction	407	(1)
Why We Test	407	(1)
How We Test	408	(1)
Phases of Testing	408	(5)
Unit Testing	410	(2)
System Testing	412	(1)
Integration Testing	412	(1)
User Acceptance Testing	412	(1)
Testing Integration Systems	413	(1)
.NET Framework Testing Tools	413	(6)
Summary	419	(2)
Appendix A: Interacting with the Windows Registry	421	(4)
Set Up the ConfigFile in the Registry	421	(2)
Read the ConfigFile Value from the Registry	423	(2)
Appendix B: Regular Expressions	425	(4)
A Quick Trip Through Regular Expressions	425	(4)
Appendix C: SQL and SQL Server	429	(36)
Introduction	429	(1)
SQL Server	429	(18)
Accessing SQL Server and SQL Query Analyzer	429	(3)
Exploring the Query Analyzer Interface	432	(1)
Explaining and Executing SQL	433	(14)
ADO.NET	447	(12)
Namespaces That Support SQL Server Access/ADO.NET	447	(1)
Major Classes	447	(12)
Proxy Classes for the EAIFramework Tables	459	(6)
Admin Program	464	(1)
Bibliography	465	(2)
Index	467

Excerpts

Preface This book walks through several common issues that software designers and developers face when working through an integration problem. It then discusses in detail one solution, the EAI Framework project. This solution is written in C# for the .NET Framework. All the code shown in the book, as well as utilities and articles describing enhancements to portions of the framework, is available for downloading at http://www.EAIFramework.com . I have seen estimates of many billions of dollars either needlessly spent or missed entirely each year in the United States alone because disparate systems do not adequately talk among themselves within a company. This is compounded when you consider companies that are working together but that don't have adequately integrated communications. That's not to suggest that all integration problems are massive in scope. Indeed, small and medium enterprises (SME) can sometimes greatly reduce costs by implementing an integration solution for key manual processes. Not only do you gain productivity by automating routine manual processes, but you also speed the synchronization of back-end systems. In addition, you increase the likelihood that the systems contain correct, valid data by having code interact with the back-end systems rather than relying on a person to be consistently error-free. The first section of the book begins by looking at these types of issues and what kind of hazards might be waiting for you as you analyze your specific integration needs. In particular, Chapter 2, "What Do You Really Need?", speaks to an issue that a lot of developers tend to discount or ignore completely. Unfortunately for many software projects, people interactions can easily make or break these projects. Starting with Chapter 5, "The Web Service," we look at the C# code needed to drive the EAI Framework project. It is presented as a .NET Framework solution, shown in Visual Studio .NET. Familiarity with VS.NET and the .NET Framework are helpful, but wherever possible, I have given detailed instructions on writing the code. The primary audience for this book is software designers and developers. Parts of the book, especially the first four chapters, are suited to technical managers as well. For chapters that discuss actual code, it is assumed that the developer has some development experience and understands object-oriented development. When you understand the solution presented in this book, you can extend it to solve your particular integration needs with a fairly small amount of development work. This involves creating new request handlers that execute when an integration message arrives at your integration web service's front door. As the first couple of chapters point out, however, the process of gathering the requirements for integration projects can sometimes be much more arduous than the actual software-development effort. The intention of the software included in this book is not to be a commercial, off-the-shelf product. It is intended to be a learning tool, but it can certainly be used as is. I expect you to extend and modify it as you see the need, based upon your particular situation. For example, if you are implementing it as an interdepartmental integration solution, you might not need to worry about security much. If you implement it on the web to support customer requests, you probably will need to worry about security issues. Some of these types of enhancements are discussed in the book, but it would be impossible to cover every scenario that you might run into. Check the web site http://www.EAIFramework.com for additional information and enhancements. Many commercial EAI products, and no shortage of consulting companies, can help solve integration issues. They are o