Data Integration Basics And Techniques

By Peggie K. Lambert


At its core, data integration is the process of combining data from different sources so that users and applications see it as coming from a single source. There are many ways to do this, and there are just as many possible applications too. It can be used for everything from research to marketing and corporate mergers.

Ideally, this is not something to be done just as an IT initiative to lower costs or improve the system. The main reason should be to help improve business processes or to solve a problem. Such systems based on integrated information collected from different sources have indeed been developed, and are quite commonly used in many industries.

A CRM system is a good example, since it requires that information flowing into separate departments should be integrated into a centralized database. This can help the marketing and sales department make targeted pitches to prospects and customers based on information collected by other departments such as customer service and social media outreach. Another application is for a corporate merger, when disparate sources must be combined into a centralized one that includes everything from both companies.

This can be done on the application level or on a middleware layer. It can also be done using virtual integration to offer simplistic views, or by going in for full-fledged physical warehousing. Let's take each of these one at a time, in order to gain an understanding of how they work and find out which one would be more appropriate under what conditions.

Some applications have the capability to retrieve and combine information from distinct sources. In such cases, there will be no need to create a new and combined source. This is just as true if the integrating logic is built on a middleware layer. It can access all the databases at the back end and provide the applications with any information required from all sources.

Virtual integration is the simplest method to create an integrating tool which does require creation of a new storage system. Under this method, a set of pre-defined queries will access required information from separate sources. For instance, consider a case where a customer profile needs to be seen. The query extracts records from all the sources based on the main index field, which is usually a customer ID. The extracted information is then presented to the user in a single and unified view.

Warehousing is a completely new system which can siphon and store information from any number of sources. This is mostly done only at an enterprise level, where vast amounts of data coming in from all of a company's departments and locations can be collected, stored and managed in massive data centers. This centralized system can then be used by applications and users to gain enterprise-wide access, reporting and analysis capability.

The choice of data integration method and the scope of the project is a critical decision. The basic deciding factors are the number of sources and their type, along with the business benefits that are expected. The project cost is important, and so are the security and backup systems impacted. Other similar projects which are likely to have some impact are ongoing migrations and synchronization, along with MDM or master data management.




About the Author:



No Response to "Data Integration Basics And Techniques"

Post a Comment

Powered by Blogger