The Evolution of Data and Database Migration

Executive Summary

Large scale projects such as the merging of multiple sites, green IT initiatives, virtualization projects, database server consolidation and the cycle of technology refreshes are common in the database world. A common theme throughout all of these is the migration of data. From a database management perspective, data migration has traditionally been treated as an exception to normal operations. Data migration also seems to coincide with unforeseen difficulties that lead to extended downtime and the need to cancel, roll back and defer the activity.

In larger environments, data migration is no longer an intermittent distraction but a regular activity that expends an increasing number of organization hours. As with apparently everything else related to infrastructure, there are many technology options from which to choose. Selecting the correct approach is highly dependent on infrastructure limitations, data and platforms types, and time constraints and staff capabilities. No longer is the best practice the use of scripts, when there are many commercially available software packages that can ease this task. According to Rob Karel, principal analyst with Cambridge, Massachusetts based Forrester Research Inc., using the custom scripting approach for migrating data to today's data-centric applications leaves one fatal flaw. (Smalltree) "A custom script is not going to do anything about garbage data, it's just going to move it," Karel explained. (Smalltree)

Given this increase in migration activities, as well as the critical and often highly visible business impact of failed migrations, it is time to adopt dependable, repeatable migration practices to support various data center initiatives. However, various requirements within a dynamic and complex environment may sometimes conflict with one another. So, how does one begin to develop a standard database migration methodology?

Principle Reference

Smalltree, Hannah (January 14th, 2008) "Planning a data migration road map: from pitfalls to perfection." [Electronic Version] , bitpipe.com, Retrieved February 4th, 2008 from http://www.bitpipe.com/detail/RES/1201637160_242.html

Database Migration

When planning how to migrate a database, the migration methodology should address the planning, pre-migration, migration and post-migration phases and include detailed information within each phase, such as flowcharts and procedural documents. The degree of specificity within these documents should be high and include platform, operating system, network, and storage area network specific tasks. Comprehensive procedures should be included for verification and validation at each level.

The most important consideration for any migration is the overall business impact and, the impact on an application and its ability to access data. For this reason, the planning phase is the most critical. In a large migration project, the planning phase can and should consume more staff hours than the actual migration itself.

It is important to understand where database migration fits within the larger project. Is this database data center relocation where data may have to be migrated over long distances? Part of a consolidation effort focused on maximizing storage layout for efficiency while still ensuring appropriate service levels? Is this a lease expiration or technology refresh involving migration of data from one generation of storage hardware to the next? Data migration can also be part of a performance optimization or tiered data effort.

The heterogeneity of the environment, in terms of servers and storage, has a major impact on the migration process, particularly in determining the numbers and types of tools available to perform the movement. Similarly, the scale of the migration in terms of numbers of hosts impacted and the volume of data to be relocated are critical factors affecting how the migration is accomplished.

Diminishing business and application impact usually translates into ensuring a nominal outage and the least performance degradation. A crucial element of the planning effort is determining how to minimize migration time. This often involves organizing interdependent applications into move groups that will be migrated together.

Another vital part of planning is the verification and validation process. What steps will be taken to ensure that each phase of the migration has succeeded? It's equally important to plan and test the contingency process.

Migration preparation consists of detailed activities to ready the server, network and storage systems impacted by a given migration. Some of the preparation activities can include backing up data, capturing current configuration information, performing a detailed review of all planned changes at each level, identification of migration validation checkpoints, script configuration and review, if scripts are used, review and verification of