Data Migration Project Solutions

Data Migration Methods

I recently completed a project which involved consolidating customer accounts across multiple billing platforms. The two billing platforms have fundamentally different data structures and methods of data management – from customer address fields and data types to the package catalog structure. The data to be migrated included:

  • Customer name
  • Customer address Information (address11, … address4, email, other contact details)
  • Customer package (package id, cost)
  • Customer equipment (equipment id, in conjunction with package details sent to middle ware provisioning server for provisioning end user devices)
  • Outstanding balance

This was a relatively simple project. The migration plan included only a small subset of customers (approximately 2000). The average customer was provisioned with 5 packages and 3 pieces of equipment. In addition, only current data was to be migrated onto the new platform (specifically excluding data greater than 45 days old).

There were essentially two options for consolidating this data. The first option was manual input and required staff resources (preferably familiar with both systems), and budget allocations for staff overtime. This option was attractive for a number of reasons:

  • Data cleaning could be completed while being entered into the new system
  • Eliminated the need for a separate package mapping process, which would be required for SQL import (and which would have required staff resources regardless)
  • Estimated cost (primarily staff overtime charges) was less than using vendor services for data import
  • Gave ownership of the data and data migration process to the staff (key stakeholders) who would be using the system (more on this below)

Stakeholder Ownership
The greatest risk for this project (highest impact, greatest probability) was inaccurate data. Billing systems are fundamental to revenue assurance and core to customer management. In giving ownership of data accuracy to the system users – those who would be interacting with customers and using the data on a daily basis – the project manager is acting to preempt stakeholder rejection of the data. Since the system users are more familiar with the system (and presumably with the company policies, procedures, products and services), they are likely to capture inaccuracies. In addition, since they have a high stake in the quality of the system, they are likely to be accurate in data input. Management control and thorough review is necessary to support this process.

The second option for consolidating the data was to outsource the import to the platform vendor. This option was attractive for a number of reasons as well:

  • Fewer internal resources would be required. In theory, data cleansing alone should require less time than data cleansing plus data entry.
  • Quicker completion (though considerably more testing and iterations required)

The decision was made to move forward with manual data consolidation. This decision was made based on the following criteria:

  • Staff resources would be required regardless of method chosen. The two systems used significantly different data structures for capturing and managing customer data, so considerable cleaning was required.
  • Low confidence in the ability of the vendor to complete this task with few iterations and rollbacks. The testing process, regardless of method, is time intensive and each iteration requires comprehensive testing.
  • Cost. Staff overtime would be required in either scenario. The vendor’s quote for performing the import only was four times the most aggressive estimates for required staff overtime.
  • Staff were interested in participating for a variety of reasons: overtime and personal career objectives were the most common reasons given. Because of the high staff interest level, management and key stakeholders were more confident of a positive outcome.

In the end, the manual data input method required a substantial amount of time, but tested virtually error free. With 2000 customer records, staff were able to, on average, input 8 records per hour. This project used 7 staff. Including testing, corrections and data input, all of the data was successfully consolidated within 1 week. Costing was substantially cheaper than using the vendor. For small data migration customers (assuming resource availability), manual data consolidation is worth considering – the main caveat being that any transformation needs to be completed quickly, as current systems are actively updated.

You may also like...