Required time to read: < 10 minutes

Introduction

The most important and generally overlooked activity during cloud transformation is data migration. We always consider data migration activities as one-time technical tasks. In my experience, more than 75% of cloud implementation delays are due to data issues that are identified during the later stage in the project (integration or user acceptance testing). This leads to increased cost, timeline, go-live delays and challenges with resource planning. I have observed that the common factor in all successful cloud transformation projects is well-planned data conversion activities and thorough execution with business ownership throughout the project life-cycle.

Organizations implementing Oracle applications cloud will have the requirements of migrating legacy / on-premise / third-party cloud applications data into Oracle applications cloud. Oracle applications cloud provides several utilities to migrate data in both real-time and batch.

Consider a global organization implementing Oracle multi-pillar (Human Capital Management (“HCM”), Enterprise Resource Planning (“ERP”), Customer Experience (“CX”) and Supply Chain Management (“SCM”)) fusion applications cloud to help their global transformation.  The most common global implementation projects adopt one of the two deployment models.

1. Modular implementation deploys one module at a time globally. A phased approach will be based on functionalities such as Human Resources, Financials, Sales, Service, Order Management, and so on in the order of your business priorities.

2. Regional/Country based implementation deploys all modules at the same time per region/country. A phased approach will be based on regions/countries such as (Asia Pacific, North America, Europe, Middle East, Africa, Japan, Australia, etc.) in any order depends on the business priorities.

For more information on Oracle multi-pillar fusion applications, refer to my previous blogs:

1. Oracle Fusion Applications Cloud Instance Strategy and Global Single Instance Guidelines
2. Best Practices for Configuring and Extending Fusion Applications Cloud

Irrespective of the deployment model chosen, accurate data is the foundation of any application for a higher business adoption rate. When you migrate data from your legacy application to the new target application, data issues such as inaccurate, incomplete, inconsistent and duplicate data may cause business functions to fail in the new application. Hence, understanding of existing source system, existing business processes, and the target system is a critical component for successful data migration.

In real life, when we are moving from an old house to a new one or from one house to another, we always pack what we need and filter out the unwanted items. We spend our time classifying unwanted items to either dispose of or give to goodwill/thrift stores. One should plan to apply the same concept for data migration to cloud.

Data Migration Approach

Organizations must plan for a reusable strategy for efficient data migration especially on a phased implementation approach. Some organizations would take extreme measures on focusing on automation of data migration processes, and spend millions of dollars which will derail their broader deployment focus. Thorough planning and creating a data migration strategy in the early stage of the project will be realized as the major critical success factor at the end of any successful deployment.

While these two data migration options and guidelines are given as a reference for your Oracle Fusion application cloud implementation projects, these may be applicable for all cloud transformation projects. 

1) Lift and Shift : A traditional data migration approach to convert data from your legacy systems “as-is” to the new target application format.

2) Transform and Shift: An effective and efficient data migration strategy to identify data issues from legacy data and transform the data for the modern use. This option is best suited for organizations undergoing cloud transformations.

Lift & Shift

The Lift & Shift approach will facilitate the “as-is” data migration from legacy applications into Oracle Fusion applications. Technical teams plan on data mappings between source and target applications and execute migrations. Many organizations select this approach for its simplicity, and due to their limited availability of business resources to support the project team.

Benefits:

• Simple approach

• No additional tools required.

• Technical focused and limited business resources need.

Challenges:

• No/limited data analysis performed (Garbage-In --> Garbage-Out).

• All legacy and historical data are migrated “as-is” to cloud.

• Data issues are identified after data migration as the business user starts using the application.

• The time and cost for troubleshooting, maintenance, and fixing data issues in production are unknown (reactive approach)

• If the data issues are identified in production, it will have significant impact for business continuity.

• Typically, data issues are identified by business users during user acceptance testing which cause delays to projects. Lack of volume and performance testing could impact the timelines as well.

Transform & Shift

The Transform & Shift approach will facilitate clean data migration from legacy applications into Oracle Fusion applications. Technical teams collaborate with business users to help them in data analysis, quantifying data issues, and identifying the required mappings & rules to migrate cleansed data. This approach involves business users throughout the life-cycle of the project and helps them resolve their data issues quickly. Organizations that recognize the value of data, data governance and ownership often use this approach and have benefited from it.

At the very minimum, data analysis must be planned in your project and should be performed with the utilities/tools. While SQL (Structured Query Language) or custom-written programs will work for smaller data volume and simple objects, it will become more complex for large data volume and organizations with several business rules. If your organization already has a data quality software, you could leverage it for analyzing your data. Otherwise, Oracle Enterprise Data Quality(“OEDQ”) may be used for data analysis and data preparation.  You can perform data analysis and verification using OEDQ on millions of records within hours, and your business will benefit from the results very early in the project.

Data analysis will quantify your data issues, which will help your business plan their resources to address the data conflicts. Many of these data conflicts can be fixed by rules, such as data standardization, verification and merging rules based on your business needs. This analysis will also help business users to come up with validation and correction rules that could be automated during the data load.

Legacy/source data with more data issues can be quarantined and business users can decide how they should fix their data in the source system or ignore them from loading into the target.

Benefits:

• Data analysis will help pro-actively quantify the legacy/source data issues early in the project.

• The business is involved throughout the project which helps them train the users as well.

• The business takes responsibility & ownership of its data and decision making.

• The data correction and conflict resolution process can be defined during the project.

• There is limited impact on the business continuity as the process for data conflict resolution is well defined.

• You would have a reusable data migration framework to benefit from for phased global implementation, and potentially the same framework can be used for the merger & acquisition data migration process.

• This option will also help the testing/quality assurance team define their test cases and test data efficiently.

Challenges:

• Additional tools, time and resources would be required.

• Planning and development may be required for historical data conversion and reporting.

Data Migration Guidelines

• Clearly define the scope for data migration and strategy containing the approach, resources requirements, data cleanup process, ownership, tasks, and project plan for data migration.

• Do not forget to plan data migration window and system downtime possibilities.

• Plan for iterative data analysis on sample data and full data set.

• Consider data volume, environment sizing, and plan your cut-over timings based on your mock-run results.

• Consider tasks in your project plan for the availability of your environments (Test/ Development) and code migration required prior to data migration.

• Do not forget to plan optimizing, sequencing, parallel execution and tuning your data migration process.

• Collaboration with developers and data migration team is critical for managing the application extensions and custom rule configurations, that will need to be applied to data migration also.

• Plan at least two cycles of mock data migration to ensure identifying major data issues prior to go-live.

• Plan to perform functional testing as a post-data migration task.     

• Develop clear guidelines, security considerations around personally identifiable information (PII) and compliance-related information.

• If you have third-party/supplier data, you must plan data analysis to transform/map the right third-party entity/attributes to your target application.

Conclusion

Ultimately, the end goal for every organization is to have efficient business processes and accurate data in their enterprise applications.

Choosing your data migration approach is an integral part of your cloud transformation program. While the lift and shift approach will help simple use cases, the transform & shift approach is proven to be successful, helping several customers on their cloud transformation.

References

Oracle Enterprise Data Quality