Why your cloud migration should be data driven

Emma Frame 29th January 2020

Cloud migration is a complex undertaking for large, highly regulated businesses. The rewards are myriad: greater agility, flexibility and performance – and the potential to slash costs by being smart in the use of pay-as-you-go cloud platforms. With digital transformation comes the ability to innovate and launch new services faster and at a lower cost too, boosting time to value.

But migrating to the cloud isn’t plain sailing.

The principal driver is to unlock siloed data and make it work better for the business; data is a cornerstone for successful application deployments, analytics workflows, and machine learning insights.

Typically, though, when a business thinks about data migration, they assume it is an IT problem. This is a mistake. The reality is that without strong business sponsorship and a proven approach for the data migration strategy, it will likely fail to deliver on the promised benefits.

The main challenge relates to executing the cloud migration with minimal disruption to business operations, at the lowest cost and over the shortest period of time. If a subset of a firm’s systems, applications or data becomes inaccessible during a migration, there is the risk of costly downtime. This could negatively impact the bottom line and cause reputational damage.

You won’t be surprised to hear that careful planning and project management are key. To help you get started, here are some recommendations:

1. Put the business in the driver’s seat

Data migration is a business-driven initiative. Any project needs the support, engagement and commitment of senior business stakeholders from the outset.

The business should take the lead on the project, guiding the backend data team. With the business in charge of scoping and prioritising the migration phases, you can be sure that this is in line with the goals of the organisation.

This is crucial in ensuring that business-critical dependencies, concerns and downstream use cases are communicated and fed into the migration planning as well as making sure that roll-back strategies are tried and tested.

2. Clearly define your objectives by validating your hypotheses

Objectives can be framed as hypothetical statements that can be tested throughout the migration planning and execution stages to measure progress against business outcomes.

This will make it clear to both the business and the project teams how the new system should operate. As an example: a hypothesis could be formed around an automatic data retention policy to ensure that stale data is automatically retired in the target system on the relevant expiry date to ensure compliance. This hypothesis could then be validated early on in the migration process.

3. Take a thin slice first

Before moving data to the cloud, you need to work with the business to agree on why and where you are moving it, the types of data sets you are moving, the range of use cases, and the network resources available.

Begin by migrating low-risk data sets that reflect a large number of common use cases. Working hand in glove with the business, take a thin vertical slice through the system which surrounds the data to test the migration.

A good example of a project ripe for a vertical slice migration is a data mart and supporting data sets from your enterprise data warehouse. Assuming a common pattern in the source repository, this will give a great indication of how the rest of the marts and dependencies will fare too.

4. Experiment, fail fast and learn

Carry out a development spike to test and validate the migration hypotheses as early on as possible.  This allows time for adjustments to the schedule and approach, if needed.

For example, a migration project may hypothesise that data structures between the source system and target system are compatible and would not impede the migration. A proof of concept should take place to validate this. Anticipating any issues early on will ensure that delivery timelines can be adjusted accordingly. The business should be kept informed and be able to mitigate any issues at their end too.

5. Be metric driven

Introduce a metric-driven culture. Metrics enable teams to obtain a transparent and joint understanding of concerns and continually test their validity to prove what works, what doesn’t and measure value.

A recent data lake migration to the AWS cloud, that ECS delivered for a major bank, had to prove that it would result in scalability and cost improvements. We used metrics taken before, during and after the migration to demonstrate how the migration had dramatically reduced costs and increased scalability to meet fluctuating demand. The performance of applications using the data lake increased, and ROI and NPS scores rose too.

6. Introduce automation

Automation can be helpful where your environment has repeated patterns of activities. Highlighting these early on in the migration project is key.

As evidence, ECS’s data team recently coached a major bank looking to adopt DevOps practices for their data pipelines as part of a data migration to the cloud. We successfully used Infrastructure as Code during the deployment to drive rapid and high-quality solutions that have slashed the bank’s release cycles from weeks and months to just hours and days. And all while maintaining security and quality of service.

7. Choose the right partner

Cloud migration is an evolving, ongoing process demanding specialised skillsets and experienced people. Be sure to choose the right partner – one experienced in migrating mission-critical, complex workloads efficiently without compromising business operations  – and your journey will be a smooth one.

In summary, migrating to the cloud will transform your business. With the right approach you can rapidly deliver compelling business outcomes while maximising scalability, security and performance.

ECS has a wealth of experience in data-driven cloud migration projects. If you’d like to hear how we can help with your cloud journey, please get in touch (add link to contact page).


Found this interesting? Why not share it: