Data Management Best Practices for Cloud and Hybrid Architectures
User practices and vendor tools must adapt to the increasing presence of cloud-based IT systems.
- By Philip Russom
- November 27, 2017
For years, TDWI has seen an increasing number of organizations make strategic commitments to the cloud as a preferred computing platform. The commitments involve a wide range of use cases, from operations to analytics to compliance, and the journey to the cloud can take many forms and follow many paths.
For example, you may move your entire data center or just single applications and individual data warehouses or databases. You may move existing solutions or build greenfield ones in the cloud (the most popular choices being sales and marketing automation, Hadoop, and data lake implementations. Regardless of the IT systems involved, you may evolve into a cloud-only architecture or one that's a mix of multiple platform types in the clouds and on premises.
The journey to the cloud is driven by many business needs, such as those for new operational and analytics applications, data center cost containment, business agility, IT modernization, data-driven competitive marketing, and new use cases for big data, advanced analytics, and self-service approaches to these.
Data Management Is More Important Than Ever
The best practices of data management don't disappear just because data and workloads shift to the cloud. In fact, data management best practices (DM BPs) are more important than ever. On the one hand, DM BPs stitch together the complex multiplatform data environments within which clouds typically exist, even providing integration for extreme hybrid mixes of systems that are on multiple premises and multiple clouds. On the other hand, DM BPs are fundamental to cloud success metrics, such as trusted data that is high quality, auditable, secure, governed, fit for purpose, and open for self-service.
As more organizations begin their journey to the cloud, they need to plan how they will apply the best practices of data management to ensure that cloud-based, data-driven use cases are successful for end users while complying with enterprise governance and data standards. The good news is that existing best practices work well in cloud environments, although users usually need a few adjustments and upgrades to existing skills and tool portfolios.
What All Paths to the Cloud Need
All of these journeys involve a mass migration of diverse data and therefore require sophisticated data management infrastructure, tools, and user best practices during both development and production. For example, some organizations have adopted data catalogs to discover, curate, manage, and prioritize the data that should be migrated to the cloud.
All paths also require upgraded skills and governance. Furthermore, clouds increasingly play a special role in hybrid multiplatform data architectures (MDAs), and so the DM requirements of MDAs are also a consideration.
More Than Moving Data
On your journey to the cloud, you don't just move data. You need to improve your data and its management. If your on-premises data and its management is a hairball, you will need to tease out at least part of the hairball to migrate data and integrate them with cloud-based systems. Remember, adjusting to cloud-based data management is an opportunity to assess the mistakes of the past and do things better. It is a chance to tighten up technical best practices in data integration, quality, modeling, and architecture and adopt organizational best practices in data governance, stewardship, and curation.
To achieve such improvements, organizations turn to integration hubs. These hubs enable orchestrated data flows for cloud and hybrid deployments in a governed and self-service fashion. Hubs can also correct the point-to-point hairballs that are typical of too many data management architectures.
For Further Reading
For more information about how data management best practices and tools must adapt to the increasing presence of cloud-based IT systems, read TDWI's new Checklist Report. It also discusses how data management and its best practices can contribute to the success of individual cloud systems, as well as the success of the grander hybrid environments within which clouds typically operate.
The report is online for free download here.
About the Author
Philip Russom is director of TDWI Research for data management and oversees many of TDWI’s research-oriented publications, services, and events. He is a well-known figure in data warehousing and business intelligence, having published over 600 research reports, magazine articles, opinion columns, speeches, Webinars, and more. Before joining TDWI in 2005, Russom was an industry analyst covering BI at Forrester Research and Giga Information Group. He also ran his own business as an independent industry analyst and BI consultant and was a contributing editor with leading IT magazines. Before that, Russom worked in technical and marketing positions for various database vendors. You can reach him at prussom@tdwi.org, @prussom on Twitter, and on LinkedIn at linkedin.com/in/philiprussom.