The Role of Centralization and Self-Service in a Successful Data Hub
A hub should centralize governance, standards, and other data controls, plus provide self-service data access and data prep for a wide range of user types.
By Philip Russom, Senior Research Director for Data Management, TDWI
I recently spoke in a webinar run by Informatica Corporation, sharing the stage with Informatica’s Scott Hedrick and Ron van Bruchem, a business architect at Rabobank. We three had an interactive conversation where we discussed the technology and business requirements of data hubs, as faced today by data management professionals and the organizations they serve. There’s a lot to say about data hubs, but we focused on the roles played by centralization and self-service, because these are two of the most pressing requirements. Please allow me to summarize my portion of the webinar.
A data hub is a data platform that serves as a distribution hub.
Data comes into a central hub, where it is collected and repurposed. Data is then distributed out to users, applications, business units, and so on.
The feature sets of data hubs vary. Home-grown hubs tend to be feature poor, because there are limits to what the average user organization can build themselves. By comparison, vendor-built data hubs are more feature rich, scalable, and modern.
A true data hub provides many useful functions. Two of the highest priority functions are:
- Centralized control of data access for compliance, governance, security
- Self-service access to data for user autonomy and productivity
A comprehensive data hub integrates with tools that provide many data management functions, especially those for data integration, data quality, technical and business metadata, and so on. The hallmark of a high-end hub is the publish-and-subscribe workflow, which certifies incoming data and automates broad but controlled outbound data use.
A data hub provides architecture for data and its management.
A quality data hub will assume a hub-and-spoke architecture, but be flexible so users can customize the architecture to match their current data realities and future plans. Hub-and-spoke is the preferred architecture for integration technologies (for both data management and applications), because it also falls into obvious, predictable patterns that are easy to learn, design, optimize, and maintain. Furthermore, a hub-and-spoke architecture greatly reduces the number of interfaces deployed, as compared to a point-to-point approach, which in turn reduces complexity for greater ease of use and maintainability.
A data hub centralizes control functions for data management.
When a data hub follows a hub-and-spoke architecture, it provides a single point of integration that fosters technical standards for data structures, data architecture, data management solutions, and multi-department data sharing. That single point also simplifies important business control functions, such as governance, compliance, and collaboration around data. Hence, a true data hub centralizes and facilitates multiple forms of control, for both the data itself and its usage.
A data hub enables self-service for controlled data access.
Self-service is very important, because it’s what your “internal customers” want most from a data hub. (Even so, some technical users benefit from self-service, too.) Self-service has many manifestations and benefits:
- Self-service access to data makes users autonomous, because they needn’t wait for IT or the data management team to prepare data for them.
- Self-service creation of datasets makes users productive
- Self-service data exploration enables a wide range of user types to study data from new sources and discover new facts about the business
These kinds of self-service are enabled by an emerging piece of functionality called data prep, which is short for data preparation and is sometimes called data wrangling or data munging. Instead of overwhelming mildly technical or non-technical users with the richness of data integration functionality, data prep boils it down to a key subset of functions. Data prep’s simplicity and ease-of-use yields speed and agility. It empowers a data analyst, data scientist, DM developer, and some business users to construct a dataset with spontaneity and speed. With data prep, users can quickly create a prototype dataset, improve it iteratively, and publish it or push it into production.
Hence, data prep and self-service work together to make modern use cases possible, such as data exploration, discovery, visualization, and analytics. Data prep and self-service are also inherently agile and lean, thus promoting productive development and nimble business.
Centralization and self-service come together in one of the most important functions found in a true data hub, namely publish-and-subscribe (or simply pub/sub). This type of function is sometimes called a data workflow or data orchestration.
Here’s how pub/sub works: Data entering the hub is certified and cataloged on the way in, so that data’s in a canonical form, high quality, and audited, ready for repurposing and reuse. The catalog and its user-friendly business metadata then make it easy for users and applications to subscribe to specific datasets and generic categories of data. That way, users get quality data they can trust, but within the governance parameters of centralized control.
Summary and Recommendations.
- Establish a data architecture and stick with it. Rely on a data hub based around a hub-and-spoke architecture, not point-to-point hairballs.
- Adopt a data hub for the business benefits. At the top of the list would be self-service for data access, data exploration, and diverse analytics, followed by centralized functions for data governance and stewardship.
- Deploy a data hub for technical advancement. A hub can organize and modernize your infrastructure for data integration and data management, as well as centralize technical standards for data and development.
- Consider a vendor-built data hub. Home-grown hubs tend to be feature-poor compared to vendor-built ones. When it comes to data hubs, buy it, don’t build it.
- Demand the important, differentiating functions, especially those you can’t build yourself. This includes pub/sub, self-service data access, data prep, business metadata, and data certification.
- A modern data hub potentially has many features and functions. Choose and use the ones that fit your requirements today, then grow into others over time.
If you’d like to hear more of my discussion with Informatica’s Scott Hedrick and Rabobank’s Ron van Bruchem, please click here to replay the Informatica Webinar.
Posted on July 12, 2016