How to Make Better Data Decisions
We explore three main areas your enterprise can focus on to gain more control of your data.
- By Amnon Drori
- May 8, 2020
Corporations around the globe rely on data to make some of the largest business decisions, and when their data isn't easily accessible or accurate, poor decisions are often made, which can affect the company in catastrophic ways.
For example, last year data issues became apparent at Hawaiian Airlines when in a single week they charged customers in dollars rather than frequent flyer miles and mistakenly charged one customer $674,000 while other customers were charged zero miles for their trip. To make matters worse, when trying to fix the faux pas, the company canceled the tickets, further upsetting customers. Such data issues can create an image of unprofessionalism and taint a loyal customer relationship.
Gartner recently estimated that 40 percent of a company's data is lacking accuracy and does not communicate the full picture. What can your enterprise do to prevent these issues?
To avoid the data trap Gartner describes, outline a BI strategy that will put you in control of your data environment. Develop a deep understanding of the source(s) of your data, how it is formatted, its status (is it active or archived?), how sensitive it is, who owns it, where it should be stored, and how it has been changed. These are crucial issues about metadata that all BI teams should grasp, especially in today's tight regulatory atmosphere.
Regulatory and legal requirements in both the EU and U.S. are putting corporations to the test. Your enterprise must have a complete view and map of all company data. In fact, Facebook presented a plan for regulation that Mark Zuckerberg announced in Brussels and outlined in a white paper that emphasized global (rather than specific national) regulations. The plan was recently rejected by the EU.
"'Will companies need to modify their existing AI solely for Europe, and if so, how? Alternatively, will other jurisdictions start raising their legal standards?'" asks Ryan Dunleavy, partner at law firm Stewarts. The article points out that "Until the rules around AI governance become more standardized internationally ... '[C]ompanies are likely to need to take a mosaic approach to developing regulations and legislation affecting AI in different jurisdictions.'"
How can your enterprise develop a comprehensive approach to data that can cover international demands?
There are three main areas that companies can focus on to gain more control over their data:
- Automated data lineage
- Automated data discovery
- Automated business glossaries
Let's examine each of these focal points.
Data Lineage
Today, business decisions need to be made quickly. Often manual data mapping isn't a viable option. With automated data lineage and data discovery, what once took BI and analytics teams weeks now takes moments, creating a more accurate view of the data landscape in real time. Automating data lineage to track down the data source with graphical tools will be crucial so business analysts can locate the exact source of the data and where it has gone. This view allows analysts to easily correct errors in reports and determine how they occurred. The data lineage map also gives business intelligence teams a full view of a report to see if all data is incorporated and if it represents an accurate picture.
Data Discovery
Organizations need to be able to locate their data in order to leverage it to make effective business decisions. However, it can be extremely difficult to access and make sense of data that is scattered throughout multiple tools and systems. BI and analytics teams often rely on manual processes to comb through different data sets, reports, table views, and sources to collect the required data, which is extremely time-consuming and ineffective. Automated data discovery tools can locate data for BI teams in real time to help them make business decisions quickly and accurately.
The ability to have all of one's metadata centralized on the cloud, instantly available, searchable, and refreshable on demand ensures up-to-date data accuracy and fosters an environment where the organizations no longer have to work for their data.
Business Glossaries
A business glossary is a listing of standard company-specific business terms and their definitions. This is important for data management because it helps an enterprise align disparate data assets so that terms used in one area of the business mean the same thing in another. When all data assets adhere to the enterprise's business glossary, it's easier to locate and combine similar data from multiple data sources in the same report or dashboard without fear of comparing apples and oranges.
Building a business glossary has traditionally been an intimidating and cumbersome project that few organizations ever actually completed, but the business glossary can be generated automatically.
Final Thoughts
These actions, along with a robust data governance program, can help get a corporate data environment under control and ready to unlock the valuable information that the enterprise holds. The most effective method to resolve data management issues is with automation. When automated tools are applied to these tasks, they are completed much more quickly. Much of the grunt work is done for the BI team, allowing team members to focus on analysis. Automation in data management can reduce task errors and free the IT and BI teams for more productive, value-added pursuits.
Additionally, in the current technological landscape, machine learning and artificial intelligence are transforming automated processes in every industry. Today, metadata management solutions must incorporate artificial intelligence and machine learning to discover and define correlations within an analytics environment and access metadata to create the mapping of data lineage. These two core technologies -- AI and machine learning -- will power the automation and develop the modeling and indexing of metadata.
By incorporating these three strategies, corporations will gain more control of their data in a business and legal environment that requires high standards for data protection. With the global demand for data protection, the risks are too high for complacency, especially when automation makes it so easy to be one step ahead.
About the Author
Amnon Drori is the co-founder and chief executive officer at Octopai, a company that automates cross-platform metadata management. You can reach the author via email, Twitter, or LinkedIn.