Executive Q&A: How Decision Intelligence Can Improve Your Business Insights
Enterprises face several key challenges trying to analyze their data. Tellius CEO and founder Ajay Khanna spoke to TDWI about those issues and explains how decision intelligence can open enterprises to new insights.
- By Upside Staff
- June 27, 2022
Upside: What are the biggest challenges organizations face today when trying to analyze data to gather insights that will inform business decisions?
Ajay Khanna: The biggest challenges facing organizations trying to analyze data are a disconnect between what modern knowledge workers need from data and what their tools and internal processes deliver; a lack of collaboration between business teams and data experts; and tedious and time-consuming data tasks.
Modern knowledge workers are more data-savvy than ever and have instant access to a world of information through their mobile devices. In the workplace setting, what decision makers need is quick access to ad hoc descriptive and diagnostic analytics (to understand why things change) and prescriptive analytics (to determine the next best action). Instead, they are limited to curated dashboards that monitor high-level KPIs with few options to ask new questions or perform deeper analysis outside of exporting data to a spreadsheet. The fallback is for business teams to make a request of their data team for a new report or analysis, which creates analytics bottlenecks.
Next, advanced data teams are usually siloed off from business users. However, what is needed is tighter collaboration between these two groups. Getting advanced insights is an iterative process that works best when those who know the business and those who know the data can work together and break down the language and technology barriers that make it difficult to turn raw data into insights that can be used to make informed data-driven decisions.
Finally, there is a data problem. Data is getting more complex every day. A recent survey from ChaosSearch found respondents spend almost as much time prepping data (over 6 hours/week) as they do analyzing it (over 7 hours/week). Not only is this a waste of resources (because these processes can be fully automated), but the result is slower, less-informed decision-making that directly impacts a business's bottom line.
What is the modern data stack? How can it help organizations make better data-driven decisions?
The modern data stack is an amalgamation of tools utilized for centralizing data, managing data, and making data accessible to end user applications. The rise of the modern data stack has been driven by the adoption of cloud data warehouses and lakehouses, followed by an ecosystem of tools to optimize the use of these data stores. Tools in a modern data stack are characterized as cloud native, available on-demand for DIY data experts, and subject to usage-based pricing, all of which allow customers to choose best-of-breed tools and prevent vendor lock-in.
The modern data stack is comprised of the following layers: data sources, ingestion, storage, transformation, and analytics.
Data sources and applications are defined by applications and databases. These sources feed the ingestion layer, which extracts data through automated pipeline tools. Following ingestion, the data must be stored. Once the data has been stored, transformation is necessary to clean up raw data to enable analytics. Several companies specialize in the transformation layer. At the top sits analytics, which is defined by business intelligence platforms such as Looker, data science platforms such as DataRobot, as well as a new breed of modern decision intelligence tools.
As new analytics tools such as decision intelligence emerge to further extend the power of the modern data stack, organizations will have an easier time leveraging all the data at their disposal. Innovations such as automation, AI, and ML in analytics tools are helping organizations generate smarter, more comprehensive insights that empower users to make critical decisions faster.
What is decision intelligence? Can you provide some use cases?
Decision intelligence has rocketed from obscurity to rank as a Gartner Top Strategic Technology Trend for 2022, which predicts it will be practiced by 33 percent of large organizations by 2023. Decision intelligence takes yesterday’s business intelligence dashboards and complex data science methodologies and augments them with automated, explainable insights in a unified user experience. Decision intelligence allows anyone, business users and data scientists alike, to access descriptive, diagnostic, and prescriptive analytics to understand what is happening in their business, uncover the reasons why metrics change, and get recommendations to optimize business outcomes.
Decision intelligence leverages analytics queries and machine learning techniques using disaggregated data to reveal deeper, more granular insights. Companies across industries, from financial services and life sciences to consumer packaged goods (CPG), are using decision intelligence to make better, data-backed decisions related to sales and marketing performance, supply chain, HR, and pricing analytics, to name a few
For example, one global CPG company was concerned with a drop in its product’s market share. With decision intelligence, the team was able to search the company’s data with questions such as, “What is happening with our product’s market share week over week?” Decision intelligence would then provide detailed trends outlining key drivers impacting market share based on millions of data points, which showed the drop was the result of a changing purchasing habits in specific customer segments. The team was then able to execute targeted marketing campaigns for these segments. They also leveraged AutoML to build, train, and deploy a predictive model to classify high propensity buyer segments to expand the marketing campaigns and drive more sales.
How does decision intelligence differ from existing tools such as business intelligence?
Decision intelligence differs from the likes of business intelligence (BI) in three major ways. First, BI is designed for data consumers such as business users, whereas decision intelligence is suited for both data consumers and analytics creators such as analysts and data scientists.
Second, BI is built for descriptive analytics to answer a limited set of questions from aggregated data. Decision intelligence, on the other hand, expedites the analysis of the underlying “why” and “how” to improve by applying statistical techniques and machine learning to generate granular insights.
Finally, BI lacks the proactive intelligent automation that defines decision intelligence from natural language query/processing, automated visualization and insights, automated data prep, and AutoML.
When compared to manual analysis, coding in SQL and Python, and technical data science machine learning (DSML) tools, decision intelligence is built to enable both data creators and consumers. Manual analysis and DSML tools are better suited for predictive and prescriptive analytics to model the future and identify ways to improve outcomes, whereas decision intelligence expedites all forms of analytics (descriptive, diagnostic, predictive, and prescriptive). DSML tools are growing more in their automation in the form of AutoML, but many are still driven by heavy manual analysis, whereas decision intelligence is marked by intelligent automation.
There are so many analytics solutions on the market today. How can organizations identify which ones are going to meet their unique needs? Which features and capabilities should they look for?
According to the Data Literacy Project, only 24 percent of the workforce is confident in their ability to read, work with, and analyze data. This 24 percent is largely made up of resource-strapped data scientists and IT teams. As a result, it’s become even more apparent that organizations will never become truly data-driven until they can ensure effective collaboration between data creators (such as data scientists) and consumers (non-technical business users).
Unfortunately, it is not easy to find a solution that can cater to both creators and consumers. Many tools lean towards expert data scientists or lack the technical depth for a citizen data scientist. To that end, companies should deploy tools that support intelligent automation to expedite the creator’s analysis and enable self-service analytics to broaden adoption of information consumers. Furthermore, tools should live query data warehouses and data lakes to enable real-time analysis.
Ultimately, collaboration between creators and consumers is critical to reducing time between iterations and it accelerates speed to insights, so look for tools that can also promote data exploration and discovery. Organizations can augment exploration and discovery with automation to expedite insights as well as AutoML capabilities to help deploy models that help predict outcomes. Companies can also focus on making models more accessible through explainable AI and natural language search capabilities, which will further enhance collaboration.
How can organizations effectively introduce modern analytics tools to their employees? What steps should they take?
Introducing a new modern analytics tool should be strategic. It’s best to start small by identifying a few specific use cases that could benefit from decision intelligence. After deploying the tool across a single team, companies can start to think about how to extend the power of the platform to other departments and analytics challenges their organizations are facing.
Beyond starting with a few easy use cases, business leaders also need to focus on training and improving the skills of employees (so-called upskilling) to leverage new analytics tools to their fullest capacity. Not only will this ensure that teams fully understand how to leverage the new technology their company has deployed (and thus ensure a higher return on investment), but upskilling has also been proven to enhance hiring and retention efforts. In fact, roughly half of American workers would switch jobs if they were offered more upskilling and training opportunities.
Basic data analysis and statistical skills are the leading attributes to advance anyone’s career because more companies are seeking prospects that can apply data to drive better business outcomes according to employment site, Indeed.com. Upskilling initiatives allow employees to learn new skills that create more seamless and intuitive workflows while fostering career growth.
What does the future hold for the analytics market?
First, enterprise-wide data literacy will be critical to becoming data-driven. Today, we are all accustomed to applications that interact in natural language and have simple tap interfaces, all of which provide personalized and contextualized information to us 24/7. We will carry forward this expectation to our analytics tools, which are quickly evolving to streamline complex data analysis and increase accessibility. Business users need to be able to analyze data on demand, seamlessly and painlessly, without the need to rely on data scientists or analysts.
Additionally, organizations will increase investments to further democratize data so that every employee can make informed, data-driven decisions faster. The data scientist shortage will increase the demand for innovations such as decision intelligence because existing data teams might lack the bandwidth to prep, analyze, and report on every query request. By democratizing both data and analytics and deploying tools that everyone within an organization can use, organizations can minimize disruptions from the limited resources they have.
Finally, the modern data stack will continue to evolve. Tools will continue to improve and make it easier for organizations of all sizes to extract greater value from their data assets. Innovations such as decision intelligence will put better data insights in the hands of more people.