Achieving Success with Modern Analytics (Part 1 of 2)
Fern Halper, TDWI’s vice president and senior director of research for advanced analytics, discusses her recent TDWI Best Practices Report on achieving success with modern analytics -- including exciting use cases and common challenges organizations need to overcome.
- By Upside Staff
- August 24, 2023
In a recent “Speaking of Data” podcast, TDWI’s Fern Halper discussed her recent TDWI Best Practices Report: Achieving Success with Modern Analytics. Halper is vice president and senior research director for advanced analytics at TDWI. [Editor’s note: Speaker quotations have been edited for length and clarity.]
Halper began by explaining that TDWI operates on a fairly expansive definition of “modern” analytics. It includes advanced data and analytics techniques such as machine learning and natural language processing, as well as newer self-service tools that promote data democratization.
“Not everything that we consider modern analytics is new,” she added. “Some things, such as machine learning, have been around since the 90s but are being used in new ways. For example, automated and augmented tools that allow for natural language query. These weren’t around five or 10 years ago.” This is not to mention the obvious new technologies such as generative AI (a subset of machine learning) and deep learning for images -- things organizations may have wanted to do but did not have the computing power to achieve.
Halper went on to describe some of the more exciting use cases for modern analytics.
“The report breaks these use cases down into three categories: industry-specific, operational, and sales and marketing,” she said. “One exciting example of an industry-specific use case is disease detection in healthcare.” Halper offered other examples from industries such as retail and finance such as predicting churn or detecting fraud. Operational use cases she mentioned included predicting no-shows for appointments and predictive maintenance.
Sales and marketing, she added, are making especially good use of innovations in analytics and AI.
“I’m sure most people have heard of ChatGPT by now,” Halper said, “and we’re seeing a lot sales and marketing teams using it and similar technologies for tasks such as content creation and customer service chat bots.”
Organizations are using several other tools as well.
“About 60% of the people we surveyed who said they were using advanced analytics said they were using machine learning,” Halper explained. “About 40% said they were using natural language understanding (a combination of natural language processing and generation).” Respondents were also interested in the newer generative AI technologies, but where about 35% had plans to use them in the near future, only half as many said they were currently using them.
When asked about the challenges organizations were facing, Halper explained the report looked at the issue in two ways.
“We surveyed organizations that said they were using modern analytics in one group and those who were trying to implement modern analytics but hadn’t yet gotten there in another group,” she said. Those in the second group said their main challenges were organizational -- lack of skills and funding were cited as primary concerns by nearly 40% of the respondents. However, both groups reported facing technical challenges such as slow data access (roughly 30% of each group).
Halper also noted only a small percent said that lack of executive support was an issue. “We have seen in our assessments that even though executives say analytics is valuable, they often don’t support it with funding.”
Halper was then asked what distinguishes successful companies from unsuccessful ones.
“In our survey, only 25% identified themselves as successful, so it’s a fairly small sample. However, we did see some common traits,” she said.
Successful organizations were more likely to:
- Have a committed analytics leader
- View data literacy as a priority
- View the cloud as a priority
- Use automated tools
- Have analytics governance in place
- View data monetization as important
Halper acknowledged the question of whether a company has to do all these things to be successful.
“I think the answer is yes, though not immediately,” she said. “It’s a virtuous circle. If you have some of these things in place, it builds on itself and you can do more.” She advised organizations to start with even just one project and carry it through successfully. This will get people excited and lead to more projects.
Halper noted a few other interesting results from the survey. For example, she said, it was the first time a TDWI survey has shown more people using a cloud data warehouse than an on-premises data warehouse. “We’ve known the cloud was mainstream for a while, but this may indicate the tipping point.”
The other result that surprised her was how many organizations aren’t measuring the right things. Regardless of whether respondents reported being successful with analytics or not, when asked what they were measuring, there were many cases where measurements and KPIs weren’t actually tied to business objectives. “Tying analytics success to business success is a key part of the virtuous circle I mentioned before,” Halper added.
Finally, Halper had one additional piece of advice to offer those still struggling to implement a modern analytics program.
“Communicate,” she said. “Communicate what you’re doing to the business stakeholders so they understand what you’re doing and can help you measure the right thing or answer the right question.”
[Editor’s notes: To hear the full conversation, replay the podcast episode here. To download a copy of the TDWI Best Practices Report, visit https://tdwi.org/research/2023/06/adv-all-best-practices-report-achieving-success-with-modern-analytics.aspx. The conversation continues in part 2 of this article, available on demand.]