TDWI | Training & Research | Business Intelligence, Analytics, Big Data, Data Warehousing

The Value of Leadership, Collaboration, and Setting Objectives

New TDWI research reveals the importance of good communication and collaboration and provides insight into how to get actionable results quickly.

• By David Stodder
• June 12, 2020

"It's not just the technology, it's the people." At TDWI we hear this often. It's easy to get caught up in the latest technologies or the dazzling amount of data that today's cloud platforms can handle. To be sure, adoption and implementation of technologies and cloud services will be critical to faster and more effective data-driven decision making. However, TDWI research has long found that organizations fall short of realizing the value of technology investments due to organizational and project management difficulties.

For Further Reading: Four Keys to Achieving Value Faster from BI and Analytics Is Collaboration the Critical Success Factor for Data Science? To Maximize Your DataOps Future, Take Advantage of These Four Trends

With the recent upheavals brought about by the coronavirus pandemic, the importance of good leadership, collaboration, and clear project objectives could not be more important.

In the 2020 TDWI Best Practices Report: Faster Insights from Faster Data published in January, we looked at how organizations are faring with strategy, culture, leadership, skills, and funding. Because most analytics and data management projects impact a range of stakeholders -- from executive leadership and line-of-business managers to IT developers, data scientists, business and data analysts, and frontline users -- developing good collaboration and communication among them is vital.

For the report, we asked research participants to rate how well stakeholders in their organizations collaborate to accomplish key steps in the life cycles of BI, analytics, AI, and data integration and management projects (see figure below).

Only a small percentage gave their organizations "excellent" ratings for any of the steps. Organizations surveyed appear strongest in identifying relevant data sources (65 percent combined "excellent" and "good" ratings) and identifying opportunities to achieve business benefits (61 percent combined). This suggests a relationship between the two: that is, stakeholders believe that if they can access and analyze certain data sources properly it will lead to better business outcomes.

Significant but smaller percentages of respondents gave their organizations good or excellent ratings for providing sustained leadership (47 percent combined excellent and good) and articulating strategy to attract project sponsorship (44 percent combined). Concerted leadership is key to keeping projects going and dealing with both short-term needs and long-term objectives.

Collaborating on Project Goals

Project teams need to include personnel who can interpret business needs and articulate strategic vision so leaders sustain sponsorship of projects and provide appropriate funding. Governance and data stewardship are also good reasons to include business-side stakeholders in project teams alongside developers and IT personnel, especially if the organization is democratizing data access and users are employing self-service BI and data preparation tools. Nontechnical users need to know governance rules and policies and will benefit from mentoring and stewardship.

Leadership often asks teams to quantify anticipated return on investment (ROI) or business outcome; however, only 33 percent of research participants view their organizations as either good or excellent in doing this.

In a related survey question, we asked participants how successful their organizations were at setting project objectives, measuring progress, and organizing themselves to achieve deliverables. Only 9 percent of organizations surveyed are very successful in identifying value measures and quantifiable objectives, while 26 percent are somewhat successful. Only about one-quarter (23 percent) are either very or somewhat successful in scheduling clear-cut deliverables from easiest to most difficult. The majority also fall short in defining strategy and following it within alignment with corporate objectives; only 38 percent are successful at this objective.

Analytics: Testing and Operationalization Challenges

The majority of organizations we surveyed appear to have difficulty fostering collaboration on important steps in developing analytics. Looking again at the figure, we can see that fewer than half regard their testing of prototypes and developing proofs of concept (POCs) as either good or excellent (45 percent combined excellent and good) and slightly fewer give high marks to collaboration on development and testing of analytics models (42 percent combined).

More than half of respondents give their organizations low marks on developing hypotheses to solve defined problems, which is important to connecting projects to specific issues where the data insights from analytics and AI are needed and can be tested; 30 percent regard collaboration on this step as "fair" and 23 percent indicate that it is "poor." The majority of organizations struggle with getting projects to the point of operationalization. Less than half see collaboration to drive adoption and operationalization as a strength, with only 10 percent checking "excellent" and 32 percent calling it "good."

Getting to Actionable Results

Leadership and collaboration can have an impact on how rapidly projects can generate deliverables that produce actionable results. In 2017, TDWI asked research participants about the length of their development, testing, and deployment cycles. We asked the same question for the report published this past January to see if cycles had become any faster or slower.

The results are fairly consistent between then and now; the fastest 15 percent say their cycles are two weeks or less, which is the same percentage as in 2017; in both, the slowest 20 percent say their cycles take more than four months. Both then and now, 17 percent of research participants say their cycles take 15–30 days. For cycles that take beyond 30 days, the comparison shows that organizations are becoming a little faster, with more in the 31–60 day range than previously (18 percent for this survey compared to 15 percent in 2017).

Overall, however, it appears that the time it takes for development, deployment, and testing cycles to finish has not appreciably changed. One factor that may help reduce cycle times is the use of agile, DevOps, and DataOps methodologies.

When we filter the results to look only at organizations that are using these methodologies, we see that the bulk of respondents' cycles are under 90 days, with 34 percent of this group completing cycles in 30 days or less. Thus, organizations that have not yet implemented these methodologies should experiment with them on selected projects. Our research finds that they can be helpful in resolving organizational difficulties, improving collaboration, and clarifying how to achieve deliverables.

A Final Word

Experienced organizations know that as helpful as technology solutions can be, they alone cannot solve business problems or enable organizations to innovate faster in response to changing economic conditions. Human factors of quality leadership, collaboration, and project vision are essential.