TDWI | Training & Research | Business Intelligence, Analytics, Big Data, Data Warehousing

CEO Q&A: Why the Modern Data Warehouse Is Critical to Your Analytics Success

Data warehouses designed to handle today’s big data challenges, enabled by flash memory, are helping enterprises stay on top of an avalanche of data. Yellowbrick CEO Neil Carson explains.

• By James E. Powell
• February 8, 2019

What's the key to an enterprise's data strategy? In this new Q&A series at Upside, we asked CEOs of up-and-coming companies -- in this instance, CEO of Yellowbrick Neil Carson -- to explain what technology or tool is most important and why.

For Further Reading: Why Data Warehouse Modernization Must be Coordinated with Other Modernization Projects Modernizing a Data Warehouse with Real-Time Functions Long Live the Traditional Data Warehouse

The company's Yellowbrick Data Warehouse delivers large scale, high-performance, cost-effective SQL analytics, so it's no surprise that Carson emphasizes the importance of the modern data warehouse. What may be surprising is his opinion that Hadoop, seen by many as a leading-edge technology that helps enterprises stay competitive, may be more trouble than it's worth.

Upside: What technology or methodology must be part of an enterprise's data strategy if it wants to be competitive today? Why?

Neil Carson: We see a modern data warehouse as the foundation of an effective analytics strategy. As business decisions across teams become more data driven, this puts more load on the data warehouse and data marts. It has to deal with far more concurrency and mixed workloads with real-time data. It has to be cost-effective, correct, and reliable so you can support more and more users.

What one emerging technology are you most excited about and think has the greatest potential? What's so special about this technology?

Flash memory has the most impact on the data and analytics industry. It's the largest, most cost effective, most dense, and highest-bandwidth storage medium. Flash memory has completely revolutionized the storage market but not data warehouse architecture. All other on-premises and cloud data warehouses are architected for spinning discs.

What is the single biggest challenge enterprises face today? How do most enterprises respond (and is it working)?

The biggest challenge is the top-level pressure to move everything to the cloud. However, this isn't always practical or cost-effective. The world will be a hybrid. An enterprise data warehouse has to be deployable on premises and in the cloud -- not just one place or the other.

Is there a new technology in data and analytics that is creating more challenges than most people realize? How should enterprises adjust their approach to it?

Yes, Hadoop is still adversely impacting the data warehousing industry, in many cases creating more problems than it has solved. It's a scalable data store, but it is also poor at data processing. Hadoop is not a database; as a result, its reliability, manageability, predictability, stability, concurrency, consistency, performance, and even SQL compliance are all a step backwards for the industry. The only benefit it provided was scale, but often the operational costs dwarf the infrastructure savings anyway.

Now we see a resurgence in scalable and high-performance data warehouses. These newer data warehouses can handle big data volumes cost-effectively yet don't have any of the Hadoop shortcomings.

What initiative is your organization spending the most time and resources on today?

As a data warehouse provider, we spend our time working on behalf of our customers and their most difficult data management challenges.

The initiatives we spend the most time on include: managing complex mixed workloads; supporting high concurrency; integrating with more and more ETL, BI, and data mining tools; and providing the best 24/7/365 support our customers have ever seen. We continue to ensure every Yellowbrick customer -- some of the world's largest enterprises in insurance, credit cards, telecommunications, hospitality, and retail -- are happy and willing to provide references.

Where do you see analytics and data management headed in 2019 and beyond? What's just over the horizon that we haven't heard much about yet?

In 2019, we'll see a complete rethink of the data warehouse driven by three factors:

• Traditional data warehouses are being retired by vendors or have become prohibitively expensive
• Hadoop is no longer a viable strategy for mission-critical operational analytics
• Cloud-only solutions are not providing the flexibility enterprises want and deserve

Beyond, we'll see a new wave of predictive analytics applications coupled with easy business intelligence when the following happens

• Higher-capacity data volumes are used to retain more current and historical data
• More compute power enables query performance not previously possible for new insights and discoveries
• Handling of mixed workloads with real-time data ingest and large numbers of users allows analytics to permeate throughout the enterprise

Describe your product/solution and the problem it solves for enterprises.

The Yellowbrick Data Warehouse delivers large scale, high-performance, cost-effective SQL analytics. It handles mixed workloads, real-time data, and a large number of concurrent users to enable a new set of analytics capabilities. The world's largest insurance, credit card, telco, and retail companies trust Yellowbrick to run their enterprise data warehouses. Yellowbrick integrates with existing BI, ETL, data mining, and analytics tools. Our product is deployable anywhere, from the data center to the cloud to the edge.