TDWI | Training & Research | Business Intelligence, Analytics, Big Data, Data Warehousing

Growth of Unstructured Data Tops List of Data Trends of 2022 and 2023

Data storage, especially of unstructured data, weighed on enterprise minds this year. What does that portend for next year’s data trends?

• By Carl D’Halluin
• January 13, 2023

It’s been a big year for data, especially as enterprises collect increasing volumes of unstructured data. Let’s review the top three trends from last year that stand out and what’s on the horizon in the world of data management in 2023.

For Further Reading: Executive Q&A: Getting the Most from Unstructured Data 3 Use Cases for Unstructured Data The 3 Vs and Unstructured Data Analytics

Trends of 2022

Trend #1: Continued accumulation of unstructured data forced many organizations to consider new approaches to managing the growth and life cycle of data

The continued amassing of unstructured data continued to challenge companies of all sizes. Many started to notice that although it may have taken years to accumulate their first petabyte of unstructured data, they very quickly collected three petabytes, then five, then 10, and so on. As the data growth rate continued to accelerate exponentially, companies faced rising challenges related to cost control, risk reduction (from threats such as ransomware, insider threats, and human error), and opportunity loss. In addition, as data aged, people in the organization had less knowledge of what has been stored where and who owned it, a problem exacerbated by employee turnover.

Trend #2: Unstructured data storage on NAS systems was still prevalent

When it comes to unstructured data, you’ll still find much of it stored locally on network attached storage (NAS) systems. Many applications -- whether provided by independent software vendors or developed in-house -- are built to read and write data using NAS protocols. When a company replatformed its application to the cloud using object storage, it faced a completely different paradigm. This required an application rewrite using object storage APIs, a task few enterprises had the resources to undertake. As a result, large volumes of data still resided on these on-premises systems in 2022

Trend #3: Major cloud service providers increased the number of vendor-branded file services

Given that enterprise customers are unwilling or unable to rewrite existing NAS-based applications to operate with cloud object storage, public cloud providers formed more relationships with traditional on-premises NAS vendors to offer branded cloud-based managed file services.

The managed file services provided customers three main benefits:

• They have a simple on-ramp to migrate their legacy application to the cloud provider’s environment
• Investments made in training, documentation, processes, and administration of the traditional NAS systems are preserved, meaning the enterprise customer does not have to bend their business to adopt the NAS storage services offered by the cloud provider
• As a managed service, the cloud provider takes care of the underlying hardware and operating environment so the customer can simply consume the storage service

Data Trends to Watch in 2023

Trend #1: AI/ML, IoT, and new apps will be the primary drivers for data accumulation

Data accumulation will increase in 2023 driven by the need to hold onto data virtually indefinitely to feed into AI/ML processes. This AI/ML hunger for data is why companies are investing heavily in data collection. The use of this data and the intelligence that can be obtained from it offers the promise of maximizing companies’ chances for growing their business and creating a competitive advantage.

Today and into the future, just about every kind of device will be generating a huge amount of data (think IoT). Current and new apps are also generating more data, plus there are more end users. Think about how many devices you have now and multiply that by the entire population of device users. Additionally, people are creating more content through more applications, which is adding fuel to the data fire. As these and other data volumes grow, it will become increasingly critical to protect and streamline data management, all while controlling costs and risk.

Companies currently have a simple choice: store their data on premises or in the cloud. However, in the coming year, there will be a variety of other conditions to factor in, such as whether they’re employing edge computing (where data is collected at several sites on the network edge) or a greater number of remote work sites. Companies will need a solution that helps them wrap their arms around everything in the environment, which is far different than just having to manage main storage systems in their data centers.

Trend #2: Managing storage costs will become a higher priority

Data’s value changes over time. As you roll the clock forward, there's less-frequent access to the original content; eventually, a data set might hardly, if ever, be accessed for most enterprises. In some highly regulated industries -- healthcare and financial services, for example -- “old” data cannot automatically be deleted but it also can’t be kept on primary storage because the cost would become unmanageable.

In 2023, enterprises will seek to control storage costs by reviewing their data migration policies to spell out what to do with data as it ages. Ideally this review will lead to implementing automated data storage and management that’s cost effective and accessible.

Trend #3: AI and ML will analyze data trends and characteristics to provide proactive (and possibly automated) unstructured data management

Apart from being a great consumer of vast amounts of unstructured data, AI/ML algorithms will be deployed to help organize and optimize storage of unstructured data. Many companies are wondering how to create data pipelines for pushing raw data to other storage systems (such as the cloud) for consumption by an analytics application. In 2023, we’ll see an even greater emphasis on ML and AI as organizations seek ways to take their data and run ML algorithms against it.

For example, we will see automated solutions that allow customers to first understand their unstructured data environment before committing to take any necessary actions required by the business and/or IT. Reporting will be tailored and optimized to help users make clear decisions according to their business priorities. Once they have the visibility, they can organize the data according to multiple criteria such as the data’s ownership and role, where it belongs, its risk profile, and the type of action to take on it.