Blog

Subscribe Here!

Data Management: Why Does It Matter and Why Is It So Hard?

by Catherine Chiang – October 23, 2017

With enterprise datasets growing so quickly, it’s not just about storage anymore. Enterprises need effective data management strategies to harness the value of their data.

On Thursday, October 19, we hosted a CrowdChat on Data Management: Protection, Movement, Search and Discovery, and Usage. Let’s dive into the insights shared by our panel of experts and industry leaders.

What is Data Management and Why Does It Matter?

The CrowdChat kicked off as moderator John Furrier of SiliconANGLE asked participants how they would explain data management to their grandmothers.

Igneous’ VP of Product Christian Smith compared managing data to receiving mail. “It's like mail (as in postal service), it shows up, you look at it, store some for later, keep some forever, and delete the junk. Later you wonder why you have boxes of the stuff,” he said.

Rubrik’s Andrew Miller said, “[Data management] can also be described offensively and defensively.” Managing your data defensively means you “protect your data [and] mitigate risk/liability around legal and discovery. Offensive [data management allows you to] extract competitive value from it [and] accelerate your knowledge of your customers/sector/etc.”

Igneous’ CMO Steve Pao added that when it came to ‘offensive data management,’ he considers it to be “adding business value rather than protecting business value.”

Why is Data Management So Challenging?

The conversation shifted to the difficulties of implementing effective data management, beginning with where data lives.

The common thread among participants was that data lives everywhere and always in multiple places, complicating enterprise data management. Some of the problems participants cited with managing data across silos included security, costs, and compliance.

Another looming issue for enterprises is GDPR compliance, which is only further complicated because “data is [scattered] in so many places,” according to George Crump of Storage Switzerland.

Protection

When it comes to protecting all of this data, most participants agreed with Storage Switzerland’s George that enterprises will “look for an all-in-one [solution] but end up with a portfolio.”

Rubrik’s Andrew said, “To be truly holistic, it's likely a portfolio approach simply due to all the different levels in the stack right now. Some companies have grand aspirations there (mine included) but there's so many potential data layers to handle.”

Datos IO’s Michael Colby noted “a transition from legacy solutions and architectures to cloud friendly solutions and architectures for protecting data.”

However, “real innovation” will use automation and minimize human interaction, according to Igneous’ Christian.

Movement

Moving data from where it lives to where it’s needed is another essential component of data management, but can present significant challenges when there’s so much data.

Igneous’ Christian pointed out that data movement as part of protection strategy is an old problem, but the new problem is “the multitude of locations where this data can go (3 public clouds, multiple datacenters).”

Christian continued, “New tools are needed to manage this data movement while keeping track of where it lives. [The secondary] problem [is] understanding when someone acts on secondary copies and tracking change back into ‘master.’”

Jeff Dinisco of P1 Techologies pointed out that while “[bringing] data to burstable compute is a great concept” and “works really well with known and manageable data sets,” it’s “very tough when working sets are hard to identify.”

The development of cloud has changed the landscape of data movement as well. Datos IO’s Peter Smails said, “Hybrid cloud [is] definitely the focus. Apps [are] no longer monolithic, going to micro-services. Bursting, test/dev, instantiation, archiving, BI [are] all legit use-cases spanning hybrid cloud.”

Search and Discovery

For businesses to extract value from their digital assets, they must not only protect and move their data but also be able to search and retrieve it easily.

Igneous’ Christian said, “Enterprises are starting to realize the value of search, [but] the tools are built for [terabytes] not for [petabytes].”

Unfortunately, search and discovery are often overlooked when enterprises implement data management solutions. “No enterprise pays for search. They pay for data protection and hope search is a part of the solution,” said Christian.

Usage

The conversation about usage revolved around applications in machine learning and data transformation workflows.

Many participants agreed that they would like to see more action and less talk regarding machine learning, with Chris Dagdigian, co-founder of Bio-Team and industry leader in the field of Bio-IT, saying “ML/AI are still very much in the overhyped phase in my limited niche” and Datos IOs’ Michael saying it seems like a “buzzword.”

When machine learning is used, its applications have been limited. Chris noted that “Data transformation is starting to use serverless and lambda-like design patterns.”

Igneous’ Christian added that machine learning is “used heavily in specific sectors - data is the key to what they do. It's still nascent for the average enterprise and services by cloud providers in ML and not understood. It exists in the corners where really smart people (PhDs) can play.”

When tying it back to data management, “data management must be integrated with event-driven computing and microservices. In AWS terms, S3, lambda, and elastic container services are needed to make it come together for large datasets,” according to Igneous’ Steve.

Learn More

Read the entire CrowdChat conversation below. To learn more about data management, read CMO Steve Pao’s Infoworld article on the core functions of data management.

Related Content

Top 10 IT Trends for 2019

February 19, 2019

In 2019 and beyond, 451 Research sees a key shift in the world of IT—the breaking apart and coalescing of old silos of technology. Today, technological advances feed off each other to drive innovation. With this new paradigm of technological innovation, 451 Research shares 10 IT trends they predict for 2019.

read more

“Interesting Times” for Unstructured Data Management

January 10, 2019

The expression “may you live in interesting times...” is subject to much debate. To some it is a celebration of the the opportunities to be found in times of transition. To others, it is a cautionary phrase that should be heeded to avoid misfortune. No matter which side of these interpretations you find yourself aligned with, there is no question that 2019 will be a year of significant opportunities and challenges for those responsible for the proper care, management, and stewardship of unstructured data.

read more

8 Principles for a Better Data Management Strategy

December 5, 2018

I’ve spent the better part of three decades leading one of the most demanding high-performance computing infrastructures in the world. One of the greatest challenges of HPC infrastructure is keeping data available and meeting the needs of the business with supporting engineers located in dozens of locations around the world. Here are some key takeaways for anyone struggling with this problem.

read more

Comments