Unstructured data has grown at an annual compounded rate of 25% for the past ten years, and shows no sign of slowing. For most organizations, “data management” for unstructured data has really just meant capacity management, i.e. increase capacity to keep up with data growth. This model worked at moderate scales, but as datasets have increased in size, complexity, and quantity, it has pushed the scales into petabytes of data with billions of files, and overwhelmed budgets. Enterprises are now asking for data management strategies that do more than just provide continuously increasing capacity.
To meet these new demands, Igneous was founded to create an entirely new approach to Unstructured Data Management as-a-Service (UDMaaS). By focusing on enabling data workflows for individual datasets—rather than the system or share—Igneous customers can have far greater insights into and control over how their data is protected. Additionally, a new user experience designed for archiving cold datasets enables IT to constrain storage consumption and control capacity growth. These enhanced capabilities paired with Igneous’ industry leading performance, as-a-Service delivery, and integrations with leading NAS and cloud providers empower organizations to gain more value from their data than ever before.
Management for Datasets
While IT admins group data at the system, share, or export level, end users think about their data in more granular units: datasets. Datasets are any group of data that can be manipulated as a unit. The files within a dataset are typically similar in value and are used together to serve some common purpose. A dataset could be an export, a group of directories, a single directory, or even a single file.
However, all datasets were not created equal. The most critical datasets may need to be backed up multiple times per day. Cold datasets may have been important at one time, but now should be archived to a more cost appropriate tier. Scratch datasets may only be useful momentarily, and should not consume any protection resources. Collaborative datasets might need to be replicated to a public cloud account. The list of management requirements varies greatly across industries and organizations.
The challenge for IT admins implementing a data management strategy is that these datasets with varying needs of protection can all live within the same export or share, and many data management solutions will force you to treat them all equally. Igneous now offers the ability to take action on these datasets, regardless of where they live. Igneous can take a list of file paths, or allow IT admins to browse into their systems to find the datasets they want to take an action on, whether it’s backup, archive, or ignore.
A New Approach to Archive
Backup and archive have been grouped together as long as organizations have needed both strategies. Tape backups would be held on-site until they were a certain age, then shipped to a salt mine, where they became archive copies. This is largely because moving data for backup and archive are extremely similar. The primary difference is that backups are typically done repeatedly, while archives are only done once after the data no longer belongs on the primary storage device.
However, the business case for archive is vastly different from the business case for backup. Backup is about protecting against unanticipated data loss. Archive is about optimizing storage costs to match the value of data. This difference has been highlighted in the new archive experience that will be coming soon from Igneous. The enhanced experience will be focused on finding datasets easily, taking action on many datasets simultaneously, notifying users when datasets can be removed, and easily discovering archived data where it lives today, regardless of where it came from. This emphasis will dramatically reduce the friction for IT admins to adopt a new archive strategy.
Additionally, Igneous will be offering end users direct read-only access via NFS to archived data. Many archive strategies have failed to live up to their promise simply due to end user’s fear that archiving data means their data is lost forever. By maintaining direct user access, end users will be able to confidently archive cold datasets without experiencing pain when they need to recall that data - as infrequently as that might be.
This new approach to archive will be coming soon for all Igneous DataProtect customers. As with all other software updates, these archive enhancements will be rolled out as-a-Service across the entire Igneous fleet. These updates are part of our continual improvements and updates to our product, which advance our commitments to pushing the forefront of Unstructured Data Management as-a-Service and to the success of our customers. We always welcome customer feedback to help us make our product even better!
Click below to learn more about the capabilities of Igneous DataProtect.