A large research and life sciences organization was looking to reduce storage costs by replacing their Isilon SyncIQ replication solution. To eliminate their dependency on hardware, it turned to Igneous and Google Archive. The result? 4PB of data migrated to the cloud in just weeks and a massive 64% reduction in storage costs. The company is now committed to Igneous for strategic management of their remaining multiple 10’s of petabytes of data across on premises and Google cloud storage.
So is it really possible to save storage costs by moving to the cloud? You hear stories about organizations getting huge unexpected bills from the cloud providers because they (or their vendors) didn’t take into account operational (or transaction) costs. How does Igneous manage that? It’s all about paying attention to the details, and starts with choosing the right storage class for your workloads.
Google’s coldest storage class (or tier) is Archive and, like the other guy’s coldest tiers, when used correctly for storage it can be very cost effective - much better than tape. Yes, Google Archive may cost a couple hundredths of a penny more per GiB/month than the other guys, but if you ever need to retrieve your data, Google will start sending it to you in seconds. The other guys will start sending your first bytes in hours and, depending on how you use your data, that might not be soon enough. Be aware that in Google’s Archive class,you are committing to pay for a year of storage for every byte whether it stays there or not. For those who know they will save their data for at least a year but will eventually want to retrieve it RIGHT NOW, Google’s Archive class should be on your radar.
So what about those operational costs? The “at rest cost” is only part of the picture. Google, like the other guys, will charge you for every operation performed on the data. Operations include puts, gets, updates, listings, copy - anything and almost everything. Those costs are 10x - and in some cases 125x - the cost of the same operations in the warmest classes! Understandably, this is where a lot of people get nervous about cloud storage. Particularly when you are storing a lot of data. (We consider a lot of data being north of 500TB and/or millions or billions of files - when you have that much data, the numbers get big in a hurry.)
So what do you do?
To get smart about how you manage your data and it’s storage, you need to pay attention to the details around each class of storage for each cloud vendor and work to minimize those peripheral costs that can make cloud storage expensive. This is not particularly easy but at Igneous we do it every day for petabyte-scale customers.
Minimize Transaction costs
Igneous significantly reduces operations costs by bundling individual files into “blobs” during the backup/archive process, and then compressing the resulting data before sending it to cloud storage. With data blobs, large numbers of files - in some cases thousands of individual files - can be moved in a single transaction. Recent tests show that we can reduce operations costs by an average of 95% [see Taming Cloud Costs]. Compression improves performance by reducing the amount of data that must be moved and saves money on cloud storage costs.
Identify the optimal time to delete data
We also pay attention to the age of the data stored in the cloud. Using a complex algorithm, we automatically determine the most cost-effective time to delete data. It might be counter-intuitive, but it actually makes more sense to wait to delete data than to delete it on demand. Igneous maintains an index of everything you have archived, so you can “delete” data so that you can’t get to it (this is in line with GDPR and CCPA requirements) but we won’t actually delete the bits from the cloud until it is economically advantageous to do so. You won’t get a big bill because bits were deleted before their time.
Restore data cost-effectively, too
If you ever do need to restore something from your archived data, Google will be able to start sending it back to you immediately and we will intelligently pull the data in the most cost effective way by saving on transaction and network egress costs.
Igneous is paying attention so you don’t have to
Google’s Archive class of storage can be really cost-effective if you pay attention to the details. But that’s hard. Let Igneous do it for you. You save money on storage and let us sweat the details. You can go do something more interesting like reading our True Cost of Backup whitepaper.