"We move a lot of data around. Maybe not as much as the Googles and Apples of the world, but we're moving around a few petabytes each month as we offload and process data," said Carl Lucas, VP of Information Technology at Quantum Spatial.
Considered a leader in an emerging industry, Quantum Spatial’s reputation has led to substantial growth in both customer projects and of course, large amounts of data. In addition, new technology from sensor manufacturers continues to increase the resolution and quality of the images they capture. Satellites and planes capture data using a variety of technologies including Lidar, Digital Orthoimagery, HD Video, Hyperspectral Imagery, and Thermal Infrared.
“The amount of data that we get per flight from our sensor, platform, and satellite technologies is exploding. And image processing creates even more data, often 3 to 10 times the original flight,” said Travis Spurley, Senior Systems Engineer.
Quantum Spatial did not have an effective data management strategy for their unstructured data spread across three primary sites of Isilon and Dell EMC Unity hardware, plus 4 more remote locations. “As the only person managing petabytes of data across multiple sites, the risk was too high” said Spurley.
Some backup/recovery was handled by their own homegrown backup systems, prosumer devices, tape backup, and external drives - but the systems did not work together and took way too much time to manage. “It was all over the place,” said Lucas.
The impact? Quantum Spatial was having problems hitting SLAs for backup, and they were running out of primary NAS storage. Customer contracts had data retention obligations, but their users were hesitant to archive older data for fear of not being able to get it back. And through all of this, the data kept coming in fast.
With their limited datacenter space, Quantum Spatial knew they needed to incorporate cloud storage into their data
strategy. They needed a backup platform that could keep up with their growing data footprint, and they needed a well defined archive workflow that could quickly move all their old data to and from the cloud.
Quantum Spatial considered Dell VirtuStream as a cloud repository, but quickly realized the solution could not
perform at the scale they needed. They also looked at Rubrik and Cohesity, but “none of those vendors have a real
strong play in unstructured data,” said Spurley. Replication to a secondary Isilon tier was never a feasible alternative due to their limited secondary datacenter space. Quantum Spatial was already managing over a hundred Isilon nodes - where would they put additional hardware? How would they move data to the new Isilons? Who would manage it all?
The team at Quantum Spatial was stuck and did not know where to start. Thankfully, their technology partner
from Computacenter, Lindsay, suggested they look into Igneous as a solution specifically created to manage
unstructured data. “We needed an enterprise-class solution. We needed it to be easy to manage, scalable across multiple NAS platforms, cloud-friendly, and we needed it to perform," said Lucas.
We needed an enterprise-class solution. We needed it to be easy to manage, scalable across multiple NAS platforms, cloud-friendly, and we needed it to perform.
"Igneous never blinked at the size of our data or the number of files - they explained their approach and what to expect. It was a really refreshing experience," said Spruley.
Igneous DataProtect was originally deployed at Quantum Spatial across three primary sites with cloud capabilities. The new on premises system was up and running in a day, enabling teams to immediately hit daily protection SLAs across all their primary sites.
For Quantum Spatial, the key difference with Igneous has always been speed. Since their systems were expanding or already full, in order to reduce capacity, they needed the ability to consistently archive data. With Igneous, they are able to quickly archive data to the cloud, free up primary NAS for new projects, and retrieve data when it’s needed.
Initially, the Quantum Spatial team was “shocked” at how fast Igneous’ search functioned. During one meeting, the team could not believe it when Travis was able to search and find a specific file name across a petabyte worth of data in just seconds. “Super fast“ said Lucas.
Based on their early success with DataProtect for backup and archive, Quantum Spatial was confident Igneous could also solve their visibility problem. After Igneous enabled DataDiscover with no intervention, they were finally able to see and understand their entire unstructured data footprint.
Better data visibility for Quantum Spatial ensures their most critical assets are always protected in a business where data loss would be catastrophic. This has built trust between IT and the Geographic Information Systems (GIS) specialists behind their amazing data analysis. Data stakeholders can now confidently search and retrieve their projects according to agreed upon SLAs, which allows IT to confidently archive more data.
In addition, Quantum Spatial recently deployed the new software-only version of DataProtect. This allows them to better manage their three primary sites, plus all four remote sites by archiving and protecting data directly to Azure.
“We’re a technology company. We invest heavily into the systems that enable us to provide the right products and services to our customer base in a timely manner. Primarily, we’re investing in technologies that help us manage the massive data flow through our systems. Igneous provides that security blanket with visibility, backup and recovery,” said Lucas.
My engineers are very happy with Igneous. The ability to search across millions, if not billions of files is huge. The Igneous solution definitely improved our ability to do our job.
As a single vendor for visibility, backup, and archive, Igneous provides resilient software with the ability to hit aggressive protection and compliance SLAs. For Quantum Spatial, that means scalable, fast, and enterprise-class backup and archive to Azure for petabytes of data.
The Igneous systems are operationally efficient, managed by just one person at Quantum Spatial. And since it’s software, delivered as-a-service, they never have to worry about upgrading or new feature integrations. “Igneous monitors the system and lets me know if anything is happening. It helps me sleep at night,” said Spurley.
Igneous enables Quantum Spatial to optimize their limited datacenter space by building trust and best practices between IT and production teams. Now, when data is moved between locations or to Azure, the company is confident they will be able to get it back. “That’s pretty impressive given how protective our teams are over their data,” said Spurley.
Quantum Spatial is no longer running out of space. IT can now archive data to free up primary NAS for new projects
and retrieve it quickly when needed again. Some users describe this unique Igneous differentiator as temporary
storage or a “pressure release mechanism” across their multiple locations and cloud.
“My engineers are very happy with Igneous. The ability to search across millions, if not billions of files is huge. The Igneous solution definitely improved our ability to do our job”, said Lucas.
Igneous even enabled Quantum Spatial to keep an additional ‘highly space constrained’ office up and running
that otherwise would need to be replaced by a larger and much more expensive datacenter. “It was infinitely valuable
having raw capacity available in Azure that we could utilize. Igneous saved the day, “ said Spurley.
“It’s a very reliable system. We are really impressed with the quality of the engineering team at Igneous, specifically their willingness to listen to new feature concepts. We never have to worry about how we’re supposed to configure something,“ said Lucas.