Eliminating Need to Copy Data from Storage to Compute Clusters
Copying data across a network from storage onto a compute cluster can take hours," says Konstantin Gredeskoul, CTO of online shopping community Wanelo.
"Joyent Manta Storage Service strips the need to invest any time moving the data around, making ad hoc querying and analysis near-instantaneous, seamless and cost effective. We are now able to perform complex cohort analysis and retention reports across hundreds of gigabytes of data in a couple of minutes. When compared to traditional methods such as data warehousing, this is game changing," Gredeskoul says.
"Fifty percent of the world's smartphone traffic goes through Ericsson, and we are continuously evaluating new technologies to increase the ability of the network to manage growing data volumes in the most responsive, secure, cost effective ways possible," adds Vish Nandlall, Ph.D., CTO and head of Strategy and Marketing for Ericsson North America. "Joyent's new compute-on-storage innovation is a fundamental paradigm shift that changes the economics and utility of object storage and high-performance big data analysis."
The Joyent Manta Storage Service provides the following:
- A multi-datacenter object store with fine-grained replication controls.
- No object size limits.
- Strongly consistent writes and highly available reads.
- Per-object replication policies.
- A file system-like namespace, including directory queries.
Manta Derives Its Capabilities from OS-Level Virtualization
Cantrill notes that Joyent achieves these capabilities by focusing its efforts on OS-level virtualization instead of hardware virtualization. OS virtualization gives Joyent the power to spin up compute on the objects in the Internet-facing object store where the objects live.
"Virtual hardware is great if you need to support a legacy operating system," he says. "But if you want to deliver the highest-performance experience, you shouldn't virtualize hardware, you should virtualize the operating system. The application runs directly on the hardware but in a secure and containerized way."
"You bring your analytics tools to the cloud, effectively, he adds. "You can run whatever you want on top of your data—Java, Python, Perl or just a Unix Shell. If you've got your own analytics tools that are using R or what have you, you can bring whatever analysis software you've got, effectively, and you can push into the cloud and run it where your data is."
Joyent has partnered with data storage management company Panzura to help customers securely migrate their data from existing NAS, backup and archive storage to Manta.
Sign up for Computerworld eNewsletters.