23 By Candida Valois, Field CTO, Americas, Scality Legacy storage solutions won’t suffice for today’s artificial intelligence and machine learning (AI/ML) workloads. As AI/ML technologies continue to gain traction, a new model is needed. Here’s the problem: AI and ML require huge pools of unstructured data, including images, videos, voice and text. To uncover beneficial and actionable insights, applications need faster access to massive amounts of data, which is created everywhere: in the cloud, at the edges and on-premises. For these intensive workloads, linear scalability, low latency, and the ability to support different types and sizes of payloads are required. Enterprises need a fresh approach to data delivery that’s application-centric rather than location- or technology-centric. With the widespread adoption of AI/ML and analytics, enterprise IT leaders must make a major shift in the way they think about data management and storage. Addressing mixed workloads For AI and machine learning, you need a data storage solution that can handle different types of workloads, including both small and large files. You might be dealing with just a few tens of terabytes in some cases. In others, there are many petabytes. Not all solutions are meant for huge files, just as not all can handle very small ones. The key is finding one that can flexibly handle both. Scalability is crucial Organizations looking to grow in terms of capacity and performance are often hampered by traditional storage solutions that don’t allow linear scaling performance. Algorithms for AI/ML need enormous datasets to allow for proper training of underlying models that ensure accuracy and speed, which means this capability is crucial. AI/ML workloads need a storage solution that can scale infinitely as the data grows. This is where object storage shines. It’s the only storage type that can scale limitlessly, seamlessly and elastically to tens of petabytes and beyond within a single, global namespace. In contrast, legacy file storage and block storage solutions can’t scale past a few hundred terabytes. And what’s important about object storage compared with traditional storage is that it stores objects in a completely flat address space in which there are no limitations. Users won’t encounter the hierarchy (and the capacity limitations) of a traditional file system.. Addressing performance Scaling for capacity is important, but it’s not enough. You must also scale linearly in terms of performance. With many traditional storage solutions, scaling capacity comes at the expense of performance. So, when an organization needs to scale linearly in terms of capacity, performance tends to plateau or decline. In the standard storage model, files are organized into a hierarchy, with directories and subdirectories. This architecture works quite well for small volumes of data, but as capacity grows, performance suffers beyond a certain capacity due to system bottlenecks and limitations with file lookup tables. Object storage, on the other hand, operates quite differently. It provides an unlimited flat namespace so that by simply adding additional nodes, you can scale to petabytes and beyond. For this reason, you can scale for performance as you scale for capacity. A storage solution for our times When an organization adopts artificial intelligence and machine learning, it requires storage that can handle mass amounts of data to run and scale these initiatives. Scality’s enterprise-grade object storage software is purpose-built for these demands. Organizations can start their projects on a small scale, on one server, and easily scale out both capacity and performance as needed. Fast object storage brings essential performance to the analytics applications these initiatives need, too. Object storage also provides the needed flexibility from the edge to the core and offers complete data lifecycle management across multiple clouds. This approach enables applications to easily access data on-premises — even in multiple clouds — so data is efficiently processed. Fast object storage offers the features enterprises need: low latency, the ability to scale linearly and the ability to support different types and sizes of payloads.
