Scality announces Splunk SmartStore-ready validation for multi-site architectures

256

In a perfect world, compute and storage resources would scale in lockstep as application usage and data grow. System administrators would be able to simply add resource blocks in a linear fashion as their environments expand. As easy to manage as that perfect world would be, it’s not today’s reality — global data growth is forecast to triple to 181 ZB by 2025. This kind of exponential growth is especially true for successful Splunk deployments where teams aggressively expand their storage consumption to make data-defined decisions.

How Splunk Enterprise with SmartStore increases efficiency and flexibility while reducing TCO

The team at Splunk has embraced the demand for large amounts of accessible data by introducing the SmartStore feature set. With SmartStore, storage is decoupled from compute, allowing these resources to be scaled independently and, therefore, more efficiently. This flexibility allows administrators to scale their environments in granular fashion as business needs and data demands change over time. The resulting TCO reduction is further compounded by higher utilization of both compute and storage resources. 

Another way SmartStore reduces storage costs is by offloading older, less frequently used data (warm buckets) to external storage like Scality RING — without impacting search performance. Scality RING holds the master copy of warm buckets while indexers’ local storage is used for hot and cache copies of warm buckets. With most data residing on Scality RING, the indexer maintains a local cache that contains a minimal amount of data: hot buckets, copies of warm buckets participating in active or recent searches, and bucket metadata. This renders indexers stateless for warm and cold data, boosting operational flexibility and agility.

Scality RING for Splunk SmartStore single and multi-site deployments

Software-defined Scality RING has supported the Amazon S3 API since 2010 and is an ideal storage tier for Splunk SmartStore. RING provides Splunk customers petabyte-scale storage for SmartStore single-site and multi-site deployments.

To give customers options and flexibility in their SmartStore deployments, Scality offers multiple data protection schemes:

• Single-site architecture with erasure coding protects against the loss of disks and even entire servers.

• Multi-site active-active architecture with erasure coding as well as asynchronous replication between two sites protects against the loss of disks, servers, and even an entire site. Active-active is best suited for multi-site Splunk deployments where inter-site latency goes up to 100ms.

• Multi-site stretched architecture with erasure coding as well as synchronous replication across three sites protects against the loss of disks, servers, and even an entire site. With a stretched architecture, Splunk customers’ experience is akin to what they would have on AWS within an Availability Zone. A stretched architecture is best suited for multi-site deployments where returning consistent and complete results is mandatory, even during a data center failure event.

Uniquely validated as a SmartStore-ready storage solution for Splunk deployments in all three architectures, Scality is the only on-premises storage vendor that can provide both active-active and stretched architectures, which gives Splunk customers unparallelled flexibility and freedom of choice.

A stretched RING provides multiple benefits, including full site-level failure protection, access from all data centers, and dramatically reduced storage overhead compared to mirrored RINGs. The data protection scheme for a three-site stretched RING provides protection against one complete site failure, or up to four disk/server failures per site, plus one additional disk/server failure in another site, with merely 70% space overhead. 

This compares favorably to a replication policy that might require 300-400% space overhead for similar levels of protection across these sites. Finally, thanks to zero RPO and RTO, a stretched architecture guarantees that all Splunk searches always return complete results even in the case of a data center failure.

Scality and HPE partner to test and validate multi-site active-active Splunk deployment

Scality partnered with HPE to test and validate the two Splunk multi-site architectures, which are explained in more detail in this video by Bharath Ramanathan from HPE’s Solutions Marketing team.

https://www.youtube.com/watch?v=DEGYpXa0IX4

Following the Splunk SmartStore philosophy of providing flexibility and cost optimization, HPE’s disaggregated hyper-converged infrastructure solution (dHCI) based on HPE Nimble/Alletra 6000 was utilized for this validation effort. HPE dHCI provides the ease-of-use of HCI solutions while at the same time allowing for independent compute and storage scaling — all based on the latest generation of cloud-managed HPE Alletra 6000 systems. 

The Scality RING deployment leveraged HPE Apollo 4000 series systems that provide leading capacity density and flexibility of media choices ranging from NVMe to SSD, to large and small form factor disk drives in a number of combinations. 

HPE’s storage workload solutions lab is committed to the creation and validation of optimized workload-centric solutions that leverage best-of-breed components. 

The validated configuration is available through HPE Greenlake for a cloud-like experience on premises. 

---

Authors

Additional Resources

• Splunk SmartStore documentation
• Scality listing in Splunk partner directory
• Splunk page on scality.com
• HPE storage solutions for Splunk
• Read Press Release
• Get the Evaluator Group Report