82 The Earth Observation Center in Oberpfaffenhofen, Germany uses remote sensing and custom analytic methodologies to tackle tough issues in environmental and climate science, mobility and planning, civil security and the prevention and management of natural catastrophes. The Photogrammetry and Image Analysis department, headed by Professor Peter Reinartz required a high performance, durable storage solution to store its satellite image data. After evaluating the available storage solutions, Professor Reinartz’s team chose the Scality RING storage. The Center’s satellite images are currently stored on the RING and accessed for processing and analysis. The Earth Observation Center has used these images to model topography, land use patterns and additional geo-related topics. They have even used the images to support a mountain climbing expedition by developing a highly accurate 3D model of the Karakoram mountain range, home of the venerable K2 Mountain. DLR’s researchers created highly detailed 3D views of the K2 Mountain. These 3D images allowed the climbers to accurately analyze and review ascent routes prior to the climb. This is the first time that images of this quality have been published, and Scality’s RING storage solution helped make this possible. The Problem The Photogrammetry and Image Analysis department was dependent on a data storage system that was unable to scale to meet growing data storage needs. Their existing infrastructure was becoming an expensive bottleneck for image manipulation tasks. The department’s data storage volume was six million files and growing, with many files exceeding one terabyte. Additionally, the team often needed to access files on up to 40 client workstations at once, which proved to be impossible with the incumbent infrastructure – as it was only suited to serial access. Furthermore, each scheduled imaging mission increased the volume of files that needed to be stored by multiple terabytes. The situation was becoming increasingly difficult and the team needed a new storage solution to be able to continue with their work. “The limits of our server storage and each of the attached NAS systems were quickly reached when new satellite data arrived for processing by our systems. The data for individual projects were scattered across different storage systems. Backing up data had become a continuous process, increasingly time consuming and complex. This was further complicated by the need to find available capacity for a system that had reached its scalability limit. Most of the problems during our automatic processing arose due to this problem of limited and highly fragmented disk-space.” The Requirements The Earth Observation Center performs a high volume of image analysis. Researchers work with many extremely large images and associated metadata. The key requirements for a next-generation solution included: Unlimited file size – A highly scalable storage solution that can accommodate millions of very large images without constraining either the number of files or the size of individual files. Ease of integration with DLR’s existing custom image processing tools. Internet Access Protocols: The ability to remotely access images using Internet-based protocols. High performance parallel data access to allow the team of forty engineers and scientists direct access to the data, often simultaneously. Support for highly parallel and I/O intensive processes, such as bulk image manipulation. Data durability and resilience, to protect unique, expensive and irreplaceable information assets. The Selection Process Scality was introduced to DLR by Dell. “We searched for a system mapping many physical storage devices to one logical volume which could grow transparently and also provide integrated data replication to facilitate data backup” “We surveyed different systems including open source solutions, but found the Scality RING proposed by our hardware partner Dell to be the only solution which complied with all of our requirements, and allowed the system to expand on any type of hardware.” “After a test installation of the Scality RING on different machines at our IT partner netplace GmbH in Munich, we performed several stress tests. These included removing hard disks during operation, and turning machines on and off. Following these tests it was clear that the Scality RING would meet all our stringent performance, durability and scalability requirements.” The Solution Scality’s RING was chosen as the best candidate for this application, due to its: Intrinsically parallel architecture, that enables parallel processes to be handled efficiently and without system bottlenecks. Extreme resilience, guaranteeing unrivaled durability for irreplaceable data. High performance for I/O intensive tasks. Native scalability, ensuring that the system could grow seamlessly and cost effectively as DLR’s data storage needs expand. Unified storage approach, that accommodates both DLR’s File System needs, using Scality’s Scale-Out File System (SOFS) and CIFS connectors, and remote access requirements (using a CDMI connector). The Architecture The system is installed on Dell R550 servers, each with 12 x 3TB NL-SAS disks on a 1 Gb network. Current configuration includes 200TB of storage; additional servers are planned. Technical Results “Scality’s experienced system engineers were able to quickly complete the installation of the first servers we purchased for our Scality RING, They overcame some new challenges due to our requirement that the RING be used as a single UNIX file-system. The deployed system now works without problems. We no longer have to struggle to maintain a NAS system that cannot scale, and no longer have to worry about the time-consuming job of managing backups. All requirements, including massive parallel access from over 40 machines for automatic processing, automatic replication and ease of expansion using different hardware have been met.” Business Results The DLR RING has now been deployed for almost two years. In that time the data stored has doubled. DLR is currently in the process of doubling the capacity of their RING in anticipation of even further growth.
