129 As the British Ministry of Information said during World War II, “Keep Calm and Carry On,” the motto for anyone claiming S3 API compatibility might be “Keep Calm, but Keep Up!” Since the introduction of the S3 service in 2006 there have been 60 releases of the service and protocol. Scality introduced their first version of the S3 REST based protocol in 2010, built using C language coding. The potential scale of this type of multi-petabyte object-based platform calls to the speed and robustness of a C language based solution. However, with the advent of new authentication mechanisms, increasingly sophisticated access rights management, and other improvements, along with the fact that AWS innovates when and how they see fit, any provider of a compatible interface must be able to be a fast follower of these changes. The C language code base wasn’t meeting our needs for agility. In order to keep calm and keep up at the same time, a decision was made to redevelop the protocol stack in a higher level language. There are many new and interesting programming languages available today: Go, Rust, Swift, and Haskell, to name a few. We intentionally chose to use Node.js for this development. It’s credibly fast, asynchronous and event-driven, but the choice was really no more complicated than the fact that the pool of JavaScript programmers is very large. With the design choice in hand, the results have been encouraging with a complete rewrite of the protocol now generally available and ‘ready for prime time.’ Our pace of development is now much higher, so keeping up and finding capable developers to keep up is now a reality. While the AWS S3 protocol has been massively adopted, its usage is not nearly mainstream as filesystem protocols such as SMB or NFS, which are frequently used by those with little or no inclination to do development work. There is still much to be done to better understand this type of object based protocol, its advantages and constraints. To simplify adoption, a “run anywhere” approach was taken for the protocol engine. We chose containerization to simplify adoption. Now, any user can run the protocol on their laptop or server and immediately start the work of interface development and testing interactions with the interface. The system is directly available for download from DockerHub. This allows developers to have ready access to the protocol without internet access, without requiring a credit card for an AWS account, and avoids shadow IT projects. In order to further enhance the ability to innovate, find bugs, and keep up with the rich features of the AWS S3 protocol offering, the Scality S3 Server source code is also open source with an Apache 2.0 license. The code can be found on Github and represents the first industrial strength, freely available, AWS S3 compatible interface. With implementation in a high level language with a massive developer community, a Docker container version, the open source code that runs with Node.js is available to all who want to understand or develop for the S3 protocol. Just one thing left to do: visit s3.scality.com to start the journey, then Keep Calm, and Keep Up! Read the Press Release for the Scality S3 Server Discover the Scality S3 Server
