Happy 10th Anniversary, Amazon S3 Glacier – A Decade of Cold Storage in the Cloud

Ten years in the past, on August 20, 2012, AWS introduced the overall availability of Amazon Glacier, safe, dependable, and very low-cost storage designed for knowledge archiving and backup. On the time, I used to be working as an AWS buyer and it felt like an April Fools’ joke, providing long-term, safe, and sturdy cloud storage that allowed me to archive massive quantities of information at a really low value.

In Jeff’s authentic weblog submit for this launch, he famous that:

Glacier gives, at a value as little as $zero.01 (one US penny, one one-hundredth of a greenback) per Gigabyte per thirty days, extraordinarily low-cost archive storage. You possibly can retailer a bit of bit, or you may retailer loads (terabytes, petabytes, and past). There’s no upfront price, and also you pay just for the storage that you simply use. You don’t have to fret about capability planning, and you’ll by no means run out of space for storing.

Ten years later, Amazon S3 Glacier has advanced to be the most effective place on this planet so that you can retailer your archive knowledge. The Amazon S3 Glacier storage courses are purpose-built for knowledge archiving, offering you with the very best efficiency, most retrieval flexibility, and the bottom value archive storage within the cloud.

Now you can select from three archive storage courses optimized for various entry patterns and storage length – Amazon S3 Glacier Immediate Retrieval, Amazon S3 Glacier Versatile Retrieval (previously Amazon S3 Glacier), and Amazon S3 Glacier Deep Archive. We’ll dive into every of those storage courses in a bit.

A Decade of Innovation in Amazon S3 Glacier
To grasp how we obtained right here, we’ll stroll by the final decade and revisit a number of the most important Amazon S3 Glacier launches that basically modified archive storage eternally:

August 2012 – Amazon Glacier: Archival Storage for One Penny per GB per Month
We launched Amazon Glacier to retailer any quantity of information with excessive sturdiness at a value that permits you to do away with your tape libraries and all of the operational complexity and overhead which were a part of knowledge archiving for many years. Amazon Glacier was modeled on S3’s sturdiness and dependability however designed and constructed from the bottom as much as supply an archival storage to you at an especially low value. At the moment, Glacier launched the idea of a “vault” for storing archival knowledge. You possibly can then simply retrieve your archival knowledge by initiating a request after which the information was made out there to you for obtain in three–5 hours.

November 2012 – Archiving Amazon S3 Knowledge to Glacier
Whereas Glacier was purpose-built from the bottom up for archival knowledge, many purchasers had object knowledge that originated in S3 hotter storage that they might ultimately wish to transfer to colder storage. To make that simple for patrons, Amazon S3’s Lifecycle Administration (aka Lifecycle Rule) built-in S3 and Glacier and made the main points seen by way of the storage class of every object. Lifecycle Administration permits you to outline time-based guidelines that may begin Transition (altering S3 storage class to Glacier) and Expiration (deletion of objects). In 2014, we mixed the pliability of S3 versioned objects with Glacier, serving to you to additional cut back your general storage prices.

November 2016 – Glacier Worth Reductions and Extra Retrieval Choices for Glacier
As a part of AWS’s long-term deal with decreasing prices and passing alongside these financial savings to prospects, we lowered the value of Glacier storage to $zero.004 (lower than half a cent) within the case of 1 GB for 1 month within the US East (N. Virginia) Area, from $zero.007 in 2015 and $zero.zero10 in 2012. With storing knowledge at a really low value however having flexibility in how shortly they’ll retrieve the information, we launched two extra choices for knowledge retrieval that had been primarily based on the quantity of information that you simply saved in Glacier and the speed at which you retrieved it. You possibly can choose expedited retrieval (usually taking 1–5 minutes), bulk retrieval (5–12 hours), or the prevailing customary retrieval methodology (three–5 hours).

November 2018 – Amazon S3 Glacier Storage Class to Combine S3 Experiences
Glacier prospects appreciated the best way they may simply transfer knowledge from S3 to Glacier by way of S3 lifecycle administration, and needed us to develop on that functionality to make use of the most typical S3 APIs to function immediately on S3 Glacier objects. So, we added S3 PUT API to S3 Glacier, which lets you use the usual S3 PUT API and choose any storage class, together with S3 Glacier, to retailer the information. Knowledge will be saved immediately in S3 Glacier, eliminating the necessity to add to S3 Customary and instantly transition to S3 Glacier with a zero-day lifecycle coverage. So, you possibly can PUT to S3 Glacier like some other S3 storage class.

March 2019 – Amazon S3 Glacier Deep Archive – the Lowest Value Storage within the Cloud
Whereas the unique Glacier service supplied an especially low value for archival storage, we challenged ourselves to see if we might discover a technique to invent a fair decrease priced storage providing for very chilly knowledge. The Amazon S3 Glacier Deep Archive storage class delivers the bottom value storage, as much as 75 % decrease value (than S3 Glacier Versatile Retrieval), for long-lived archive knowledge that’s accessed lower than as soon as per 12 months and is retrieved asynchronously. At simply $zero.00099 per GB-month (or $1 per TB-month), S3 Glacier Deep Archive provides the bottom value storage within the cloud at costs considerably decrease than storing and sustaining knowledge in on-premises tape or archiving knowledge off-site.

November 2020 – Amazon S3 Clever-Tiering provides Archive Entry and Deep Archive Entry tiers
In November 2018, we launched Amazon S3 Clever-Tiering, the one cloud storage class that delivers automated storage value financial savings, as much as 95 % when knowledge entry patterns change, with out efficiency affect or operational overhead. With a view to supply prospects the simplicity and suppleness of S3 Clever-Tiering and the low storage value of archival knowledge, we added the Archive Entry tier offering the identical efficiency and pricing because the S3 Glacier storage class in addition to the Deep Archive Entry tier which provides the identical efficiency and pricing because the S3 Glacier Deep Archive storage class.

November 2021 – Amazon S3 Glacier Versatile Retrieval and S3 Glacier Immediate Retrieval
The Amazon S3 Glacier storage class was renamed to Amazon S3 Glacier Versatile Retrieval and now contains free bulk retrievals together with a further 10 % value discount throughout all Areas, making it optimized to be used circumstances equivalent to backup and catastrophe restoration.

Moreover, prospects requested us for a storage resolution that had the low prices of Glacier however allowed for quick entry when knowledge was wanted in a short time. So, we launched Amazon S3 Glacier Immediate Retrieval, a brand new archive storage class that delivers the bottom value storage for long-lived knowledge that’s hardly ever accessed and requires milliseconds retrieval. It can save you as much as 68 % on storage prices in comparison with utilizing the S3 Customary-Rare Entry (S3 Customary-IA) storage class when your knowledge is accessed as soon as per quarter.

The Amazon S3 Clever-Tiering storage class additionally lately added a brand new Archive Immediate Entry tier, offering the identical efficiency and pricing because the S3 Glacier Immediate Retrieval storage class which delivers automated 68% value financial savings for patrons utilizing S3 Clever-Tiering with long-lived knowledge.

Then and Now
Clients throughout all industries and verticals use the S3 Glacier storage courses for each conceivable archival workload. Accessing and utilizing the S3 Glacier storage courses by the S3 APIs and S3 console gives enhanced performance for knowledge administration and value optimization.

As we mentioned above, now you can select from three archive storage courses optimized for various entry patterns and storage length:

• S3 Glacier Immediate Retrieval – For archive knowledge that wants instant entry, equivalent to medical pictures, information media belongings, or genomics knowledge, select the S3 Glacier Immediate Retrieval storage class, an archive storage class that delivers the bottom value storage with milliseconds retrieval.
• S3 Glacier Versatile Retrieval – For archive knowledge that doesn’t require instant entry however must have the pliability to retrieve massive units of information without charge, equivalent to backup or catastrophe restoration use circumstances, select the S3 Glacier Versatile Retrieval storage class, with retrieval in minutes or free bulk retrievals in 12 hours.
• S3 Glacier Deep Archive – For retaining knowledge for 7–10 years or longer to satisfy buyer wants and regulatory compliance necessities, equivalent to monetary companies, healthcare, media and leisure, and public sector, select the S3 Glacier Deep Archive storage class, the bottom value storage within the cloud with knowledge retrieval inside 12–48 hours.

Watch a quick introduction video for an summary of the S3 Glacier storage courses.

All S3 Glacier storage courses are designed for 99.999999999% (11 9s) of sturdiness for objects. Knowledge is redundantly saved throughout three or extra Availability Zones which are bodily separated inside an AWS Area. Listed here are some comparisons throughout the S3 Glacier storage courses at a look:

For knowledge with altering entry patterns that you simply wish to mechanically archive primarily based on the final entry of that knowledge, select the S3 Clever-Tiering storage class. Doing so will optimize storage prices by mechanically shifting knowledge to essentially the most cost-effective entry tier when entry patterns change. Its Archive Immediate Entry, Archive Entry, and Deep Archive Entry tiers have the identical efficiency as S3 Glacier Immediate Retrieval, S3 Glacier Versatile Retrieval, and S3 Glacier Deep Archive respectively. To be taught extra, see the weblog submit Mechanically archive and restore knowledge with Amazon S3 Clever-Tiering.

To get began with S3 Glacier, see the weblog submit Finest practices for archiving massive datasets with AWS for key concerns and actions when planning your chilly knowledge storage patterns. You may as well use a hands-on lab tutorial that can assist you get began with the S3 Glacier storage courses in simply 20 minutes, and begin archiving your knowledge within the S3 Glacier storage courses within the S3 console.

Blissful Birthday, Amazon S3 Glacier!
Over the last AWS Storage Day 2022, Kevin Miller, VP & GM of Amazon S3, talked about the 10th anniversary of S3 Glacier and its tempo of innovation for a lot of buyer use circumstances all through his interview with theCUBE.

On this increasing world of information progress, it’s a must to have an archiving technique. Everybody has archival knowledge — each firm, each vertical, and each trade. There’s an archiving needn’t just for firms which were round for some time but additionally for digital native companies.

A lot of AWS prospects equivalent to Nasdaq, Digital Arts, and NASCAR have used S3 Glacier storage courses for his or her backup and archiving workloads. The next are some further latest customer-authored blogs specializing in AWS archiving greatest practices from prospects within the monetary, media, gaming, and software program industries.

An enormous thanks to all of our S3 Glacier prospects from all over the world! Over 90 % of S3’s roadmap has come immediately from suggestions from prospects such as you. We’ll by no means cease listening to you, as your suggestions and concepts are important to how we enhance the service. Thanks for trusting us and for consistently elevating the bar and pushing us to enhance to decrease prices, simplify your storage, enhance your agility, and help you innovate sooner.

In accordance with Buyer Obsession, one of many Amazon Management Ideas, your suggestions is at all times welcome! If you wish to see new S3 Glacier options and capabilities, please ship any suggestions to AWS re:Publish for S3 Glacier or by your typical AWS Assist contacts.

– Channy