Amazon Cloud Now Stores 339 Billion Objects

Werner Vogels Amazon

Werner Vogels, CTO, Amazon.com, speaks at the opening session of GigaOm Structure Conference in San Fransciso.

SAN FRANCISCO - What's the state of the cloud? It's getting bigger all the time, and sophisticated enterprise applications represent one of the fastest-growing sectors. That's the bottom line from Werner Vogels, the CTO of Amazon Web Services, who gave his annual "state of the cloud" presentation Wednesday morning at the GigaOm Structure 2011 conference.

First, the numbers: Amazon's S3 storage service now houses 339 billion objects, more than doubling the volume from the same time last year, when S3 stored 150 billion objects. Vogels sees more breakneck growth ahead. "I think we're just getting to the hockey stick," he said.

Enterprise Cloud Apps grow More Sophisticated

Vogels' main message was that Amazon Web Services is seeing greater adoption by enterprises building advanced applications, some using cloud-certified versions of applications from SAP and Oracle, two software companies that are huge players in the enterprise market.

An example: The U.S. government's Recovery.gov web site, which is powered by software from Microsoft, SAP and ESRI and runs completely in the Amazon cloud.

Spot Pricing Drives Enterprise Usage

Vogel said Amazon's introduction of spot pricing for AWS instances has been a huge driver of enterprise usage. Spot pricing allows customers to bid on unused Amazon EC2 capacity and run those instances when they meet defined price points. This dynamic pricing allows companies to save money by running applications only when spot prices fall below a designated price point.

"This has become very, very successful in helping (web) architects feel comfortable about the prices they are paying," said Vogels. "The things that are being enabled by spot pricing are amazing. Some of our customers have extremely sophisticated batch processing. Their bidding strategies are fascinating. They're making sure they get the lowest price possible."

These include checkpoint/restore processes that will run when instances are available at a certain price, halt the process if spot prices rise, and then resume the job when the price falls back within their pricing parameters.

To encourage further adoption of spot pricing, Vogels said that it is now available with Elastic MapReduce, a hosted Hadoop framework that offers AWS users the ability to process huge amounts of data.

GPU Clusters Gaining Traction

Vogels also said that a growing number of customers are shifting workloads to Amazon clusters using graphics processing units (GPUs). These include Animoto, the video production company that runs entirely on the Amazon cloud and has become a poster child for AWS and its capabilities.

Vogels also noted that Amazon eats its own dog food. As of Nov. 10, 2010, all of the web traffic for Amazon.com is being served by Amazon Web Services, he said.

What's ahead for Amazon's cloud operation? "Each year we see our customers building more sophisticated applications," said Vogels. "It's still day one. there's more work to be done."