My guess is that the capacity is probably dedicated for one purpose or the other. Historically, Amazon traffic spikes sharply during the holidays (you can see this on the graphs on Alexa), and hits a mid-year low. That means from January through November or so, massive amounts of extra computing power can be taken offline. Whatever is absorbed into EC2 capacity can be replaced in main Amazon.com capacity during the last quarter of the year. Even if EC2 growth exceeds the hardware pool, and they’ve got to buy more hardware, they’ve improved the effective ROIC for the pool as a whole. And as long as EC2 keeps growing, Amazon can keep using its post-holiday hardware to fund the expansion.

You’re right about hosters with pools of decommissioned hardware, though — they potentially also have an interesting way to use depreciated hardware.

For instance, I doubt that server infrastructure supporting the holiday season for Amazon.com has historically been amortized over one season, meaning that it’s possibly dedicated to the parent business and off-limits to AWS customers over the holidays. Maybe it’s rolled into AWS during subsequent years, freeing up Amazon.com to buy new hardware every year. (Or every 2 or 3 years, as the case may be.) This being the case, AWS could be using all of Amazon.com’s old hardware, but if that wasn’t adequate for the demand they face then they would have to acquire new hardware of their own. Which puts them back into a similar position as a cloud provider without such demand synergies. (They just had lower startup capital requirements.)

Or perhaps all resources are available to support any demand, including Amazon.com as well as AWS, and there is some (hopefully never used) prioritization mechanism that puts Amazon.com needs first. (If not, could a competitor damage Amazon.com sales by simply instantiating lots of AWS virtual machines and “using up” the cloud?) This approach has more complexity than dedicated resource pools, but is probably a more efficient use of resources. Then again, when a crunch happens and Amazon.com is competing against an AWS customer for resources, they’re back to square one.

In either approach there is a risk of having too little resource available for the peak demand. Amazon may have an advantage in being able to spread costs around different businesses. And they have the benefit of “free” (depreciated) decommissioned hardware, a benefit shared by other cloud providers whom have traditional hosting businesses. But I think you’re right to be worried about how this plays out over time, for all cloud providers. With the cloud mania of this past year and the success of AWS, I think the upcoming holiday season could be revealing. Regardless of whether or not we will see issues, it should be apparent to cloud users that they need a resource availability guarantee from their provider if they take their application seriously.