Quick CloudFront set-up
I’ve got a tiny low-traffic site that I’ve just set up to use Amazon’s CloudFront CDN. I chose to do it the trivial, painless way for now — through RightScale’s dashboard. It just took a couple of minutes, most of which was waiting for systems to recognize the changes. I frankly expect that my content will never be fresh in cache, but doing this will give me something to play with, without actually costing me some unpredictable amount of money. I’ve been meaning to do some similar tinkering with SimpleCDN, too, and eventually with the Rackspace/Limelight cloud CDN (which thus far lacks a snappy short name).
I still have to finish my cloud server testing, too, which I started a few months ago and which keeps being interrupted by other work and personal demands… I always feel a bit guilty keeping demo accounts for long periods of time.
Amazon’s CloudFront CDN
Amazon’s previously-announced CDN service is now live. It’s called CloudFront. It was announced on the Amazon Web Services blog, and discussed in a blog post by Amazon’s CTO. The RightScale blog has a deeper look, too.
How CloudFront Works
Basically, to use the CloudFront CDN, you drop your static objects (your static HTML, images, JavaScript libraries, etc.) into an S3 bucket. (A bucket is essentially the conceptual equivalent of a folder.) Then, you register that bucket with CloudFront. Once you’ve done this, you get back a hostname that you use for as the base of the URL to your static objects. The hostname looks to be a hash in the cloudfront.net domain; I expect CloudFront customers will normally create a CNAME (alias) in their own domain that points to it.
The tasks use Amazon API calls, but like every other useful bit of AWS functionality, there are tools out there that can make it point-and-click, so this can readily be CDN for Dummies. There’s no content “publication”, just making sure that the URLs for your static content point to Amazon. Intelligently-designed sites already have static content segregated onto its own hostname, so it’s a simple switch for them; for everyone else, it’s just some text editing, and it’s the same method as just about every other CDN for the better part of the last decade.
In short, you’re basically using S3 as the origin server. The actual delivery is through one of 14 edge locations — in the particular case of these locations, a limited footprint, but not a bad one in these days of the megaPOP CDN.
How Pricing Works
Pricing for the service is a little more complex to understand. If your content is being served from the edge, it’s priced similarly to S3 data transfers (same price at 10 TB or less, and 1 cent less for higher tiers). However, before your content can get served off the edge, it has to get there. So if the edge has to go back to the origin to get the content (i.e., the content is not in cache or has expired from cache), you’ll also incur a normal S3 charge.
You can think of this as being similar to the way that Akamai charged a few years ago — you’d pay a bandwidth fee for delivering content to users, but you’d also pay something called an “administrative bandwidth fee”, which was charged for edge servers fetching from the origin. That essentially penalized you when there was a cache miss. On a big commercial CDN like Akamai, customers reasonably felt this was sort of unfair, competitors like Mirror Image hit back by not charging those fees, and by and large the fee disappeared from the market.
It makes sense for Amazon to charge for having to go back to the (S3) origin, though, because the open-to-all-comers nature of CloudFront means that they would naturally have a ton of customers who have long-tail content and therefore very low cache hit ratios. On a practical basis, however, quite a few people who try out CloudFront will probably discover that the cache miss ratio for their content is too high; since a cache miss also confers no performance benefit, the higher delivery cost won’t make sense, and they’ll go back to just using S3 itself for static delivery. In other words, the pricing scheme will make customers self-regulate, over time.
Performance and CDN Server Misdirection
Some starting fodder for the performance-minded: My preliminary testing shows that Amazon’s accuracy in choosing an appropriate node can be extremely poor. I’ve noted previously that the nameserver is the location reference, and so the CDN’s content-routing task is to choose a content server close to that location (which will also hopefully be close to the end-user).
I live in suburban DC. I tested using cdn.wolfire.com (a publicly-cited CloudFront customer), and checked some other CloudFront customers as well to make sure it wasn’t a domain-specific aberration. My local friend on Verizon FIOS, using a local DC nameserver, gets Amazon’s node in Northern Virginia (suburban DC), certainly the closest node to us. Using the UltraDNS nameserver in the New York City area, I also get the NoVa node — but from the perspective of that nameserver, Amazon’s Newark node should actually be closer. Using the OpenDNS nameserver in NoVa, Amazon tries to send me to its CDN node in Palo Alto (near San Francisco) — not even close. My ISP, MegaPath, has a local nameserver in DC; using that, Amazon sends me to Los Angeles.
That’s a serious set of errors. By ping time, the correct, NoVa node is a mere 5 ms away from me. The LA node is 71 ms from me. Palo Alto is even worse, at 81 ms. Both of those times are considerably worse than pure cross-country latency across a decent backbone. My ping to the test site’s origin (www.wolfire.com) is only 30 ms, making the CDN latency more than twice as high.
However, I have a certain degree of faith: Over time, Amazon has clearly illustrated that they learn from their mistakes, they fix them, and they improve on their products. I assume they’ll figure out proper redirection in time.
What It Means for the Market
Competitively, it seems like Rackspace’s Cloud Files plus Limelight may turn out to be the stronger offering. The price of Rackspace/Limelight is slightly higher, but apparently there’s no origin retrieval charge, and Limelight has a broader footprint and therefore probably better global performance (although there are many things that go into CDN performance beyond footprint). And while anyone can use CloudFront for the teensiest bit of delivery, the practical reality is that they’re not going to — the pricing scheme will make that irrational.
Some people will undoubtedly excitedly hype CloudFront as a game-changer. It’s not. It’s certainly yet another step towards having ubiquitous edge delivery of all popular static content, and the prices are attractive at low to moderate volumes, but high-volume customers can and will get steeper discounts from established players with bigger footprints and a richer feature set. It’s a logical move on Amazon’s part, and a useful product that’s going to find a large audience, but it’s not going to majorly shake up the CDN industry, other than to accelerate the doom of small undifferentiated providers who were already well on their way to rolling into the fiery pit of market irrelevance and insolvency.
(I can’t write a deeper market analysis on my blog. If you’re a Gartner client and want to know more, make an inquiry and I’d be happy to discuss it.)
Cloud “overcapacity”
Investors keep gnawing persistently at the issue of data center overcapacity and the colo market, but it doesn’t look like too many people are thinking about potential overcapacity in the cloud infrastructure market and what that means in the medium-term, economic-downturn scenario.
Many of the emerging cloud infrastructure service providers (“cloud hosters”) are building substantial chunks of capacity in order to accomodate a bunch of Web 2.0 start-ups with unpredictable and sometimes wildly variable capacity needs. (The now-classic example is Animoto.)
If this sounds naggingly familiar, it should. In the ’90s, colo providers built out tons of capacity in anticipation of the growth of the dot-coms. That growth was real enough until VCs stopped pouring money into companies that had no real revenues, causing the whole house of cards to collapse — the dot-coms folded and took many colocation companies with them. (The story is more complex than that, but that’s a succinct enough summary.)
Amazon can presumably afford to have gargantuan capacity reserves in EC2, because Amazon’s got heavily seasonal traffic that leaves it with gobs of spare capacity post-Christmas, meaning that anything it can do to get revenue from that pile of hardware is gravy. As long as EC2 keeps growing, every holiday season it can pile on more hardware and then release it thereafter to be absorbed into the EC2 pool.
Other cloud providers don’t have that luxury. And renting unmanaged hardware at commodity prices has ugly returns on invested capital, and the risk profile is much higher than it is with data center leasing or colo. Most cloud providers are trying not to overbuy, but running out of capacity also has its risks, especially when it looks like the heavens will be raining failed start-ups as they run out of cash.
Electronic marketplaces aren’t free-for-alls
Much has been made of the theoretically democratizing effect of electronic marketplaces, like the iPhone store. But it’s worth noting that such marketplaces can be, like the iPhone store, gated communities, not free-for-alls where anyone can hawk their wares.
On Friday, Google was set to launch a voice recognition app on the iPhone. Now we’re being told it will probably be available today. The reason for the delay: Apple hasn’t approved the app for the store, for reasons that currently remain mysterious. Google, obviously, wasn’t expecting it, since they had a blizzard of publicity surrounding the Friday launch in what was apparently every anticipation that it’d be available then. Now, Google is well-known for the chaos of its internal organization, but it doesn’t seem unreasonable to believe that their PR people have it pretty well together, which leaves one wondering what Apple was thinking.
There have been regular App Store approval issues, of course, but the Google app is particularly high-profile.
Vendor-controlled open marketplaces are only as open as the vendor wants to make them. That’s true of the marketplaces evolving around cloud services, too. Don’t lose sight of the fact that vendor-controlled ultimately means that the vendor has the power to do anything that the market will tolerate them doing, which at the moment can be quite considerable.
Recently-published research
Here’s a quick round-up of some of my recently-published research.
Is Amazon EC2 Right For You? This is an introduction to Amazon’s Elastic Compute Cloud, written for a mildly technical audience. It summarizes Amazon’s capabilities, the typical business case for using it, and what you’ve got to do to use it. If you’re an engineer looking for a quick briefing, or you want to show a “what this newfangled thing is” summary to your manager, or you’re an investor trying to understand what exactly it is that Amazon does, this is the document for you.
Dataquest Insight: Web Hosting, North America, 2006-2012. This is an in-depth look at the colocation and hosting business, together with market forecasts and trends. (Investors may also want to look at the Invest Implications.)
Dataquest Insight: Content Delivery Networks, North America, 2006-2012. This is an in-depth look at the CDN market, segment-by-segment, with market forecasts and trends. (Investors may also want to look at the Invest Implications.)
You’ll need to be a Gartner subscriber (or purchase the individual document) in order to view these pieces.
Upcoming research (for publication in the next month): A pricing guide for Web hosting and cloud infrastructure services; a classification scheme and service provider roadmap for cloud offerings; a toolkit for CDN requirements gathering and price estimation; a framework for gathering video requirements; and a CDN selection guide.
Quick takes: Comcast, Cogent, IronScale
Some quick takes on recent news:
Comcast P4P Results. Comcast is one of the ISPs working with hybrid-P2P CDN Pando Networks on a trial, and is showing better numbers than its competitors. The takeaway: Broadband ISPs are actively interested in P2P, CDN, and figuring out a way to monetize all of the video delivery they’re doing to their end-users.
Sprint Depeers Cogent (and Repeers). In this latest round of Cogent’s peering disputes, it’s arguing over a contract it signed with Sprint. The takeaway: Cogent is trying to keep its costs down, and is responsible for driving down bandwidth costs for everyone; its competitors are hitting back, rooted in the belief that Cogent is able to keep its prices low because it isn’t pulling its fair share of traffic carriage, which gets expressed in disputes over peering settlements.
IronScale Launches. RagingWire (a colo provider in Sacramento) has launched a managed hosting offering. Like SoftLayer, this is rapidly-provisioned dedicated servers and associated infrastructure, but unlike most of the competition in this space, it’s a managed solution. The takeaway: Like I wrote almost three years ago, it’s not about virtualization, it’s about flexibility. (“Beyond the Hype“: clients only, sorry.)
The broadband-caps disaster
The United States has now elected a President who has pledged universal broadband. On almost the same day, AT&T announced it would be following some of its fellow network operators into trialing metered broadband.
Broadband caps have been much more common in Europe, but the trend there is away from caps, not towards them. Caps stifle innovation, discouraging the development of richer Internet experiences and the convergence of voice and video with data.
AT&T’s proposed caps start at 20 GB of data transfer per month. That’s equivalent to about 64 kbps of sustained data — barely the kind of speed a modem can manage. Or, put another way, that’s five 4 GB, DVD-quality movie downloads. And those caps are generous compared to Time Warner’s — AT&T proposes 20-150 GB caps, vs. TW’s 5-40 GB. (Comcast is much more reasonable, with its 250 GB cap.)
The US already has pathetic broadband speeds compared to much of the rest of the developed world. There are lots of good reasons for that, of course, like our broader population spread, but we don’t want to be taking steps backwards, and caps are certainly that. (And that’s not even getting into the important question of what “broadband” really constitutes now, and what, in a universal deployment, should be the minimum speed necessary to justify the infrastructure investment against future sustainable usefulness.)
Yes, network operators need to make money. Yes, someone has to pay for the bandwidth. Yes, customers with exceptionally high bandwidth usage should expect to pay for that usage. But the kind of caps that are being discussed are simply unreasonable, especially for a world that is leaning more and more heavily towards video.
A few months ago, comScore reported video metrics showing 74% of the US Internet audience watched video, consuming an average of 228 minutes of video. 35% of that was YouTube, so let’s call that 320 kbps. It looks like the remainder is mostly higher-quality. Hulu makes a good reference — 480 kbps – 700 kbps, with the highest quality topping 1 Mbps. For purposes of calculation, let’s call it 700 kbps. Add it all up and you’re looking at about 1 GB of content delivered to the average video-watcher.
Average page weights are on the rise; Keynote, an Internet performance measurement company, recently cited 750k in a study of media sites. I’d probably cite the average page as more in the 250-450k range, but I don’t dispute that heavier pages are where things are going (compare a January 2008 study). At that kind of weight, you can view around 1500 pages in 1 GB of transfer — i.e., about 50 pages per day.
A digital camera shot is going to be in the vicinity of 1.5 MB, but a photo on Flickr is typically in the 500k range, so you can comfortably view a photo gallery of five dozen shots every day in your 1 GB.
Email sizes are increasing. Assuming you get attachments, you’re probably looking at around 75k per email, as an average. 1 GB will let you get around 450 emails per day, but if you’re downloading your spam, at the 95% mark, that gets you about 20 legit messages per day.
If you’re a Vonage customer or the like, you’re looking at around 96 kbps, or around 45 minutes of VoIP talk time per day, in 1 GB of usage.
Now add your anti-virus updates, and your Windows and other app software updates. Add your online gaming (don’t forget your voice chat with that), your instant messaging, and other trivialities of Internet usage.
And good luck if you’ve got more than one computer in your household — which a substantial percentage of broadband households do. You can take those numbers and multiply them out by the number of users in your household.
A 5 GB cap is nothing short of pathetic. Casual users can easily run up against that kind of limit with the characteristics of today’s content, and families will be flat-out hosed. With content only getting more and more heavyweight, this situation is only going to get worse.
20 GB will probably suffice for single-person, casual-user households that don’t watch much video. But families and online entertainment enthusiasts will certainly need more, and the low caps violate, I think, reasonable expectations of what one can get out of a broadband connection.
Making users watch their usage is damaging to the entire emerging industry around rich Internet content. I respect the business needs of network operators, but caps are the wrong way to achieve their goals, and counterproductive in the long term.
User or community competence curve
I was pondering cloud application patterns today, and the following half-formed thought occurred to me: All new technologies go through some kind of competence curve — where they are on it determines the aggregate community knowledge of that technology, and the likely starting point for a new user adopting that technology. This might not entirely correlate to its position on the hype cycle.
The community develops a library of design patterns and recipes, over the lifetime of the technology. This is everything from broad wisdom like, “MVC is probably the right pattern for a Web app”, to trivia like “you can use Expando with Google App Engine’s user class to get most of what you want out of traditional session management”. And the base level of competence of the engineers grows in seemingly magical ways — the books get better, Googling answers gets easier, the random question tossed over your shoulder to another team member has a higher chance of being usefully answered. As the technology ages, this aggregate knowledge fades, until years down the road, the next generation of engineers end up repeating the mistakes of the past.
So far, we have very little aggregate community knowledge about cloud.
Cloud enables offshoring
The United States is a hotbed of cloud innovation and adoption, but cloud is also going to be a massive enabler of the offshoring of IT operations.
Peter Laird (an architect at Oracle) had an interesting blog post about a month ago on cloud computing mindshare across geographies, analyzing traffic to his blog. And Pew Research’s cloud adoption study indicate that uptake of cloud apps among American consumers is very high. But where the users are doesn’t matter.
Today, most companies still do most of their IT Ops locally (i.e., wherever their enterprise data centers are), even if they’ve sent functions like help desk offshore. Most companies server-hug — their IT staff is close to wherever the equipment is. But the trend is moving towards remote data centers (especially as the disparity between data center real estate prices between, say, New York City and Salt Lake City grows), and cloud exacerbates that even more. Data centers don’t move off-shore because of network latency, data protection laws, etc., so that won’t change — but a big Internet data center only employs about 50 people.
What the future looks like could be very similar to the NaviSite model — local data centers staffed by small local teams who handle physical hardware, but all the monitoring, remote management, and software development for automation and other back-office functions handled offshore.
Being a hardware wrangler isn’t a high-paying job. In fact, a lot of hosting and Web 2.0 companies hire college students, part-time, to do it. So in making a move to cloud, we seem to be facilitating the further globalization of the IT labor market for the high-paying jobs.
Does architecture matter?
A friend of mine, upon reading my post on the cloud skills shift, commented that he thought that the role of the IT systems architect was actually diminishing in the enterprise. (He’s an architect at a media company.)
His reasoning was simple: Hardware has become so cheap that IT managers no longer want to spend staff time performance-tuning, finding just the right configuration, or getting the sizing and capacity planning at its most efficient, any longer.
Put another way: Is it cheaper to have a senior operations engineer on staff, or is it cheaper to just buy more hardware?
The one-size-fits-all nature of cloud may very well indicate the latter, for organizations for whom cutting-edge technology expertise does not drive competitive advantage.
CloudPundit: Massive-Scale Computing 