Blog Archives
News round-up
A handful of quick news-ish takes:
Amazon has released the beta of its EC2 management console. This brings point-and-click friendliness to Amazon’s cloud infrastructure service. A quick glance through the interface makes it clear that effort was made to make it easy to use, beginning with big colorful buttons. My expectation is that a lot of the users who might otherwise have gone to RightScale et.al. to get the easy-to-use GUI will now just stick with Amazon’s own console. Most of those users would have just been using that free service, but there’s probably a percentage that would otherwise have been upsold who will stick with what Amazon has.
Verizon is courting CDN customers with the “Partner Port Program”. It sounds like this is a “buy transit from us over a direct peer” service — essentially becoming explicit about settlement-based content peering with content owners and CDNs. I imagine Verizon is seeing plenty of content dumped onto its network by low-cost transit providers like Level 3 and Cogent; by publicly offering lower prices and encouraging content providers to seek paid peering with it, it can grab some revenue and improve performance for its broadband users.
Scott Cleland blogged about the “open Internet” panel at CES. To sum up, he seems to think that the conversation is now being dominated by the commercially-minded proponents. That would certainly seem to be in line with Verizon’s move, which essentially implies that they’re resigning themselves to the current peering ecosystem and are going to compete directly for traffic rather than whining that the system is unfair (always disengenuous, given ILEC and MSO complicity in creating the current circumstances of that ecosystem).
I view arrangements that are reasonable from a financial and engineering standpoint, that do not seek to discriminate based on the nature of the actual content, to be the most positive interpretation of network neutrality. And so I’ll conclude by noting that I heard an interesting briefing today from Anagran, a hardware vendor offering flow-based traffic management (i.e., it doesn’t care what you’re doing, it’s just managing congestion). It’s being positioned as an alternative or supplement to Sandvine and the like, offering a way to try to keep P2P traffic manageable without having to do deep-packet inspection (and thus explicit discrimination).
Google builds a CDN for its own content
An article in the Wall Street Journal today describes Google’s OpenEdge initiative (along with a lot of spin around net neutrality, resulting in a Google reply on its public policy blog).
Basically, Google is trying to convince broadband providers to let it place caches within their networks — effectively, pursuing the same architecture that a deep-footprint CDN like Akamai uses, but for Google content alone.
Much of the commentary around this seems to center on the idea that if Google can use this to obtain better performance for its content and applications, everyone else is at a disadvantage and it’s a general stab to net neutrality. (Even Om Malik, who is not usually given to mindless panic, asserts, “If Google can buy better performance for its service, your web app might be at a disadvantage. If the cost of doing business means paying baksheesh to the carriers, then it is the end of innovation as we know it.”)
I think this is an awful lot of hyperbole. Today, anyone can buy better performance for their Web content and applications by paying money to a CDN. And in turn, the CDNs pay baksheesh, if you want to call it that, to the carriers. Google is simply cutting out the middleman, and given that it accounts for as more traffic on the Internet than most CDNs, it’s neither illogical nor commercially unreasonable.
Other large content providers — Microsoft and AOL notably on a historical basis — have built internal CDNs in the past; Google is just unusual in that it’s attempting to push those caches deeper into the network on a widespread basis. I’d guess that it’s YouTube, more than anything else, that’s pushing Google to make this move.
This move is likely driven at least in part by the fact that most of the broadband providers simply don’t have enough 10 Gbps ports for traffic exchange (and space and power constraints in big peering points like Equinix’s aren’t helping matters, making it artificially hard for providers to get the expansions necessary to put big new routers into those facilities). Video growth has sucked up a ton of capacity. Google, and YouTube in particular, is a gigantic part of video traffic. If Google is offering to alleviate some of that logjam by putting its servers deeper into a broadband provider’s network, that might be hugely attractive from a pure traffic engineering standpoint. And providers likely trust Google to have enough remote management and engineering expertise to ensure that those cache boxes are well-behaved and not annoying to host. (Akamai has socialized this concept well over much of the last decade, so this is not new to the providers.)
I suspect that Google wouldn’t even need to pay to do this. For the broadband providers, the traffic engineering advantages, and the better performace to end-users, might be enough. In fact, this is the same logic that explains why Akamai doesn’t pay for most of its deep-network caches. It’s not that this is unprecedented. It’s just that this is the first time that an individual content provider has reached the kind of scale where they can make the same argument as a large CDN.
The cold truth is that small companies generally do not enjoy the same advantages as large companies. If you are a small company making widgets, chances are that a large company making widgets has a lower materials cost than you do, because they are getting a discount for buying in bulk. If you are a small company doing anything whatsoever, you aren’t going to see the kind of supplier discounts that a large company gets. The same thing is true for bandwidth — and for that matter, for CDN services. And big companies often leverage their scale into greater efficiency, to boot; for instance, unsurprisingly, Gartner’s metrics data shows that the average cost to running servers drops as you get more servers in your data center. Google employs both scale and efficiency leverage.
One of the key advantages of the emerging cloud infrastructure services, for start-ups, is that such services offer the leverage of scale, on a pay-by-the-drink basis. With cloud, small providers can essentially get the advantage of big providers by banding together into consortiums or paying an aggregator. However, on the deep-network CDN front, this probably won’t help. Highly distributed models work very well for extremely popular content. For long-tail content, cache hit ratios can be too low for it to be really worthwhile. That’s why it’s doubtful that you’ll see, say, Amazon’s Cloudfront CDN, push deep rather than continuing to follow a megaPOP model.
Ironically, because caching techniques aren’t as efficient for small content providers, it might actually be useful to them to be able to buy bandwidth at a higher QoS.
There’s no free lunch on the Internet
There’s no free lunch on the Internet. That’s the title of a research note that I wrote over a year ago, to explain the peering ecosystem to clients who wanted to understand how the money flows. What we’ve got today is the result of a free market. Precursor’s Scott Cleland thinks that’s unfair — he claims Google uses 21 times more bandwidth than it pays for. Now, massive methodological flaws in his “study” aside, his conclusions betray an utter lack of understanding of the commercial arrangements underlying today’s system of Internet traffic exchange in the United States.
Internet service providers (whether backbone providers or broadband providers) offer bandwidth at a particular price, or a settlement-free peering arrangement. Content providers negotiate for the lowest prices they can get. ISPs interconnect with each other for a fee, or settlement-free. And everyone’s trying to minimize their costs.
So, let’s say that you’re a big content provider (BCP). You, Mr. BCP, want to pay as little for bandwidth as possible. So if you’ve got enough clout, you can go to someone with broadband eyeballs, like Comcast, and say, “Please can I have free peering?” And Comcast will look at your traffic, and say to itself, “Hmm. If I don’t give you free peering, you’ll go buy bandwidth from someone like Level 3, and I will have to take your singing cow videos over my peer with them. That will increase my traffic there, which will have implications for my traffic ratios, which might mean that Level 3 would charge me for the traffic. It’s better for me to take your traffic directly (and get better performance for my end-users, too) than to congest my other peer.”
That example is a cartoonishly grotesque oversimplification, but you get the idea: Comcast is going to consider where your traffic is flowing and decide whether it’s in their commercial interest to give you settlement-free peering, charge you a low rate for bandwidth, or tell you that you have too little traffic and you can pay them more money or buy from someone else. They’re not carrying your traffic as some kind of act of charity on their part. Money is changing hands, or the parties involved agree that the arrangement is fairly reciprocal and therefore no money needs to change hands.
Cleland’s suggestion that Google is somehow being subsidized by end-users or by the ISPs is ludicrous. Google isn’t forcing anyone to peer with them, or holding a gun to anyone’s head to sell them cheap bandwidth. Providers are doing it because it’s a commercially reasonable thing to do. And users are paying for Internet access — and part of the value that they’re paying for is access to Google. The cost of accessing content is implicit in what users are paying.
Now, are users paying too little for what they get? Maybe. But nobody forced the ISPs to sell them broadband at low prices, either. Sure, the carriers and cable companies are in a price war — but this is capitalism. It’s a free market. A free market is not license to act stupidly and hope that there’s a bailout coming down the road. If you, a vendor, price a service below what it costs, expect to pay the piper eventually. Blaming content providers for not paying their “fair share” is nothing short of whining about a commercial situation that ISPs have gotten themselves into and continue to actively promote.
Google has posted a response to Cleland’s “research” that’s worth reading, as are the other commentaries it links to. I’ll likely be posting my own take on the methodological flaws and dubious facts, as well.
CloudPundit: Massive-Scale Computing 