Some quick takes on recent news:
Comcast P4P Results. Comcast is one of the ISPs working with hybrid-P2P CDN Pando Networks on a trial, and is showing better numbers than its competitors. The takeaway: Broadband ISPs are actively interested in P2P, CDN, and figuring out a way to monetize all of the video delivery they’re doing to their end-users.
Sprint Depeers Cogent (and Repeers). In this latest round of Cogent’s peering disputes, it’s arguing over a contract it signed with Sprint. The takeaway: Cogent is trying to keep its costs down, and is responsible for driving down bandwidth costs for everyone; its competitors are hitting back, rooted in the belief that Cogent is able to keep its prices low because it isn’t pulling its fair share of traffic carriage, which gets expressed in disputes over peering settlements.
IronScale Launches. RagingWire (a colo provider in Sacramento) has launched a managed hosting offering. Like SoftLayer, this is rapidly-provisioned dedicated servers and associated infrastructure, but unlike most of the competition in this space, it’s a managed solution. The takeaway: Like I wrote almost three years ago, it’s not about virtualization, it’s about flexibility. (“Beyond the Hype“: clients only, sorry.)
I noted previously that the Microsoft CDN study, titled “Measuring and Evaluating Large-Scale CDNs”, had disappeared. Now its lead author, Microsoft researcher Cheng Huang, has updated his home page to note that the study has been withdrawn.
Also removed from his home page, but still available from one of his collaborators, is a study from earlier this year, “Understanding Hybrid CDN-P2P: Why Limelight Needs Its Own Red Swoosh“. I assume the link was removed because it extensively details the CDN discovery methodology also used in the more recent Microsoft CDN study, so if you missed reading the study while it was available, you might want to read this slightly older paper for the details.
I just read the P2P study, which reveals something that I conjectured in my earlier analysis of the study’s blind spots: the visibility into Verizon was almost non-existent. The P2P study asserts that Akamai is present in just four locations inside Verizon’s network. This seems improbable. Verizon is Akamai’s most significant carrier reseller and one of its largest enterprise-focused resellers. It is also one of the largest broadband networks in the United States, and is a significant global network service provider. It was also a close partner of Netli, who inked a deal making Verizon its primary source of bandwidth; I would expect that even though Akamai integrated Netli into its network after acquiring it, it would have kept any strategic points of presence in Verizon’s network. One would have expected that the researchers would have wondered what the chances were that a close partner wouldn’t have substantial Akamai footprint, especially when their chart of Limelight indicated 10 Verizon locations. (Remember that the charting methodology is much less accurate for a deep-footprint CDN.)
The researchers then go on to explore the effects of hybrid P2P using those Verizon nodes (along with AT&T, which also looks like an incomplete discovery). Unfortunately, they don’t tell us much of value about peer-assisted offload; the real world has made it amply clear that actual P2P effectiveness depends tremendously on the nature of your content and your audience.
The methodological flaws make the hybrid-P2P paper’s conclusions deeply and fundamentally flawed. But like the other study, it is an interesting read.
I was recently briefed by MediaMelon, a just-launched CDN offering a “video overlay network”. The implications of their technology are worth considering, even though I think the company itself is going to have a difficult road to travel. (MediaMelon has two customers thus far, and is angel-funded; it is entering an extremely tough, competitive market. I wish them luck, since their model essentially forces them to compete in the ever-more-cutthroat CDN price war, as their entire value proposition is tied up in lowering delivery costs.)
In brief, when a content provider publishes its video to MediaMelon, MediaMelon divides the video into small chunks, each of which is a separate file that can be delivered via HTTP, and relies upon the video player to re-assemble those chunks. This chunk-based delivery is conceptually identical to Move Networks streamlets. MediaMelon then publishes the content out to its CDN partners (currently Velocix plus an unannounced second partner). MediaMelon’s special sauce is that these chunks are then delivered via multiple sources. This is normally MediaMelon’s P2P network, with a fallback to MediaMelon’s CDN partners. Since the video is in chunks, the source can switch from chunk to chunk. The video player also reports its performance to MediaMelon’s servers, allowing MediaMelon to draw conclusions about how to serve content. As a delivery-focused company, MediaMelon has decided to leave the value-adds to its media platform partners, currently thePlatform.
Whatever the challenges of their business model, though, the overlay model is interesting, and from a broader market perspective, MediaMelon’s technology highlights several things about video player capabilities that should be kept in mind:
- You can carve up your video and let the player re-assemble it.
- You can deliver using multiple sources, including P2P.
- The player knows what kind of performance it’s getting, and can report it.
These three key things make it extremely clear that it is technically feasible to create a “neutral” CDN overlay network, without requiring the cooperation of the CDNs themselves. MediaMelon is halfway there. It just hasn’t put together all the pieces (the technical hurdles are actually nontrivial), and it is designed to work with partner CDNs rather than force them into competition.
Basically, what a (non-AnyCast) CDN like Akamai or Limelight does, is that they’ve got a central engine gathering network performance data, which it uses to choose an individual CDN server, based on what it believes best for you (where “you” is defined by where your nameserver is). That individual CDN server then delivers the content to you.
What an overlay would have is a central engine that gathers performance data directly from the video player, and has a list of sources for a given piece of content (where that list includes multiple CDNs and maybe a P2P network). Based on historical and currently-reported performance data, it would direct the player to the source that delivers acceptable performance for the least cost. Dividing the content into chunks makes this easier, but isn’t strictly necessary. What you’d effectively have is a CDN-of-CDNs, with the overlay needing to own no infrastructure other than the routing processor.
That is the next-generation CDN. If it were vendor-neutral, allowing the customer to choose whomever it wanted to work with, it would usher in an era of truly brutal price competition.
Mythic has stated publicly that all of the US game servers are located in Virginia, near Mythic’s offices. A couple of traceroutes seem to indicate that they’re in Verizon, almost certainly in colocation (managed hosting is rare for MMOGs), and seem to have purely Verizon connectivity to the Internet. The webservers, on the other hand, look to be split between Verizon, and ThePlanet in Dallas. FileBurst (a single-location download hosting service) is used to serve images and cinematics.
During the beta, Mythic used BitTorrent to serve files. With the advent of full release, it doesn’t appear that they’re depending on peer-to-peer any longer — unlike Blizzard, for instance, which uses public P2P in the form of BitTorrent for its World of Warcraft updates, trading off cost with much higher levels of user frustration. MMO updates are probably an ideal case for P2P file distribution — Solid State Networks, a P2P CDN, has done well by that — and with hybrid CDNs (those combining a traditional distributed model with P2P) becoming more commonplace, I’d expect to see that model more often.
However, I’m not keen on either single data center locations or single-homing, for anything that wants to be reliable. I also believe that gaming — a performance-sensitive application — really ought to run in a multi-homed environment. My favorite “why you should use multiple ISPs, even if you’re using a premium ISP that you love” anecdote to my clients is an observation I made while playing World of Warcraft a few years ago. WoW originally used just AT&T’s network (in AT&T colocation). Latency was excellent — most of the time. Occasionally, you’d get a couple of seconds of network burp, where latency would spike hugely. If you’re websurfing, this doesn’t really impact your experience. If you’re playing an online game, you can end up dead. When WoW switched to Internap for the network piece (remaining in AT&T colo), overall latencies went up — but the latencies were still well below the threshold of problematic performance, and more importantly, the latencies were rock-solidly in a narrow window of variability. (This is the same reason multi-homed CDNs with lots of route diversity deliver better consistency of user experience than single-carrier CDNs.)
Companies like Fileburst, by the way, are going to be squarely in the crosshairs of the forthcoming Amazon CDN. Fileburst will do 5 TB of delivery at $0.80 per GB — $3,985/month. At the low end, they’ll do 100 GB or less at $1/GB. The first 100 MB of storage is free, then it’s $2/MB. They’ve got a delivery infrastructure at the Equinix IBX in Ashburn (Northern Virginia, near DC), extensive peering, but any other footprint is vague (they say they have a six-location CDN service, but it’s not clear whether it’s theirs or if they’re reselling).
If Amazon’s CDN pricing is anything like the S3 pricing, they’ll blow the doors off those prices. S3 is $0.15/GB for space and $0.17/GB for the first 10 TB of data transfer. So deliver 5 TB worth of content, out of a 1 GB store, would cost me $5,785/month with Fileburst, and about $850 with Amazon S3. Even if the CDN premium on data transfer is, say, 100%, that’d still be only $1,700 with Amazon.
Amazon has a key cloud trait — elasticity, basically defined as the ability to scale to zero (or near-zero) as easily as scaling to bogglosity. It’s that bottom end that’s really going to give them the potential to wipe out the zillion little CDNs that primarily have low-volume customers.