Blog Archives
Trialing a lot of cloud IaaS providers
I’ve just finished writing the forthcoming Public Cloud IaaS Magic Quadrant (except for some anticipated tweaks when particular providers come back with answers to some questions), which has twenty providers. Although Gartner normally doesn’t do hands-on evaluations, this MQ was an exception, because the easiest way to find out if a given service can do X, was generally to get an account, and attempt to do X. Asking the vendor sometimes requires a bunch of back-and-forth, especially if they don’t do X but and are weaseling their reply, forcing you to ask a set of increasingly narrow, specific questions until you get a clear answer. Also, I did not want to constantly bombard the vendors with questions, since, come MQ time, it tends to result in a firedrill whether or not you intended the question as urgent or even particularly important. (I apologize for the fact that I ended up bombarding many vendors with questions, anyway.)
I’ve used cloud services before, of course, and I am a paying customer of two cloud IaaS providers and a hosting provider, for my personal hobbies. But there’s nothing quite like a blitzkrieg through this many providers all at once. (And I’m not quite done, because some providers without online sign-up are still getting back to me on getting a trial account.)
In the course of doing this, I have had some great experiences, some mediocre experiences, and some “you really sell this and people buy it?” experiences. I have online chatted with support folks for basic questions not covered in the documentation (like “if I stop this VM, does it stop billing me, or not?” which varies from provider to provider). I have filed numerous support tickets (for genuine issues, not for evaluation purposes). I have filed multiple bug reports. I have read documentation (sometimes scanty to non-existent). I have clicked around interfaces, and I have actually used the APIs (working in Python, and in one case, without using a library like libcloud); I have probably weirded out some vendors by doing these things at 2 am, although follow-the-sun support has been intriguing. Those of you who follow me on Twitter (@cloudpundit) have gotten little glimpses of some of these things.
Ironically, I have tried to not let these trials unduly influence my MQ evaluations, except to the extent that these things are indisputably factual — features, availability of documentation, etc. But I have taken away strong impressions about ease of use, even for just the basic task of provisioning and de-provisioning a virtual machine. There is phenomenal variation in ease of use, and many providers could really use the services of a usability expert.
Any number of these providers have made weird, seemingly boneheaded decisions in their UI or service design, for which there’s no penalty to anything in MQ scoring, but did occasionally make me stare and go, “Seriously?”
I’m reluctant to publicly call out vendors for this stuff, so I’ll pick just one example from a vendor that has open online sign-up, where it’s not a private issue that hasn’t been raised on a community forum, and they’re not the sort of vendor (I hope) to make angry calls to Gartner’s Ombudsman demanding that I take this post down. (Dear OpSource folks: Think of this as tough love, and I hope Dimension Data analyst relations doesn’t have conniptions.)
So, consider: OpSource has pre-built VMs, that come with a set amount of compute and RAM, bundled with an OS. Great. Except that you can’t alter a bundle at the time of provisioning. So, say, if I want their Ubuntu image, it comes only in a 2 CPU core config. If I want only 1 core, I have to provision that image, wait for the provision to finish, go in and edit the VM config to reduce it to 1 core, and then wait for it to restart. After I go through that song and dance once, I can clone the config… but it boggles the mind why I can’t get the config I want from the start. I’m sure there’s a good technical reason, but the provider’s job is to mask such things from the user.
The experience has also caused me to wholly revise my opinion of vCloud Director as a self-service tool for the average goomba who wants a VM. I’d always seen vCD as a demo being given by experts, where it looked like despite the pile of complex functionality, it was easy enough to use. The key thing is that the service catalogs were always pre-populated in those demos. If you’re starting from the bare vCD install that a vCloud Powered provider is going to give you, you face a daunting task. Complexity is necessary for that level of fine-grained functionality, but it’s software that is in desperate need of pre-configuration from the service provider, and quite possibly an overlay interface for Joe Average Developer.
Now we’ll see if my bank freezes my credit card for possible fraud, when I’m hit with a dozen couple-of-cents-to-a-few-dollar charges come the billing cycle — I used my personal credit card for this, not my corporate card, since Gartner doesn’t actually reimburse for this kind of work. Ironically, once I spent a bunch of time on these sites, Google and the other online ad networks have started displaying ads that consist of nothing but cloud providers, including “click here for a free trial” or “$50 credit” or whatever, but of course you can’t apply those to existing accounts, which makes every little, “hey, you’ve spent another ten cents provisioning and de-provisioning this VM” charge which I’m noting in the back of my head now, into something which will probably annoy me in aggregate come the billing cycle.
Some things, you just can’t know until you try it yourself.
Results of Symposium workshop on Amazon
I promised the attendees at my Gartner Symposium workshop, called “Using Amazon Web Services“, that I would post the notes from the session, so here they are — with some context for public consumption.
A workshop is a structured, facilitated discussions that are designed to assist participants in working through a problem, coming up with best practices, etc. This one had thirty people, all from IT organizations that were either using Amazon or planning to use Amazon.
Because I didn’t know what level of experience with Amazon the workshop attendees would have, I actually prepared two workshops in advance. One of them was a highly structured work-through of preparing to use Amazon in a more formal way (i.e., not a single developer with a credit card or the like), and the other was a facilitated sharing of challenges and best practices amongst current adopters. As the room skewed heavily towards people who already had a deployment well under way, this workshop focused on the latter.
I started the workshop with introductions — people, companies, current use cases. Then, I asked attendees to share their use cases in more details in their smaller working groups. This turned into a set of active discussions that I allowed extra time for, before I asked each of the group to make a list of their most significant challenges in adopting/using Amazon, and their solutions if any. Throughout, I circulated the room, listening and, rarely, commenting. Each group then shared their findings, and I offered some commentary and then did an open Q&A (with some more participant sharing of their answers to questions).
Broadly, I would say that we had three types of people in the room. We had folks from the public sector and education, who were at a relatively early stage in adoption; we had people who were test/dev oriented but in a significant way (i.e., formal adoption, not a handful of developers doing their thang); and we had people who were more e-business oriented (including people from net-native businesses like SaaS, as well as traditional businesses with a hosting type of need), although that could be test/dev or production. Most of the people were mid-level IT management with direct responsibility for the Amazon services.
Some key observations:
Dealing with the financial aspects of moving to the cloud is hard. Understanding the return on investment, accurately estimating costs in advance, comparing costs to internal costs, and understanding the details of billing were common challenges of the participants. Moreover, it raises the issue of “is capital king or is expense king?” Although the broader industry is constantly talking about how people are trying to move to expense rather than capital, workshop participants frequently asserted that it was easier for them to get capital than to up their recurring expenses. (As a side note, I have found that to be a frequent assertion in both inquiry and conference 1-on-1s.) Finally, user management, cost control, and turning resources on/off appropriately were problematic in the financial context.
Move low-risk workloads first. The workshop participants generally assessed Amazon as being suitable only to test/dev, non-mission-critical workloads, and things that had specifically been designed with Amazon’s characteristics in mind. Participants recommended a risk profile of apps, and moving low-risk apps first. They also saw their security organizations as being a barrier to adoption. Many had issues with their Legal departments either trying to prevent use of services or causing issues in the contracting process (what Amazon calls an Enterprise Agreement); participants recommended not involving Legal until after adopting the service.
Performance is a problem. Performance was cited as a frequent issue, especially storage performance, which participants noted was unsuitable to their production applications, and one participant made the key point that many test/dev situations also require highly performant storage (something he had first discovered when his ILM strategy placed test/dev storage at a lower more commodity tier and it impacted his developers).
Know what your SLA isn’t. Amazon’s limited SLAs were cited as an issue, particularly the mismatch in what many users thought the SLA was versus what it actually was, and what it’s actually turned out to be in practice (given Amazon’s outages this year). Participants also stressed business continuity planning in this context.
Integration is a challenge. Participants noted that going to test/dev in the cloud, while maintaining production in an internal data center, splits the software development lifecycle across data centers. This can be overcome to some degree with the appropriate tools, but still creates challenges and sometimes outright problems. Also, because speed of deployment is such a driving factor to go to the cloud, there is a resulting fragmentation of solutions. A service catalog would help some of these issues.
Data management can be a challenge. Participants were worried about regulatory compliance and the “where is my data?” question. Inexperienced participants were often not aware that non-S3 data is generally local to an availability zone. But even beyond that, there’s the question of what data is being put where by the cloud users. Participants with larger amounts of data also faced challenges in moving data in and out of the cloud.
Amazon isn’t the right provider for all workloads in the cloud. Several workshop participants used other cloud IaaS providers in addition to Amazon, for a variety of other reasons — greater ease of use for users who didn’t need complex things, enterprise-grade availability and performance, better manageability, security capabilities, and so forth.
I have conducted cloud workshops and what Gartner calls analyst/user roundtables at a bunch of our conferences now, and it’s always interesting what the different audiences think about, and how much it’s evolving over time. Compared to last year’s Symposium, the state of the art of Amazon adoption amongst conference attendees has clearly advanced hugely.
VMware vCloud Global Connect and commoditization
At VMworld, VMware has announced vCloud Global Connect, a federation between vCloud Datacenter Provider partners.
My colleague Kyle Hilgendorf has written a good analysis, but I wanted to offer a few thoughts on this as well.
The initial partners for the announcement are Bluelock (US, based in Indianapolis), SingTel (Singapore), and SoftBank Telecom (Japan). Notably, these vendors are landlocked, so to speak — they have deployments only within their home countries, and who probably will not expand significantly beyond their home territories. Consequently, they’re not able to compete for customers who want multi-region deployments but one throat to choke. (Broadly, there are still an insufficient number of high-quality cloud providers who have multi-region deployments.)
These providers are relatively heavyweight — their typical customers are organizations which are going through a formal sourcing process in order to procure infrastructure, and are highly concerned about security, availability, performance, and alignment with enterprise IT. I expect that anyone who chooses federation with Global Connect is going to apply intense scrutiny to the extension provider, as well. At least because the vCloud Datacenter architecture is to some extent proscriptive, and has relatively high requirements, in theory all federation providers should pass the buyer’s most basic “is this cloud provider architected in a reasonable fashion” checks.
However, I think customers will probably strongly prefer to work with a truly global provider if they need truly global infrastructure (as opposed to simply trying to globally source infrastructure that will be used in unique ways within each region) — and those with specific regional needs are probably going to continue to buy from regional (or local) providers, especially given how fragmented cloud IaaS sourcing frequently is.
It’s an important technical capability for VMware to demonstrate, though, since, implicitly, being able to do this between providers also means that it should be possible to move workloads between internal vClouds and external vClouds, and to disaster-recover between providers.
Importantly, the providers chosen for this launch are also providers who are not especially worried about being commoditized. Their margin is really made on the value-added services, especially managed services, and not so much from just providing compute cycles. Each of them probably gains more from being able to address global customer needs, than they lose from allowing their infrastructure to be used by other providers in this fashion.
I do believe that the core IaaS functionality will be commoditized over time, just like the server market has become commoditized. I believe, however, that IaaS providers will still be able to differentiate — it’ll just be a differentiation based on the stuff on top, not the IaaS platform itself.
In the early years of the market, there is significant difference in features/functionality between IaaS providers (and how that relates to cost), but the roadmaps are largely convergent over the next few years. Just like hosters don’t depend on having special server hardware in order to differentiate from one another, cloud IaaS providers eventually won’t depend on having a differentiated base infrastructure layer — the value will primarily come higher up the stack.
That’s not the say that there won’t still be difference in the quality of the underlying IaaS platforms, and some providers will manage costs better than others. And the jury’s still out on whether providers who build their own intellectual property at the IaaS platform layer, vs. buying into vCloud (or Cloud.com, some future OpenStack-based stack, or one of many other “cloud stacks”), will generate greater long-term value.
(For further perspective on commoditization, see an old blog post of mine.)
OpenStack, community, and commercialization
I wrote, the other day, about Citrix buying Cloud.com, and I realized I forgot to make an important point about OpenStack versus the various commercial vendors vying for the cloud-building market; it’s worthy of a post on its own.
OpenStack is designed by the community, which is to say that it’s largely designed by committee, with some leadership that represents, at least in theory, the interests of the community and has some kind of coherent plan in mind. It is implemented by the community, which means that people who want to contribute simply do so. If you want something in OpenStack, you can write it and hope that your patches are included, but there’s no guarantee. If the community decides something should be included in OpenStack, they need some committers to agree to actually write it, and hope that they implement it well and do it in some kind of reasonable timeframe.
This is not the way that one normally deals with software vendors, of course. If you’re a potentially large customer and you’d like to use Product X but it doesn’t contain Feature Y that’s really important to you, you can normally say to the vendor, “I will buy X if you had Y within Z timeframe,” and you can even write that into your contract (usually witholding payment and/or preventing the vendor from recognizing the revenue until they do it).
But if you’re a potentially large customer that would happily adopt OpenStack if it just had Feature Y, you have miminal recourse. You probably don’t actually want to write Feature Y yourself, and even if you did, you would have no guarantee that you wouldn’t be maintaining a fork of the code; ditto if you paid some commercial entity (like one of the various ventures that do OpenStack consulting). You could try getting Feature Y through the community process, but that doesn’t really operate on the timeframe of business, nor have any guarantees that it’ll be successful, and also requires you to engage with the community in a way that you may have no interest in doing. And even if you do get it into the general design, you have no control over implementation timeframe. So that’s not really doable for a business that would like to work with a schedule.
There are a growing number of OpenStack startups that aim to offer commercial distributions with proprietary features on top of the community OpenStack core, including Nebula and Piston (by Chris Kemp and Joshua McKenty, respectively, and funded by Kleiner Perkins and Hummer Winblad, respectively, two VCs who usually don’t make dumb bets). Commercial entities, of course, can deal with this “I need to respond to customer needs more promptly than the open source community can manage” requirement.
There are many, many entitities, globally, telling us that they want to offer a commercial OpenStack distribution. Most of these are not significant forks per se (although some plan to fork entirely), but rather plans to pick a particular version of the open source codebase and work from there, in order to try to achieve code stability as well as add whatever proprietary features are their secret source. Over time, that can easily accrete into a fork, especially because the proprietary stuff can very easily clash with whatever becomes part of OpenStack’s own core, given how early OpenStack is in its evolution.
Importantly, OpenStack flavors are probably not going to be like Linux distributions. Linux distributions differ mostly in which package manager they use, what packages are installed by default, and the desktop environment config out of the box — almost cosmetic differences, although there can be non-cosmetic ones (such as when things like virtualization technologies were supported). Successful OpenStack commercial ventures need to provide significant value-add and complete solutions, which, especially in the near term when OpenStack is still a fledgling immature project, will result in a fragmentation of what features can be expected out of a cloud running OpenStack, and possibly significant differences in the implementation of critical underlying functionality.
I predict most service providers will pick commercial software, whether in the form of VMware, Cloud.com, or some commercial distribution of OpenStack. Ditto most businesses making use of cloud stack software to do something significant. But the commercial landscape of OpenStack may turn out to be confusing and crowded.
Cloud IaaS coverage at Gartner
I’ve got a pair of new European colleagues, and I thought I’d take a moment to introduce, on my blog, the folks who cover public cloud infrastructure as a service here at Gartner, and to answer a common question about the way we cover the space here.
There are three groups of analysts here at Gartner who cover cloud IaaS, who belong to three different teams. Those teams are our Infrastructure and Operations (I&O) team, which is part of the division that offers advice to technology buyers (what Gartner calls “end-user organizations”) in the traditional Gartner client base of IT managers; our High-Tech and Telecom Provider (“HTTP”) division, which offers advice to vendors and investors along with end-users, and also produces quantitative market data such as forecasts and market statistics; and our IT1 division (formerly our Burton Group acquisition), which offers advice to technology implementors, generally IT architects and senior engineers in end-user organizations.
We all collaborate with one another, but these distinctions matter for anyone buying research from us. If you’re just buying what Gartner calls Core Research, you’ll have access to what the I&O analysts publish, along with anything that HTTP analysts publish into Core. To get access to HTTP-specific content, though, you’ll need to buy an upgrade, usually in the form of a Gartner for Business Laeders (GBL) research seat. The IT1 resesarch is sold separately; anything that IT1 analysts write (that’s not co-authored with analysts in other groups) goes solely to IT1 subscribers. The I&O analysts and HTTP analysts are available via inquiry by anyone who buys Gartner research, but the IT1 analysts are only inquiry-accessible by those who buy IT1 research specifically. You can, however, brief any of us — client status doesn’t matter for briefings.
So, we’re:
- Lydia Leong (HTTP, North America) – Cloud IaaS, Web hosting and colocation, content delivery networks, cloud computing and Internet infrastructure in general.
- Ted Chamberlin (I&O, North America) – Web and app hosting, colocation, cloud IaaS, network services (voice, data, and Internet).
- Drue Reeves (IT1, North America) – Data centers and cloud infrastructure, both internal and external.
- Kyle Hilgendorf (IT1, North America) – Data centers and cloud infrastructure, both internal and external.
- Tiny Haynes (I&O, Europe) – Web and app hosting, colocation, cloud IaaS, carrier services.
- Gregor Petri (HTTP, Europe) – Cloud IaaS, Web hosting and colocation, carrier services.
- Chee-Eng To (HTTP, Asia) – Carrier services in Asia, including cloud IaaS.
- Vincent Fu (HTTP, China) – Carrier services in China, including cloud IaaS.
Tiny Haynes and Gregor Petri are brand-new to Gartner, and they’ll be deepening our coverage of Europe as well as contributing to global research.
Citrix buys Cloud.com
(This is part of a series of “catch-up” posts of announcements that I’ve wanted to comment on but didn’t previously find time to blog about.)
Recently, Citrix acquired Cloud.com. The purchase price was reported to be in the $200m+ vicinity — around 100x revenues. (Even in this current run of outsized valuations, that’s a rather impressive payday for an infrastructure software start-up. I heard that VMware’s Paul Maritz was talking about how these guys were shopping themselves around, into which some people have read that they ‘had’ to sell, but companies that sell themselves for 100x trailing revenues don’t ‘have’ to be doing anything, other than sniffing around to see if anyone is willing to give them even more money.)
Cloud.com (formerly known as VMOps) is one of a great many “cloud operating system” companies — it competes with Abiquo, OpenStack, Eucalyptus, Nimbula, VMware (in the form of vCloud Director), and so on. By that, I mean that you can take Cloud.com and use it to build cloud IaaS of your very own. While you can use Cloud.com to build a private cloud, the reason that Cloud.com commanded such a high valuation is that it’s currently the primary alternative to VMware for service providers who want to build public cloud IaaS.
Cloud.com is a commercial open-source vendor, but realistically, it’s heavily on the commercial side, not the open-source side; people running Cloud.com in production are generally using the licensed, much more featureful, version. Large service providers who want to build commodity clouds, particularly on the Xen hypervisor (especially Citrix Xen, rather than open-source Xen), are highly likely to choose Cloud.com’s CloudStack product as the underlying “cloud OS”. We’re also increasingly hearing from service providers who intend to use Cloud.com to manage VMware-based environments (using the VMware stack minus vCloud Director), as part of a hypervisor-neutral strategy.
Key service provider customers include GoDaddy and Tata Communications. A particular private cloud customer of note is Zynga, which uses Cloud.com to provide Amazon-compatible (and thus Rightscale-compatible) infrastructure internally, letting it easily move workloads across their own infrastructure and Amazon’s.
Citrix, of course, now has a significant commitment to OpenStack, in the form of Project Olympus, their planned commercial distribution. The Cloud.com acquisition is nevertheless complementary, though, not competitive to the OpenStack commitment.
Cloud.com provides a much more complete set of features than OpenStack — it’s got much of what you need to have a turnkey cloud. Over time, as OpenStack matures, Cloud.com will be able to replace the lower levels of its software stack with OpenStack components instead. For Citrix, though (and broadly, service providers interested in VMware alternatives), this is a time-to-market issue as well as a solution-completeness issue.
In my conversations with a variety of organizations that are deeply strategically involved with OpenStack and working in-depth on the codebase, consensus seems to have developed that OpenStack is about 18 months from maturity (in the sense that it will be stable enough for a service provider who needs to depend on it to run their business to be able to reasonably do so). That’s forever in this fast-moving market. While Swift (the storage piece) is currently reliable and in production use at a variety of service providers, Nova (the compute piece) is not — there are no major service providers running Nova, and it’s acknowledged to not be service-provider-ready. (Rackspace is running the code it got via the acquisition of Slicehost, not the Nova project.) Service providers want to work with proven, stable code, and that’s not Nova right now — that’s Cloud.com. (Or VMware, and even there, people have been touchy about vCloud Director.)
It’s not that the service providers have a deep interest in running an open-source codebase; rather, they are looking for an alternative to VMware that is less expensive. Cloud.com currently fills that need reasonably well.
Similarly, it’s not that most of the members of the OpenStack coalition are vastly interested in an open-source cloud world, but rather, that they realize that there needs to be an alternative to VMware’s ecosystem, and it is in the best interests of VMware’s various competitors to pool their efforts (and for vendors in more of an “arms merchant” role, to ensure that their stuff works with every ecosystem out there). Open source is a means to an end there. Cloud.com’s stack, whether commercial or open source, is only a benefit to the OpenStack project, in the long term.
This acquisition means something pretty straightforward: Citrix is ensuring that it can deliver a full service provider stack of software that will enable providers to successfully compete against vCloud — or to have hypervisor-neutral solutions peacefully coexist, in a way that can be easily blended to meet business needs for a broad range of IaaS solutions. While Citrix would undoubtedly love to sell more XenServer licenses, ultimately the real money is in selling the rest of its portfolio to service providers — like NetScaler ADCs. Having a hypervisor-neutral cloud stack benefits Citrix’s overall position, even if some Cloud.com customers will choose to go VMware or KVM or open-source Xen rather than Citrix Xen for the hypervisor.
It certainly doesn’t hurt that Cloud.com’s Amazon-compatible APIs (and thus support of RightScale’s functionality) is also tremendously useful for organizations seeking to build Amazon-compatible private clouds at scale. No one else has really addressed this need, and VMware (in an infrastructure context) has largely targeted the market for “dependable”, classically enterprise-like infrastructure, rather than explored the opportunities in the emerging demand for commodity cloud.
In short, I think Cloud.com is a great buy for Citrix, and VMware-watchers interested in whether or not their vCloud service provider initiative is working well should certainly track Cloud.com wins vs. vCloud wins in the service provider space.
Riverbed acquires Zeus and Aptimize
(This is part of a series of “catch-up” posts of announcements that I’ve wanted to comment on but didn’t previously find time to blog about.)
Riverbed made two interesting acquisitions recently, which I think signal a clear intention to be more than just a traditional WAN optimization controller (WOC) vendor — Zeus, and Aptimize. If you’re an investment banker or a networking vendor, who has talked to me over the last year, you know that these two companies have been right at the top of my “who I think should get bought” list; these are both great pick-ups for Riverbed.
Zeus has been around for quite some time now, but a lot of people have never heard of them. They’re a small company in the UK. Those of you who have been following infrastructure for the Web since the 1990s might remember them as the guys who developed the highest-performance webserver — if a vendor did SPECweb benchmarks for its hardware back then, they generally used Zeus for the software. It was a great service provider product, too, especially for shared Web hosting — it had tons of useful sandboxing and throttling features that were light-years ahead of anyone else back then. But despite the fact that the tech was fantastic, Zeus was never really commercially successful with their webserver software, and eventually they turned their underlying tech to building application delivery controller (ADC) software instead.
Today, Zeus sells a high-performance, software-based ADC, with a nice set of features, including the ability to act as a proxy cache. It’s a common choice for high-end load-balancing when cloud IaaS customers need to be able to deploy a virtual appliance running on a VM, rather than dropping in a box. It’s also the underlying ADC for a variety of cloud IaaS providers, including Joyent and Rackspace (which means it’ll also get an integration interface to OpenStack). Notably, over the last two years, we’ve seen Zeus supplanting or supplementing F5 Networks in historically strong F5 service provider accounts.
Aptimize, by contrast, is a relatively new start-up. It’s a market leader in front-end optimization (FEO), sometimes also called Web performance optimization (WPO) or Web content optimization (WCO). FEO is the hot new thing in acceleration — it’s been the big market innovation in the last two years. While previous acceleration approaches have focused upon network and protocol optimization, or on edge caching, FEO optimizes the pages themselves — the HTML, Cascading Style Sheets (CSS), JavaScript, and so forth that goes into them. It basically takes whatever the webserver output is and attempts to automatically apply the kinds of best practices that Steve Souders has espoused in his books.
Aptimize makes a software-based FEO solution which can be deployed in a variety of ways, including as a virtual appliance running on a VM. (FEO is generally a computationally expensive thing, though, since it involves lots of text parsing, so it’s not unusual to see it run on a standalone server.)
So, what Riverbed has basically bought itself is the ability to offer a complete optimization solution — WOC, ADC, and FEO — plus the intellectual property portfolio to potentially eventually combine the techniques from all three families of products into an integrated product suite. (Note that Riverbed is fairly cloud-friendly already with its Virtual Steelhead.)
I think this also illustrates the vital importance of “beyond the box” thinking. Networking hardware has traditionally been just that — specialized appliances with custom hardware that can do something to traffic, really really fast. But off-the-shelf servers have gotten so powerful that they can now generate the kind of processing umph and network throughput that you used to have to build custom hardware logic to achieve. That’s leading us to the rise of networking vendors who make software appliances instead, because it’s a heck of a lot easier and cheaper to launch a software company than a hardware company (probably something like a 3:1 ratio in funding needed), you can have product to market and iterate much more quickly, and you can integrate more easily with other products.
ObPlug for new, related research notes (Gartner clients only):
- Riverbed Reports 2Q11 Results, and Acquires Zeus and Aptimize: My colleague Frank Marsala presents an Invest Insight for institutional investors, on Riverbed’s recent announcements.
- How to Accelerate Internet Websites and Applications: My latest note, on how to combine network-based and front-end optimization techniques, looking at CDNs, ADCs, FEO, and more.
Amazon and Equinix partner for Direct Connect
Amazon has introduced a new connectivity option called AWS Direct Connect. In plain speak, Direct Connect allows an Amazon customer to get a cross-connect between his own network equipment and Amazon’s, in some location where the two companies are physically colocated. In even plainer speak, if you’re an Equinix colocation customer in their Ashburn, Virginia (Washington DC) data center campus, you can get a wire run between your cage and Amazon’s, which gives you direct connectivity between your router and theirs.
This is relatively cheap, as far as such things go. Amazon imposes a “port charge” for the cross-connect at $0.30/hour for 1 Gbps or $2.25/hour for 10 Gbps (on a practical level, since cross-connects are by definition nailed up 100% of the time, about $220/month and $1625/month respectively), plus outbound data transfer at $0.02/GB. You’ll also pay Equinix for the cross-connect itself (I haven’t verified the prices for these, but I’d expect they would be around $500 and $1500 per month). And, of course, you have to pay Equinix for the colocation of whatever equipment you have (upwards of $1000/month+ per rack).
Direct Connect has lots of practical uses. It provides direct, fast, private connectivity between your gear in colocation and whatever Amazon services are in Equinix Ashburn (and non-Internet access to AWS in general), vital for “hybrid cloud” use cases and enormously useful for people who, say, have PCI-compliant e-commerce sites with huge databases Oracle RAC and black-box encryption devices, but would like to put some front-end webservers in the cloud. You can also buy whatever connectivity you want from your cage in Equinix, so you can take that traffic and put it over some less expensive Internet connection (Amazon’s bandwidth fees are one of the major reasons customers leave them), or you can get private networking like ethernet or MPLS VPN (an important requirement for enterprise customers who don’t want their traffic to touch the Internet at all).
This is not a completely new thing — Amazon has quietly offered private peering and cross-connects to important customers for some time now, in Equinix. But this now makes cross-connects into a standard option with an established price point, which is likely to have far greater uptake than the one-off deals that Amazon has been doing.
It’s not a fully-automated service — the sign-up is basically used to get Amazon to grant you an authorization so that you can put in an Equinix work order for the cross-connect. But it’s an important step in the right direction. (I’ve previously noted the value of this partnership in a blog post called “Why Cloud IaaS Customers Care About a Colo Option“. Also, for Gartner clients, see my research note “Customers Need Hybrid Cloud Compute IaaS” for a detailed analysis.)
This is good for Equinix, too, for the obvious reasons. For quite some time now, I’ve been evangelizing the importance of carrier-neutral colocation as a “cloud hub”, envisioning a future where these providers facilitate cross-connect infrastructures between cloud users and cloud providers. Widespread adoption of this model would allow an enterprise to say, get a single rack of network equipment at Equinix (or Telecity or Interxion, etc.), and then cross-connect directly to all of their important cloud suppliers. It would drive cross-connect density, differentiation and stickiness at the carrier-neutral colo providers who succeed in being the draw for these ecosystems.
It’s worth noting that this doesn’t grant Amazon a unique capability, though. Just about every other major cloud IaaS provider already offers colocation and private connectivity options. But it’s a crucial step for Amazon towards being suitable for more typical enterprise use cases. (And as a broader long-term ecosystem play, customers may prefer using just one or two “cloud hubs” like an Equinix location for their “cloud backhaul” onto private connectivity, especially if they have gateway devices.)
The forthcoming Public Cloud IaaS Magic Quadrant
Despite having made various blog posts and corresponded with a lot of people in email, there is persistent, ongoing confusion about our forthcoming Magic Quadrant for Public Cloud Infrastructure as a Service, which I will attempt to clear up here on my blog so I have a reference that I can point people to.
1. This is a new Magic Quadrant. We are doing this MQ in addition to, and not instead of, the Magic Quadrant for Cloud IaaS and Web Hosting (henceforth the “cloud/hosting MQ”). The cloud/hosting MQ will continue to be published at the end of each calendar year. This new MQ (henceforth the “public cloud MQ”) will be published in the middle of the year, annually. In other words, there will be two MQs each year. The two MQs will have entirely different qualification and evaluation criteria.
2. This new public cloud MQ covers a subset of the market covered by the existing cloud/hosting MQ. Please consult my cloud IaaS market segmentation to understand the segments covered. The existing MQ covers the traditional Web hosting market (with an emphasis on complex managed hosting), along with all eight of the cloud IaaS market segments, and it covers both public and private cloud. This new MQ covers multi-tenant clouds, and it has a strong emphasis on automated services, with a focus on the scale-out cloud hosting, virtual lab environment, self-managed virtual data center, and turnkey virtual data center segments. The existing MQ weights managed services very highly; by contrast, the new MQ emphasizes automation and self-service.
3. This is cloud compute IaaS only. This doesn’t rate cloud storage providers, PaaS providers, or anything else. IaaS in this case refers to the customer being able to have access to a normal guest OS. (It does not include, for instance, Microsoft Azure’s VM role.)
4. When we say “public cloud”, we mean massive multi-tenancy. That means that the service provider operates, in his data center, a pool of virtualized compute capacity in which multiple arbitrary customers will have VMs on the same physical server. The customer doesn’t have any idea who he’s sharing this pool of capacity with.
5. This includes cloud service providers only. This is an MQ for the public cloud compute IaaS providers themselves — the services focused on are ones like Amazon EC2, Terremark Enterprise Cloud, and so forth. This does not include any of the cloud-enablement vendors (no Eucalyptus, etc.), nor does it include any of the vendors in the ecosystem (no RightScale, etc.).
6. The target audience for this new MQ is still the same as the existing MQ. As Gartner analysts, we write for our client base. These are corporate IT buyers in mid-sized businesses or enterprises, or technology companies of any size (generally post-funding or post-revenue, i.e., at the stage where they’re looking for serious production infrastructure). We expect to weight the scoring heavily towards the requirements of organizations who need a dependable cloud, but we also recognize the value of commodity cloud to our audience, for certain use cases.
At this point, the initial vendor surveys for this MQ have been sent out. They have gone out to every vendor who requested one, so if you did not get one and you wanted one, please send me email. We did zero pre-qualification; if you asked, you got it. This is a data-gathering exercise, where the data will be used to determine which vendors get a formal invitation to participate in the research. We do not release the qualification criteria in advance of the formal invitations; please do not ask.
If you’re a vendor thinking of requesting a survey, please consider the above. Are you a cloud infrastructure service provider, not a cloud-building vendor or a consultancy? Is your cloud compute massively multi-tenant? Is it highly automated and focused on self-service? Do you serve enterprise customers and actively compete for enterprise deals, globally? If the answers to any of these questions are “no”, then this is not the MQ for you.
App categorization and the commodity vs. dependable cloud
At the SLA@SOI conference, my colleague Drue Reeves gave a presentation on the dependable cloud, which he defined as “a cloud service that has the availability, security, scalabilty, and risk management necessary to host enterprise applications… at a reasonable price.” We’ll be publishing research on this in the months to come, so this blog post contains relatively early-stage musings on my part.
We need enterprise-grade, dependable cloud infrastructure as as service (IaaS). But there’s also a place in the world for commodity cloud IaaS. They serve different sorts of use cases, different categories of applications. (Everything in this post refers to IaaS, but I’m just saying “cloud” for convenience.)
There are four types of applications that will move into the cloud:
- Existing enterprise applications, capable of being virtualized
- New enterprise-class applications, almost certainly Web-based
- Internet-class applications, Web 1.0 and early Web 2.0
- Global-class applications, highly sophisticated super-scalable Web 2.0 and beyond
Enterprise-class applications are generally characterized by the expectation that the underlying infrastructure is at least as reliable, performant, and secure as traditional enterprise data center infrastructure. They expect resilience at the infrastructure layer. Over the last decade, applications of this type have generally been written as three-tier, Web-based apps. Nevertheless, these apps often scale vertically rather than horizontally (scale up rather than scale out), but a very large percentage of them are small applications — ones that use a core or less of a modern CPU — and so even if they could scale out on multiple VMs, it often doesn’t make sense, from a capacity efficiency standpoint, to deploy them that way.
In the future, while an increasing percentage of new business applications will be obtained as SaaS, rather than being internally-hosted COTS apps or in-house-written apps, and more will be deployed onto business process management (BPM) suite platforms or the like, businesses will still be writing custom apps of this sort. So we will continue to need dependable infrastructure.
Moreover, many enterprise-class applications are written not just by business IT, but also by external vendors, whether ISVs, SaaS, or otherwise. Even tech companies that make their living off their websites may write enterprise-class apps. Indeed, many such apps have previously used managed hosting for the underlying infrastructure, and these companies have infrastructure dependability as an expectation.
By contrast, Internet-class applications are written to scale out. They might or might not be written to be easily distributed. They assume sufficient scale that there is an expectation that at least some things can fail without causing widespread failure, although there may still be particularly vulnerable points in the app and underlying infrstracture — the database, for instance. Resilience is generally built into the application, but these are not apps designed to withstand the Chaos Monkey.
Finally, global-class applications are written to be scale-out, highly-distributed, and to withstand massive infrastructure failures. All the resiliency is built into the application; the underlying infrastructure is assumed to be fragile. Simple underlying infrastructure components that fail cleanly and quickly (rather than dying slow deaths of degradation) are prized, because they are cheap to buy and cheap to replace; all the intelligence resides in software.
Global-class applications can use commodity cloud infrastructure, as can other use cases that do not expect a dependable cloud. Internet-class applications can also use commodity cloud infrastructure, but unless efforts are made to move more resiliency into the application layer, there are risk management issues here, and depending upon scale and needs, a dependable cloud may be preferable to commodity cloud. Enterprise-class applications need a dependable cloud.
Where resiliency resides is an architectural choice. There is no One True Way. Building resilience into the app may be the most cost-effective choice for applications which need to have “Internet scale”, but it may add unwarranted and unnecessary complexity to many other applications, making dependable infrastructure the more cost-effective choice.
CloudPundit: Massive-Scale Computing 