Daily Archives: November 9, 2011
IT Operations and button-pushing
The fine folks at Nodeable gave me an informal introductory briefing today; they’ve got a pretty cool concept for a cloud-oriented monitoring and management SaaS-based tool that’s aimed at DevOps.
I’ve been having stray thoughts on DevOps and the future of IT Operations in the couple of hours that have passed since then, and reflecting on the following problem:
At an awful lot of companies, IT Operations, especially lower-level folks, are button-pushing monkeys — specifically, they are people who know how to use the vendor-supplied GUI to perform particular tasks. They may know the vendor-recommended ways to do things with a particular bit of hardware or software. But only a few of them have architect-level knowledge, the deep understanding of the esoterica of systems and how this stuff is actually built and engineered. (Some of this is a reflection of education; a lot of IT Operations people don’t come from a computer science background, but have what they’ve needed to know on the job.)
Today’s DevOps person is likely to have a skillset that we used to call systems programming. They understand systems architecture, they understand operating systems, they can write system-level code, including the scripting necessary for automation. The programmatic access to infrastructure exemplified by cloud IaaS providers has moved this up a layer of abstraction, so that you don’t have to be a deep-voodoo guy to do this kind of thing.
We’re moving towards a world where you have really low-level button-pushers — possibly where the button-pushing is so simple that you don’t need a specialist to do it any longer, anyone reasonably technical can do it — and senior architects whoo design things, and systems programmers who automate things. Whether those systems programmers work in application development and are “DevOps”, or whether those systems programmers work in IT Operations and just happen to be systems guys who program (mostly scripting), doesn’t really matter — the era of the button-pusher is drawing towards its close either way, at least for organizations who are going to efficiently increase IT Operations efficiency.
I want to share a story. It is, in some ways, a story about cruelty and unprofessionalism, but it’s funny in its own way.
About fifteen years ago, I was working as an engineer at Digex (the first real managed hosting company). We had a pretty highly skilled group of engineers there, and we never did anything using a GUI. We had hundreds of customers on dedicated Sun servers, and you’d either SSH into the systems or, in a pinch, go to the data center and log in on console. We were also the kind of people who would fix issues by making kernel modifications — for instance, the day that the SYN flood attack showed up, a bunch of customers went down hard, meaning that we could not afford to wait for Sun to come up with a patch, since we had customer SLAs to meet, so one of our security engineers rewrote the kernel’s queueing code for TCP accepts.
We were without a manager for some time, and they finally hired a guy who was supposedly a great Sun sysadmin. He didn’t actually get a technical interview, but he had a good work history of completed projects and happy teams and so forth. He was supposed to be both the manager and the technical lead for the team.
The problem was that he had no idea how to do anything that wasn’t in Sun’s administrator GUI. He didn’t even know how to attach a console cable to a server, much less log in remotely to a system. Since we did absolutely nothing with a GUI, this was a big problem. An even bigger problem was that he didn’t understand anything about the underlying technologies we were supporting. If he had a problem, he was used to calling Sun and having them tell him what to do. This, clearly, is a big problem in a managed hosting environment where you’re the first line of support for your customers, who may do arbitrary wacky things.
He also worked a nine-to-five day at a startup where engineers routinely spent sixteen hours at work. His team, and the other engineers at the company, had nothing but contempt for him. And one night, having dinner at 10 pm as a break before going right back into work, someone had an idea.
“Let’s recompile his kernel without mouse support.” (Like all the engineers, he had a Sun workstation at his desk.)
And so when he came to work the next morning, his mouse didn’t work — and every trace of the intrusion had been covered, thanks to the complicity of one of the security engineers.
Someone who had an idea of what he was doing wouldn’t have been phazed; they’d have verified the mouse wasn’t working, then done an L1-A to put the workstation into PROM mode, and easily done troubleshooting from there (although admittedly, nobody thinks, “I wonder if somebody recompiled my kernel without mouse support after I went home last night”). This poor guy couldn’t do anything other than pick up his mouse to make sure the underside hadn’t gotten dirty. It turned out that he had no idea how to do anything with the workstation if he couldn’t log in via the GUI.
It proved to be a remarkably effective demonstration to management that this guy was a yahoo and needed to be fired. (Fortunately, there were plenty of suspect engineers, and management never found out who was responsible. Earl Galleher, who ran that part of the business at the time, and is the chairman at Basho now, probably still wonders… It wasn’t me, Earl.)
But it makes me wonder what is the future of all the GUI masters in IT Operations, because the world is evolving to be more like the teams that I had before I came to Gartner — systems programmers with strong systems and operations skills, who could also code.
DevOps: Now you know how to deal with the IT Operations guy who can only use a GUI…
Common service provider myths about cloud infrastructure
We’re currently in the midst of agenda planning for 2012, which is a fancy way to say that we’re trying to figure out what we’re going to write next year. Probably to the despair of my managers, I am almost totally a spontaneous writer, who sits down on a plane and happens to write a research note on whatever it is that’s occurred to me at the moment. So I’ve been pondering what to write, and decided that I ought to tap into the deep well of frustration I’ve been feeling about the cloud IaaS market over the last couple of months.
Specifically, it started me in on thinking about the most common fallacies that I hear from current cloud IaaS providers, or from vendors who are working on getting into the business. I think each of these things is worthy of a research note (in some cases, I’ve already written one), but they’re also worth a blog post series, because I have the occasional desire to explode in frustrated rants. Also, when I write research, it’s carefully polite, thoughtfully-considered, heavily-nuanced, peer-reviewed documents that will run ten to twenty pages and be vaguely skimmed, often by mid-level folks in product marketing. If I write a blog post, it will be short and pointed and might actually get the point through to people, especially the executives who are more likely to read my blog than my research.
So, here’s the succinct list to be explored in further posts. These are things I have said to vendor clients in inquiries, in politely measured terms. These are the blunt versions:
Doing this cloud infrastructure thing is hard and expensive. Yes, I know that VMware told you that you could just get a VCE Vblock, put VMware’s cloud stack on it (maybe with a little help from VMware consulting), and be in business. That’s not the case. You will be making a huge number of engineering decisions (most of which can screw you in a variety of colorful ways, either immediately or down the road). You will be integrating a ton of tools and doing a bunch of software development yourself, if you want to have a vaguely competitive offering for anything other than the small business migrating from VPS. Ditto if you use Citrix (Cloud.com), OpenStack, or whomever. Even with professional services to help you. And once you have an offering, you will be in a giant competitive rat race where the best players innovate fast, and the capabilities gap widens, not closes. If you’re not up to it, white-label, resell, or broker instead.
There is more to the competition than Amazon, but ignore Amazon at your peril. Sure, Amazon is the market goliath, but if your differentiation is “we’re not like Amazon, we’re enterprise-class!”, you’re now competing against te dozens of other providers who also thought that would be a clever market differentiation. Not to mention that Amazon already serves the enterprise, and wants to deepen its inroads. (Where Amazon is hurting is the mid-market, but there’s tons of competition there, too.) Do you seriously think that Amazon isn’t going to start introducing service features targeted at the enterprise? They already have, and they’re continuing to do so.
Not everything has to be engineered to five nines of availability. Many businesses, especially those moving legacy workloads, need reliable, consistently high-performance infrastructure. Howeve, most businesses shouldn’t get infrastructure as one-size-fits-all — this is part of what is making internal data centers expensive. Instead, cloud infrastructure should be tiered — one management portal, one API, multiple levels of service at different price points. “Everything we do is enterprise-class” unfortunately implies “everything we do is expensive”.
Your contempt for the individual developer hugely limits your sales opportunities. Developers are the face of the business buyer. They are the way that cloud IaaS makes inroads into traditional businesses, including the largest enterprises. This is not just about start-ups or small businesses, or about the companies going DevOps.
Prospective customers will not call Sales when your website is useless. Your lack of useful information on your website doesn’t mean that eager prospects will call sales wanting to know what wonderful things you have. Instead, they will assume that you suck, and you don’t get the cloud, and you are hiding what you have because it’s not actually competitive, and they will move on to the dozens of other providers trying to sell cloud IaaS or who are pretending to do so. Also, engineers hate talking to salespeople. Blind RFPs are common in this market, but so is simply signing up with a provider that doesn’t make it painful to get their service.
Just because you don’t take online sign-ups doesn’t mean your cloud is “safe”. Even if you only take “legitimate businesses”, customers make mistakes and their infrastructure gets compromised. Sure, your security controls might ensure that the bad guys don’t compromise your other customers. But that doesn’t mean you won’t end up hosting command-and-control for a botnet, scammers, or spammers, inadvertently. Service providers who take credit card sign-ups are professionally paranoid about these things; buyers should beware providers who think “only real businesses like you can use our cloud” means no bad guys inside the walls.
Automation, not people, is the future. Okay, you’re more of a “managed services” kind of company, and self-service isn’t really your thing. Except “managed services” are, today, basically a codeword for “expensive manual labor”. The real future value of cloud IaaS is automating the heck out of most of the lower-end managed services. If you don’t get on that bandwagon soon, you are going to eventually stop being cost-competitive — not to mention that automation means consistency and likely higher quality. There’s a future in having people still, but not for things that are better done by computers.
Carriers won’t dominate the cloud. This opinion is controversial. Of course, carriers will be pretty significant players — especially since they’ve been buying up the leading independent cloud IaaS providers. But many other analyst firms, and certainly the carriers themselves, believe that the network, and the ability to offer an end-to-end service, will be a key differentiator that allows carriers to dominate this business. But that’s not what customers actually want. They want private networking from their carrier that connects them to their infrastructure — which they can get out of a carrier-neutral data center that is a “cloud hub”. Customers are better off going into a cloud hub with a colocated “cloud gateway” (with security, WAN optimization, etc.), cross-connecting to their various cloud providers (whether IaaS, PaaS, SaaS, etc.), and taking one private network connection home.
Stay tuned. More to come.