Blog Archives
The challenge of hiring development teams
A recent blog post on Forbes by Venkatesh Rao, The Rise of Developernomics, has ignited a lot of controversy around the concept that some developers are as much as 10x more productive than others. It’s not a new debate; the assertion that some developers are 20x more productive than others has been around forever, and folks like Jole Spolsky have asserted that it’s not just a matter of productivity, but also a developer’s ability to hit the high notes of real breakthrough achievement that makes for greatness.
Worth reading out of all of these threads: Avichal Garg of Spool’s blog post on building 10x teams, which has a very nice dissection of the composition of great teams.
Also, for those of you who haven’t read it: Now Discover Your Strengths
Jon Evans of TechCrunch wrote a good blog post a few months ago, Why the New Guy Can’t Code, which illustrates the challenges of hiring good developers. (There are shocking numbers of developers out there who have never really produced significant code in their jobs. Indeed, I once interviewed a developer with five years of experience who had never written code in a work context — he kept being moved from project to project that was only in the formal requirements phase, so all he had was his five-years-stale student efforts from his CS degree.)
Even with the massive pile of unemployed developers out there, it’s still phenomenally challenging to hire good people. And if your company requires a narrow and specific set of things that the developer must have worked with before, rather than hiring a smart Swiss army knife of a developer who can pick up anything given a few days, you will have an even bigger problem, especially if you require multiple years of experience with brand-new technologies like AWS, NoSQL, Hadoop, etc.
With more and more Web hosters, systems integrators, and other infrastructure-specialist companies transforming themselves into cloud providers, and sometimes outright buying software companies (such as Terremark buying CloudSwitch, and Virtustream buying Enomaly), serious software development chops are becoming a key for a whole range of service providers who never really had significant development teams in the past. No one should underestimate how much of a shortage there is for great talent.
As a reminder, Gartner is hiring!
Designing to fail
Cloud-savvy application architects don’t do things the same way that they’re done in the traditional enterprise.
Cloud applications assume failure. That is, well-architected cloud applications assume that just about anything can fail. Servers fail. Storage fails. Networks fail. Other application components fail. Cloud applications are designed to be resilient to failure, and they are designed to be robust at the application level rather than at the infrastructure level.
Enterprises, for the most part, design for infrastructure robustness. They build expensive data centers with redundant components. They buy expensive servers with dual-everything in case a component fails. They buy expensive storage and mirror their disks. And then whatever hardware they buy, they need two of. All so the application never has to deal with the failure of the underlying infrastructure.
The cloud philosophy is generally that you buy dirt-cheap things and expect they’ll fail. Since you’re scaling out anyway, you expect to have a bunch of boxes, so that any box failing is not an issue. You protect against data center failure by being in multiple data centers.
Cloud applications assume variable performance. Well-architected cloud applications don’t assume that anything is going to complete in a certain amount of time. The application has to deal with network latencies that might be random, storage latencies that might be random, and compute latencies that might be random. The principle of the distributed application of this sort is that just about anything that you’re talking to can mysteriously drop off the face of the Earth at any point in time, or at least not get back to you for a whlie.
Here’s where it gets funkier. Even most cloud-savvy architects don’t build applications this way today. This is why people howl about Amazon’s storage back-end for EBS, for instance — they’re used to consistent and reliable storage performance, and EBS isn’t built that way, and most applications are built with the assumption that seemingly local standard I/O is functionally local and therefore is totally reliable and high-performance. This is why people twitch about VM-to-VM latencies, although at least here there’s usually some application robustness (since people are more likely to architect with network issues in mind). This is the kind of problem things like Node.js were created to solve (don’t block on anything, and assume anything can fail), but it’s also a type of thinking that’s brand-new to most application architects.
Performance is actually where the real problems occur when moving applications to the cloud. Most businesses who are moving existing apps can deal with the infrastructure issues — and indeed, many cloud providers (generally the VMware-based ones) use clustering and live migration and so forth to present users with a reliable infrastructure layer. But most existing traditional enterprise apps don’t deal well with variable performance, and that’s a problem that will be much trickier to solve.
Cloud ecosystems for small businesses
As I’ve been predicting for a while, Microsoft and Intuit have joined forces around Quickbooks and Azure: Microsoft and Intuit announced that Intuit would name Microsoft’s Windows Azure as the preferred platform for cloud app development on its Intuit Partner Platform. This is an eminently logical partnership. MSDN developers, are a critical channel for reaching the small business with applications, Azure is evolving to be well-suited to that community, and Intuit’s Quickbooks is a key anchor application for the small business. Think of this partnership as the equivalent of Force.com for the small business; arguably, Quickbooks is an even more compelling anchor application for a PaaS ecosystem than CRM is.
A lot of non-IT companies are thinking about cloud strategies these days. I get a great deal of inquiry from companies seeking to target the small business with cloud offerings, and the question that I keep having to ask is, “What natural value does your existing business bring when extended to the cloud?” An astounding number of strategy people at miscellaneous companies seem to believe that they ought to be cloud IaaS providers, or resellers of other people’s SaaS solutions for small businesses — without being natural places for small businesses to turn for either infrastructure or software.
Whatever your business is, if you want to create a cloud ecosystem, you need an anchor service. Take something that you do today, and leverage cloud precepts. Consider doing something like creating a data service around it, opening up an API, and the like. (Gartner clients: My colleague Eric Knipp has written a useful research note on this topic entitled Open RESTful APIs are Big Business.) Use that as the centerpiece for an ecosystem of related services from partners, and the community of users.
Are multiple cloud APIs bad?
Rackspace has recently launched a community portal called Cloud Tools, showcasing third-party tools that support Rackspace’s cloud compute and storage services. The tools are divided into “featured” and “community”. Featured tools are ones that Rackspace has looked at and believes deserve highlighting; they’re not necessarily commercial projects, but Rackspace does have formal relationships with the developers. Community tools are fro any random joe out there who’d like to be listed. The featured tools get a lot more bells and whistles.
While this is a good move for Rackspace, it’s not ground-breaking stuff, although the portal is notable for a design that seems more consumer-friendly (by contrast with Amazon’s highly text-dense, spartan partner listings). Rather, what’s interesting is Rackspace’s ongoing (successful) efforts to encourage an ecosystem to develop around its cloud APIs, and the broader question of cloud API standardization, “de facto” standards, and similar issues.
There are no small number of cloud advocates out there that believe that rapid standardization in the industry would be advantageous, and that Amazon’s S3 and EC2 APIs, as the APIs with the greatest current adoption and broadest tools support, should be adopted as a de facto standard. Indeed, some cloud-enablement packages, like Eucalyptus, have adopted Amazon’s APIs — and will probably run into API dilemmas as they evolve, as private cloud implementations will be different than public ones, leading to inherent API differences, and a commitment to API compatibility means that you don’t fully control your own feature roadmap. There’s something to be said for compatibility, certainly. Compatibility drives commoditization, which would theoretically lower prices and deliver benefits to end-users.
However, I believe that it’s too early in the market to seek commoditization. Universal commitment to a particular API at this point clamps standardized functionality within a least-common-denominator range, and it restricts the implementation possibilities, to the detriment of innovation. As long as there is rapid innovation and the market continues to offer a slew of new features — something which I anticipate will continue at least through the end of 2011 and likely beyond — standardization is going to be of highly limited benefit.
Rackspace’s API is different than Amazon’s because Rackspace has taken some different fundamental approaches, especially with regard to the network. For another example of significant API differences, compare EMC’s Atmos API to Amazon’s S3 API. Storage is a pretty simple thing, but there are nevertheless meaningful differences in the APIs, reflecting EMC’s different philosophy and approach. (As a sideline, you might find William Vambenepe’s comparison of public cloud APIs in the context of REST, to be an interesting read.)
Everyone can agree on a certain set of core cloud concepts, and I expect that we’ll see libraries that provide unified API access to different underlying clouds; for instance, libcloud (for Python) is the beginning of one such effort. And, of course, third parties like RightScale specialize in providing unified interfaces to multiple clouds.
One thing to keep in mind: Most of the cloud APIs to date are really easy to work with. This means that if you have a tool that supports one API, it’s not terribly hard or time-consuming to make it support another API, assuming that you’re confining yourself to basic functionality.
There’s certainly something to be said in favor of other cloud providers offering an API compatibility layer for basic EC2 and S3 functionality, to satisfy customer demand for such. This also seems to be the kind of thing that’s readily executed as a third-party library, though.
Google and Salesforce.com
While I’ve been out of the office, Google has made some significant announcements. My colleague Ray Valdes has been writing about Google Wave and its secret sauce. I highly encourage you to go read his blog.
Google and Salesforce.com continue to build on their partnership. In April, they unveiled Salesforce for Google Apps. Now, they’re introducing Force.com for Google App Engine.
The announcement, in a nutshell, is this: There are now public Salesforce APIs that can be downloaded, and will work on Google App Engine (GAE). Those APIs are a subset of the functionality available in Force.com’s regular Web Services APIs. Check out the User Guide for details.
Note that this is not a replacement for Force.com and its (proprietary) Apex programming language. Salesforce clearly articulates web services vs. Force.com in its developer guide. Rather, this should be thought of as easing the curve for developers who want to extend their Web applications for use with Salesforce data.
A question that lingers in my mind: Normally, on Force.com, a Developer Edition account means that you can’t affect your organization’s live data. If a similar restriction exists on the GAE version of the APIs, it’s not mentioned in the documentation. I wonder if you can do very lightweight apps, using live data, with just a Developer Edition account with Salesforce, if you do it through GAE. If so, that would certainly open up the realm of developers who might try building something on the platform.
My colleague Eric Knipp has also blogged about the announcement. I’d encourage you to read his analysis.
Google App Engine and other tidbits
As anticipated, Java support on Google App Engine has been announced. To date, GAE has supported only the Python programming language. In keeping with the “phenomenal cosmic power, itty bitty living space” sandboxing that’s become common to cloud execution environments, GAE/Java has all the restrictions of GAE/Python. However, the already containerized nature of Java applications means that the restrictions probably won’t feel as significant to developers. Many Python libraries and frameworks are not “pure Python”; they include C extensions for speed. Java libraries and frameworks are, by contrast, usually pure Java; the biggest issues for porting Java into the GAE environment are likely to be the restrictions on system calls and the lack of threads. Generically, GAE/Java offers servlets. The other things that developers are likely to miss are support for JMS and JMX (Java’s messaging and monitoring, respectively).
Overall, the Java introduction is a definite plus for GAE, and is presumably also an important internal proof point for them — a demonstration that GAE can scale and work with other languages. Also, because there are lots of languages that now target the Java virtual machine (i.e., they’ve got compilers/interpreters that produce byte code for the Java VM) — Clojure and Scala, for instance — as well as ports of other languages, like JRuby, we’ll likely see additional languages available on GAE ahead of Google’s own support for those environments.
Google also followed through on an earlier announcement, adding support for scheduld tasks (“cron”). Basically, at a scheduled time, GAE cron will invoke a URL that you specify. This is useful, but probably not everything people were hoping it would be. It’s still subject to GAE’s normal restrictions; this doesn’t let you invoke a long-running background process. It requires a shift in thinking — for instance, instead of doing the once-daily data cleanup run at 4 am, you ought to be doing cleanup throughout the day, every couple of minutes, a bit of your data set at a time.
All of that is going to be chewed over thoroughly by the press and blogosphere, and I’ve contributed my two cents to a soon-to-be-published Gartner take on the announcement and GAE itself, so now I’ll point out something that I don’t think has been widely noticed: the unladen-swallow project plan.
unladen-swallow is apparently an initiative within Google’s compiler optimization team, with a goal of achieving a 5x speed-up in CPython (i.e., the normal, mainstream, implementation of Python), starting from the 2.6 base (the current version, which is a transition point between the 2.5 used by App Engine, and the much-different Python 3.0). The developers intend to achieve this speed-up in part by moving from the existing custom VM to one built on top of LLVM. (I’ve mentioned Google’s interest in LLVM in the past.) I think this particular approach answers some of the mystery surrounding Google and Python 3.0 — this seems to indicate longer-term commitment to the existing 2.x base, while still being transition-friendly. As is typical with Google’s work with open-source code, they plan to release these changes back to the community.
All of which goes back to a point of mine earlier this week: Although programming language communities strongly resemble fandoms, languages are increasingly fungible. We’re a long way from platform maturity, too.
Scala, Ruby, cost, and development trends
A recent interview of some Twitter developers, on Twitter’s use of Scala has touched off a fair amount of controversy in the Ruby community, and prompting Todd Hoff of the High Scalability to muse on an interesting statement: At some point, the cost of servers outweighs the cost of programmers.
We all know that the scripting languages that are frequently favored in Web development today — Ruby, Python, and PHP — do not perform as well as Java, and Java in turn can be outperformed by well-written native C/C++ code. However, these popular dynamic programming languages typically lead to better programmer productivity. The argument has been that it’s more cost-effective to have more productive developers, than it is to buy less infrastructure. There is a point, though, when that scale equation can be flipped on its head — when the cost of the servers, due to the performance sacrifices, gets too high. (I would add that you can’t look at simple hardware spend alone, either. You’ve got a infrastructure TCO to look at. It’s not just about more people to maintain more servers, either — that equation is not linear, as a sysadmin can manage more systems if they’re all identical and there are good automation tools. But systems that are struggling due to performance issues soak up operations time with daily firefighting.)
Twitter’s developers are not advocating that people abandon what they know and love, but they’re forging a new path for themselves, with an open-source language developed in academia. Scala can be compiled to either Java or .NET bytecode, allowing it to interoperate bidirectionally with Java and CLR code; this is important for driving adoption because programmers generally like to work with languages that have a solid base of libraries (i.e., someone else has conveniently done the work of producing code for commonly-needed capabilities), and because this makes it possible for Scala to leverage the existing tools community for Java and .NET. Scala’s equivalent of Rails, i.e., a convenient framework, is Lift.
Scala doesn’t have much adoption now, but it’s worth noting that the rapid pace of Web 2.0 innovation is capable of driving extremely fast uptake of things that turn out to solve real-world problems. (For comparison: Not long ago, practically no one had heard of Hadoop, either, but it’s built quite a bit of buzz now.) That’s important for anyone contemplating the long-term future of particular platforms, particularly APaaS offerings that are tied to specific programming languages. The favored platforms can and do change in a tidal fashion — just look at the Google trend graph for Ruby on Rails to see just how aggressively interest can increase over a single year (2005 to 2006).
As a coda to all of this, Twitter’s Alex Payne has a smart blog post, noting that social media fills the vacuum between peer-reviewed journals and water-cooler conversations, yet deploring the fact that in these mediums, emotion can rule over what is measurable. The takeaway — whether you’re an IT manager, a marketing manager at a vendor, or an investor — from my perspective, is this: There’s an emotional context to programming language choice. These are not merely technical communities; these are fandoms, and they form part of a developer’s self-identity.
AWS in Eclipse, and Azure announcements
Amazon’s announcement for today, with timing presumably associated with EclipseCon, is an AWS toolkit for the Eclipse IDE.
Eclipse, which is an open-source project under the aegis of IBM (who also offers a commercial version), is one of the most popular IDEs (the other is Microsoft Visual Studio). Originally designed for Java applications, it has since been extended to support many other languages and environments.
Integrating with Eclipse is a useful step for Amazon, and hopefully other cloud providers will follow suit. It’s also a competitive response to the integration that Microsoft has done between Visual Studio and its Azure platform.
Speaking of Azure, as part of a set of announcements, Microsoft has said that it’s supporting non-.Net languages on Azure via FastCGI. FastCGI is a webserver extension that basically compiles and loads your scripts once, instead of every time they’re accessed, resulting in a reduction of computational overhead. You can run most languages under it, including Java, but it doesn’t really give you the full featureset that you get with tight integration with the webserver through a language-specific extension. (Note that because .NET’s languages encompass anything that supports the CLR, users already had some reasonable access to non-C# languages on Azure — implementations like Ruby.NET, IronRuby, IronPython, etc.)
Also, in an interesting Q&A on a ZDnet blog post, Microsoft said that there will be no private Azure-based clouds, i.e., enterprises won’t be able to take the Azure software and host it in their own data centers. What’s not clear is whether or not the software written for Azure will be portable into the enterprise environment. Portability of this sort is a feature that Microsoft, with its complete control over the entire stack, is uniquely well-positioned to be able to deliver.
A little SourceForge frustration
SourceForge puzzles me. I think it’s the combination of what is obviously eager effort to improve the site, and the fumbling to get the basics right.
On the plus side, SourceForge recently made a very welcome addition — adding “hosted apps”, including WordPress and MediaWiki — as an option for all projects, for free. And the announcement of support for additional repository types, notably git, is also a nice move.
But SourceForge is plagued by sluggish response (which is especially stark when compared to the consistent zippiness of Google Code) — across its website, source code repositories, etc. — as well as occasional outages. And the continual redesign of the site, especially in its current bright-orange incarnation, hasn’t seemed like a positive to me. With every redesign, I’ve felt like SourceForge was becoming harder and harder to use. As an example, one redesign ago, the Project Admin menu got so long it was basically unusable on smaller screens (like laptops). To SourceForge’s credit, the next iteration promptly fixed it; unfortunately, the chosen fix was by burying vitally important functionality like the file release system under the “Feature Settings” page (found under Project Admin). That led me on a wild hunt through most of the UI before I finally stumbled upon it the functionality I was looking for by accident.
SourceForge offers a tremendous amount of functionality for free, which is what’s allowing it to stay dominant against the proliferating number of alternative services out there. But not only does SourceForge need to innovate, it needs to make sure that it gets the basics right. It has to add functionality while still being fast and simple to use, and over the years, SourceForge seems to have grown tendrils of new features while the main octopod body has grown sessile and mottled with confusion.
Google App Engine updates
For those of you who haven’t been following Google’s updates to App Engine, I want to call your attention to a number of recent announcements. At the six-month point of the beta, I asked when App Engine would be enterprise-ready; now, as we come to almost the year mark, these announcements show the progress and roadmap to addressing many of the issues I mentioned in my previous post.
Paid usage. Google is now letting applications grow beyond the free limits. You set quotas for various resources, and pay for what you use. I still have concerns about the quota model, but being able to bill for these services is an important step for Google. Google intends to be price-competitive with Amazon, but there’s an important difference — there’s still some free service. Google anticipates that the free quotas are enough to serve about five million page views. 5 MPVs is a lot; it pretty much means that if you’re willing to write to the platform, you can easily host your hobby project on it for free. For that matter, many enterprises don’t get 5 MPVs worth of hits on an individual Web app or site each month — it’s just that the platform restrictions are a barrier to mainstream adoption.
Less aggressive limits and fewer restrictions. Google has removed or reduced some limits and restrictions that were significant frustrations for developers.
Promised new features. Google has announced that it’s going to provide APIs for some vital bits of functionality that it doesn’t currently allow, like the ability to run scheduled jobs and background processes.
Release of Python 3.0. While there’s no word on how Google plans to manage the 3.0 transition for App Engine, it’s interesting to see how many Python contributors have been absorbed into Google.
Speaking personally, I like App Engine. Python is my strongest scripting language skill, so I prefer to write in it whenever possible. I also like Django, though I appreciate that Google’s framework is easier to get started with than Django (it’s very easy to crank out basic stuff). Like a lot of people, I’ve had trouble adjusting to the non-relational database, but that’s mostly a matter of programming practice. It is, however, clear that the platform is still in its early stages. (I once spent several hours of a weekend tearing my hair out at something that didn’t work, only to eventually find that it was a known bug in the engine.) But Google continues to work at improving it, and it’s worth keeping an eye on to see what it will eventually become. Just don’t expect it to be enterprise-ready this year.
CloudPundit: Massive-Scale Computing 