Thursday, August 20, 2009

I'm still waiting for a World World Conference

You know you've arrived as a technology when you get a 'World' conference named after you. Such apparently is the case with Apache Hadoop, the Java-based distributed data storage and analysis framework built by Doug Cutting and others initially while developing the Lucene-based Nutch Search Engine, on a foundation of its own distributed file system and Google's Map-Reduce programming model. This might seem like a particularly narrow focus for a technology conference but it goes to show the many and varied uses of Hadoop by the industry.

The conference won't really be about Hadoop, of course - it'll be about getting a look at the details behind all those interesting problems people are using the framework to help solve. This class of problem is anything but narrow. Massively distributed, massively concurrent data storage, retrieval and analysis/processing challenges are an ever growing reality for an ever growing percentage of IT shops across the world. It certainly is where I hang my hat (or where I would hang my hat if I was losing my hair and needed to wear one). We don't yet use Hadoop (or the techniques behind it) but we probably should.

No comments:

Post a Comment