Those who can do sometimes can still teach

I just finished Martin Fowler's new opus, Domain Specific Languages.  Years in the making, it's a good read.  It's Fowler, so that's not a surprise: even if I disagreed with everything he wrote, I'd still enjoy reading his work.  I don't want to regurgitate it here but suffice to say that the book is laid out in a form familiar to Fowler fans (the first half is a narrative and the second, a reference).   You may not think DSLs affect what you're currently into but if you're involved with technology at all, you'd be wrong.  You might also be surprised at what Fowler considers DSLs and his ideas around fluent interface driven APIs (fluent APIs are generally a bad thing from a design perspective when viewed purely in an imperative light but are essential in the context of a DSL).

Speaking of good writers / communicators, I've been lucky enough to have been introduced to James Iry's blog in recent months, now that I've down deep into the Scala rabbit hole at work.  I'm into my third month of every day Scala coding and loving it more having had a chance to develop in anger with the language.  I was off put by Scala when first playing around with it last year, thinking a hybrid Object/Functional language was a Bad Thing.  It most certainly can be a bad and overly complicated thing, but that's true more or less of any language in the wrong hands.  But with the right infrastructure built up around the scala core (maven plugins, the right libraries, Spring, Scala IDE plugins - Eclipse, Intellij and Netbeans all have working versions that are everyday getting better), it's the simplicity that I'm left with.  And I love the Object/Functional bifurcation (OO in the large and functional in the small tends to be the best approach for most things, I've found).  Anyway, back to James.  The key of success with Scala (or any functional language) is to understand the basics around FP.  One of the key concepts is the Monad.  The concepts and properties around Monads can be a tad intimidating to the uninitiated but like many things, at their core, Monads are pretty basic.  James has some of the best blog posts around such topics, laid out in such a way even us mere mortals finally get it.   His 4-parter on Monads are a case in point.  Check it out.

Continuous Continuum

There are classics in technology literature, much like there is in any other category of writing.  Many classics are at least superficially dated and for more than a few, this runs deeper.  While this might be true for other forms, the rapid pace at which technology evolves means that a software engineering tome is much more susceptible to obsolescence than a novel concerning the folly of war.  And, of course, the pace at which something can attain "classic" status tends to be much quicker in the tech world as well.    I think there are at least a few books out there (and more every year) that I keep coming back to, and not just as a novelty (like when I pick up one of my original X Windows books from the 80s for a chuckle).    Fowler's  Refactoring, Brian Goetz's Concurrency in Java and Joshua Bloch's Effective Java are books that don't get old for me (and the later two are not great only within the context of the Java programming language).

Well, there's a new book in town.  Too early to tell if it'll take its place among those other tomes in my pantheon, but I have a feeling it'll make it to the second tier (along side such notables as Sam Ruby's RESTful Web Services, Kent Beck's Test Driven Development, Peter Coad's Java Design and Martin Ordesky's Programming in Scala).  The book?  Continuous Delivery by Jez Humble and David Farley.  It doesn't have anything ground breaking in it (technology books rarely do - the industry moves much too fast  while the publishing industry lumbers along much too slowly for that to happen).  But the subject itself is, to my mind, one of the most fundamental pieces of the IT pie, growing ever more fundamental as an organization grows.  It's also fundamental if utility computing is ever to approach its potential and become the underpinnings of a real enterprise level transformation (cloud everything hype aside).  

I've been consulting for the better part of 10 years now and have probably spent about half that time in trying to bridge the gulf between development and operations (and the other half helping to automate bits and pieces of the build/deploy/provision/configure puzzle or troubleshooting problems when some manual portion of that death march went haywire).    So it's a subject near and dear to my heart.  The book's well written and I'm liking it thus far (I just hope enough people who are not already members of the choir get their hands on this religion).    Let's pray that it does.

Deck Lifting

I'm cutting code on the deck and recovering from a wicked sunburn. I know better than to venture outdoors without SPF 10000 plastered head to toe or a space suit. But apparently not better enough. I'm attempting to learn the lift framework by knocking out a pet store (hey, I'm relatively old school). All part of my year-long scalification. Lift is a nice web framework, not that we needed another to choose from. I wouldn't pick it to use for a real world app, probably, but it suits my immediate needs. It seems that even with all the fancy frameworks and increasing choice of languages, the LAMP stack still rules for large scale web apps, at least on the front end. As it should be, though I personally haven't, living mostly in the world of enterprisey applications, except for the past three years when the large scale ecommerce platform I was a part of was Flash, JavaScript and Spring MVC. Now I'm in the telecomm world and it's Flash/Flex and Spring MVC. I try my best to stay out of the front end's hair for the most part (I'm not very good at it and prefer integration and backend server side challenges. Which is where Scala shines. So why then Lift? First, it's well put together. I'm always a sucker for well architected frameworks, whatever layer they serve. Second, it provides a means by which a great deal of the language can be exercised in interesting ways.

So, gotta get back to my light lifting. And hiding from that evil sun.

thinking about thinking

Despite my best intentions, I've neglected this blog.  I've been spending whatever spare time I've had on my other non-techie blogs and frankly haven't been taken with anything on the technology front that's inspired me much lately.  

One thing I've noticed that certainly isn't inspiring has nonetheless been on my mind lately: thinking, or rather the lack of it.  It  has been an unfortunate constant to greater or lesser degree over the quarter century I've been making a living writing software: we too often as both developers and managers forget to include in time estimations what might well be the most important thing: time to think.

Estimates might include the time it takes to "design" (usually, the time to type up the design or sketch it out), the time to type in the code and tests, time to debug, refactor, adjust, and time for documentation (user doc, examples, etc.).  Each of these activities as described are necessary things but are in the end essentially documenting the results of thinking.  The code documents your thinking so that the machine understands and can execute it, the debugger helps you figure out where your thinking has gone wrong (so do, obviously, your tests).  It seems to me, though, that the actual thinking itself is never raised as a formal activity.  Ideally, it shouldn't need to be (it should be assumed as the primary driver for everything else).  But in my experience that isn't always the case. All the rote things you have to do take time as well, and often if something doesn't fit into that vein, it's not considered.  Of course, you still have to do the thinking, but because no time has been allocated for it, it often gets rushed and relegated to a supporting part where it really ought to be the star.  Maybe it's just me, but I have a feeling in this case, it's not.

Of course, this malady has been noted over the years by a number of folks.  Check out Andy Hunt's Pragmatic Thinking and Learning for a great read that touches on this (among many other things).

Scala Revisited

I've gone over my love-hate relationship with Scala on these pages before but it's skewed toward love now that I've seen it in action for more than prototypes and poking around (it's used increasingly on the server-side with the products developed at my current client).

I want to be clear that I'll always have reservations about any language that tries to straddle the divide and be both object oriented and functional: it makes the language too big and too complex.  Scala's approach toward operator overloading (really, just allowing operator characters such as + and / and == as valid method names) has me flashing back to the nightmare of C++ debugging while having so many ways to shortcut different expression idioms bothers me as well; it allows you to write Perl-like self obfuscating code.  It's a positive trade-off for people with an aversion to typing but I'm sick: I enjoy writing and don't mind verbose expressions if they make things clearer, though I'm happy to be done with extraneous type declarations, parens and semi-colons.

The thing I'm digging with Scala is the relative simplicity in reasoning about concurrent operations and the power of being able to pass around little chunks of functionality.  Now I know these aren't necessarily limited to functional languages: Ruby provides blocks/lambdas/closures and with Java 7's fork/join framework, the concurrency library of the JDK itself is helping to drain some of the unnecessary complexity out of concurrent programming with Java.  But the elegant and immutable nature of functional programming brings out the beauty of software development like I haven't experienced in a while. And all this from just perusing the stuff.

I guess I'm lucky with my first taste of Scala in a real project.  It seems most of it is written pretty well.  That means not as a "better Java" ignoring the functional and Scala conventions and idioms.  It also means not using all the language bells and whistles - it's damn complex enough on its own.  You can write shitty code in any language and Scala will give you more than your share of rope in that regard - we haven't yet.  Then again, I haven't taken my first crack at it yet here.  :-)

Speaking of things functional, there's some good info in this InfoQ talk with Stuart Holloway on Clojure,  a more purely functional Lisp-like language for the JVM (and the CLR).   Makes me want to learn Clojure, but my brain's already full.  Besides, I'm more English Lit than I am Comp Sci and I need to get back to my roots again (I haven't picked up a non-techie book in a few weeks and think I'm going through some sort of withdrawal).  Then there are those two non-techie blogs I've been neglecting of late.  Thank god for the rain today so I can catch up ...

Enterprise Architecture is not Platform Architecture

The whole NOSQL thing has been a hot topic over the past year or so in the tech blogosphere and I don't aim to add to the discussion (suffice to say that both relational and non-relational data stores have their place and it all depends on your requirements; this is not my area of religion).  No, the reason I bring it up has more to do with the larger concerns behind the rising tide of NOSQL and other like uprisings against conventional enterprise architectural thinking.

It seems that folks are starting to understand that those things very important to enterprise applications are often different than those most critical to an internet-scale application (or other what some like to call "platform" applications).  The people designing and developing these apps tend to come from the same community of technologists, with their own preconceptions of the architectural characteristics an application or system should have.  Since enterprise applications came first, you get the things important to them applied to platform systems, even when it might be clear to a dispassionate observer that different concerns should be at play. So you get ACID or even 2PC transactions at every corner, relational databases, lots of small custom applications, failures treated as fatal.  You might have some of those things in a platform architecture, but often they are not nearly as important as the twin towers of horizontal scalability and the expectation - and thus resilience in the face - of failure.   The middleware vendors orient their best practices around enterprise applications, since that's their bread and butter from a financial perspective (but that just adds to the confusion).

It's good to see that these preconceptions are being pushed aside as platform applications are becoming more prevalent.

Enterprise architecture can learn from platform architecture too.  I just hope folks don't go whole hog in the other direction ...

things i'm paying attention to

Yeah, Yeah, I know: a list of things I'm "paying attention to" is just another way of saying that I have no ideas or opinions of my own at the moment.   Sort of like the tour guide at the Duff Brewery when he says, "We've got a lot of exciting new ideas in the works."  Homer presses him on it, "Like what?"   The guy is dumbfounded by the probing question and stands mute while we see a single pipe in the background originating from one big vat of beer and then splitting off into three chutes labeled Duff Dry, Duff Dark and Duff Regular.  Sort of like that.

Actually, I have a lot of opinions and even some hair brained ideas.  Most of my technology-related opinions today are centered on the increasingly important constructs enabling software to run efficiently and correctly under truly (and highly) concurrent conditions.  Lots of adjectives there.  This is increasingly important for (at least) two reasons: 1.) More and more machines have more processors and more cores per processor and 2.) More organizations have the need for larger scale systems, generating more data, requiring significantly more parallel (read: concurrent) processing in order to match its scale.  Often this is distributed concurrency (which has its own challenges) but within that larger framework exist increased needs for in-process concurrency.    Ya can't really scale up and take advantage of additional processors on a box if the software itself was not designed to execute concurrently.  That means doing it efficiently and correctly or suffering the consequences.

Java 5 provided a whole host of high-order concurrency capabilities to the JDK with java.util.concurrent and Java 7 is adding the fork/join construct (the master of java concurrency, Brian Goetz, explains the latter here).  Still, even when we get to the point where Java 7 goes first GA and then mainstream (a few years off), all of this good stuff is still fundamentally built on the concept of synchronizers guarding shared, mutable data, and that's pretty darn low level for your average developer to be able to reason about consistently.  One slip-up and you've got a ticking time bomb.  A time-bomb that often doesn't go off until you're in production running on machines with more and faster processors with additional cores than ya had in development or QA.

Concurrency is something where Scala shines. I've written elsewhere on this blog about my love-hate relationship with Scala, but for highly concurrent capabilities running on a JVM, it's only love-love I have for this half-breed OO/functional concoction.  After all, Scala is short for scalable.  The reasons for this are two-fold: 1.) Scala's Erlang actor framework provides a high level mailbox messaging construct that is easy to reason about when it comes to formulating software that is correct in the face of concurrency and 2.) Scala's functional programming heritage means it's easier and more efficient to make your data immutable than it is with Java, and we all know immutable data is one of concurrency's best friends.

I said I was just going to list the things I'm paying attention to today, and so now I will:

• Elastic Search
• Scala 2.8
• Heroku
• Akka Project
• How to Measure Scale (an "oldie" but a goodie)
• Comparison of grid frameworks
• InfoQ Comparison of Spring MVC and JAX-RS
• InfoQ: Automating Operations with Spring Batch and Spring Integration
• InfoQ: The Future of Flash
• JClouds
• IBM's Java Concurrency Practice Series by Brian Goetz, et. al.

Agile by any other name might just be Salvageable

I agree totally with Brad Murphy's fear of the coronation of Agile as "mainstream" by industry analysts and CIO magazine.   I've been working primarily as a consultant over the past decade or so and have been to a lot of organizations.  Sadly, all too often, "Agile" is rolled out as though it were a software package (just like SOA and now Cloud Computing).  They then declare themselves iterative and agile because they've instituted a couple of practices that are associated with "agile" (usually, things like continuous integration and refactoring and far less often things like test driven development and pair programming).

Sometimes an organization actually really does try and do emergent design and truly iterative development but again, far too often, where they've gone this extra mile, they've declared the results a failure and stepped back from the ledge.  Of course, it fails usually because senior management does not also commit to these things and enforce their practices across all the stakeholders involved.  It was always, "Well, QA will still test things at the end."  Or, "we'll still run the project the way we have."  Or, "We'll still do up-front requirements gathering." Or, "production operations folks will still only accept the complete and 'final' version of the software." And almost always, "We already have a deadline and the features committed for that date."  This last is rarely flexible, even when in almost every case, things are delivered late and without all the features anyway.  Still, can't let senior management in on the fact that they've got their heads firmly buried in the sand.

Not all organizations are like this of course.  And not all parts of even large organizations have fallen in this regard.  But a lot have.  And claiming agile is now truly mainstream is simply validating an organization's false understanding of what agile principles truly are.

Perhaps, like SOA, we simply need now to find another name for Agile in order to save it from itself.

Service Oriented Ambiguity Redux

I've been neglecting this blog in favor of my non-tech screeds for far too long now.  When I have posted, I've done so only to point out some particularly interesting (to me) ideas and opinions from others.  It's high time I put something out there that I'm particularly passionate about, so here goes ...

One of the things that has irked me over the past five or so years are the many myths and misconceptions around Service Oriented Architecture (SOA) (or, as Martin Fowler rightly labeled it, Service Oriented Ambiguity).  I'm pretty much in agreement with Fowler (as I usually am) that SOA has long become a hopelessly sprintered term that means everything from "SOAP" to "Business/Technology Organizational Transformation" and should be abandoned to the dustbin of techno history (not necessarily the things people mean when they say "SOA" but simply "SOA" itself).  Still, though the rampant hype has died down and replaced as the buzz word de jour by the equally ambiguous "Cloud Computing", SOA has stuck around.  So I'd like to at least state what it means to me, strictly from an technology architecture perspective.

Generally, I tend to follow Tom Erl's literature when it comes to service orientation, particularly his principles and the manifesto he spearheaded.  It quite literally means that your architecture is built and operates through the consumption and production of services.  Yeah, yeah - services don't necessarily mean web services.  Except that 99.9% of the time they do.  They certainly don't mean just any web services: RPC-based SOAP "services" are too often simply procedure calls wrapped up in http and XML dressing.  RESTful Services tend to be the most pure when they are really RESTful rather than just labeled that way (they all too often degenerate into "returning a bunch of shit wrapped in JSON/XML in response to a random http request").  Of course, the best RESTful services often follow a Resource Oriented Architecture (ROA), which is somewhat different.  REST != SOA.  REST is an architectural style, not an architecture.  Read RESTful Web Services for more details on ROA.  And check out the Web Oriented Architecture (Un)-Manifesto, which is not just about REST, but WOA is to my mind much more interesting than "SOA" ever was.

To get back to the point I'm trying to make, service oriented architecture means that the architecture of the system/platform/application itself is oriented around and talks services internally.  It does not necessarily mean that the system/platform/application exposes services for use by external entities.  You can have an architecture that exposes all kinds of wonderful web services for use by external clients and it might still be an ages-old monolith or a modern and perfectly reasonable component-based model. It's not necessarily service oriented.  I'd call it service exposed.  Likewise, you may expose no interfaces outside of the system itself, or you might publish an RPC API, but the internal architecture itself could still well be service oriented.  The interfaces exposed to the client should be orthogonal to the architecture of the underlying system implementing them.  Now, practically speaking, the two are likely to influence one another strongly, but the point is that they do not have to.

Software as a Service (SaaS) - and its offspring, Platform and Infrastructure as Services - also get intermixed into this equation. SaaS as a term predates SOA by many years but is more popular than ever now, while its much-hyped children, PaaS and IaaS, are even more the rage with Cloud Computing at the forefront.  But each of these qualities of architecture have their own inherent characteristics, many not necessarily the most paramount or even applicable to service orientation (multi-tenancy, availability, scalability).

Too often I see all these architectural characteristics lumped together or used interchangeably and thought that at least for my own clarity, I'd get my 2 cents on the matter down "on record."

What I want to be when I grow up? Well ...

I've made a recent move from full-time information technology architect for an e-commerce solutions provider back to the role I played for the seven years prior to joining that company in 2007: technology consultant.  Probably I'll be asked by clients to perform duties that on the surface seem pretty similar to those I was doing for my full-time employer; however, appearance can be deceiving.  A lot of what you end up doing in a full-time gig, especially if you're in a senior role, only marginally relates to your "primary" responsibilities.  These "secondary" duties often consume a large chunk of the day (project management, procurement of hardware, software, etc.).  A consultant, on the other hand, is often hired to perform a very specific role for a very specific time period.  Sometimes organizations bring on consultants for an unspecified period of time to essentially fill a full-time position, but I'll be specifically shying away from those engagements this go-round.

I'm not suggesting that one role is better than the other.  Both clearly have their pros and cons.  But I'm ready to focus more on the task(s) I feel I'm good at and that I like doing and consulting gives me what I feel is the best opportunity to do that.  Or to go broke trying. :-)  Of course, it'll also land me in places I'd rather not be (but I'll ensure that this only happens for short periods of time by keeping the contract lengths short).  With consulting, I have to keep on my toes over a broad spectrum of technologies so that I can be 'the expert' on short notice.  I like that.  I also have to deal with the uncertainty of (un)employment from time to time and the possibility of significant travel, which I don't like so much.  Still, on balance, the siren song of consulting finally won out, at least until I can find out what I really want to do when I grow up. :-)

If I have my druthers, I'll be focusing on large scale web capacity, scalability and concurrency challenges and resulting solutions for clients.  That's where my passions are these days and will be how I orient my marketing message to potential clients.  But whatever the gig, I'll do my best to make the most of each engagement, always on the look-out for that next special domain and/or organization that I might want to stick around with on a more permanent basis.