I turns out that my Collegue, Jim Brosseau has been going through some of the same issues that I have. Jim and I don't totally agree on every issue (who does?), but he does say some things in a recent newsletter that I think are rather profound, and I'm going to provide a couple of exerpts.
Here's the first, which talks about his experience at Agile Vancouver:
I have no problems with an idea that fills a room and gets people talking about techniques that help them be more effective. What I do have problems with is the over inflation of the value, or even the knowing neglect to correct some flawed assumptions about an idea, all in the name of further padding of the wallet.
At one point in the final panel session, there was the suggestion that teams starting down an Agile path should enlist the support of an expert for training. There were more than 2 of the experts that I counted at the front of the room that clearly recognized this as a "cha-ching" moment. This was expressed either politely as a satisfied smile and nod, all the way to the blatant Tiger Woods pumping of the arm for the hole in one. Who is the primary beneficiary of this Agile movement, anyways?
Consumer Tip: If any company or organization suggests a specific methodology, tool or framework as a reasonable solution for you before you can safely say that they truly understand and empathize with you, your culture, your challenges and your products, do your best to make the door hit them on the ass on the way out!
Even if that tool or methodology is what you called them about in the first place.
What is Agile, if not a fully fledged product? Perhaps an enabling technology, perhaps a series of patterns, perhaps a philosophy. Good stuff, but currently over-hyped.
What I saw at the conference was well beyond a philosophy, it was a religion, bordering on a cult. There was a common enemy that everyone could rally against, with an 'If you are not with us, you are against us' fervor. All things non-Agile were seen as bad, and generally lumped into the category of Waterfall or hacking. Plenty of ad hominem discussions about evil managers.
There were a large collection of ideas that were embraced by and expressed as Agile, even though the vast majority have been around and practiced in effective 'waterfall' projects for decades. Retrospectives are Agile. Estimation based on size is Agile. Estimation Poker is Agile (hello...Wide Band Delphi, anyone?). Early stage focus on test is Agile (and has been a well known approach for early stage validating scope for years).
Perhaps the current push extends some of these ideas a bit further, but there is nothing that I would call disruptive technology. Hell, we wrote what looks a lot like XUnit over a dozen years ago on a huge ATC project. Decidedly pre-dating Agile, but we did some smart things.
What is happening is that a group of people are re-discovering things that work, assuming that they can be generally applicable, and evangelizing a bit too aggressively. I wrote an article for the Cutter IT Journal in June of 2004 titled Beyond the Hype of a New Approach, a cautionary tale expressing many of the concerns I had then and still have, at the time in response to Jim Highsmith's Agile Project Management book and the corresponding movement. Then and now, a lot of good ideas are being thrust upon us in a manner that will cause the mainstream to cringe rather than embrace. We need to understand the market before we can sell the product.
The argument is not that the ideas in Agile are bad, but that we technologists thrive in a Boolean world while reality is analog. The best answer before we have all the data really is "it depends." Euthanize the dogma, please. Don't get me going on the suggested prospect for a Unified Agile approach. It's not going to happen.
There are a couple of things I keep noticing in the software world, a big one of which is a desire for simple, straightfoward, "easy" approaches that allow practitioners to turn their brains off and follow the process.
I have a real problem with that, because software development is knowledge work, and "easy" approaches make the process responsible for the outcome, not the people. That might work fine at McDonalds, where everything is consistent (just not very good), but do you really want software like that?
Sadly, where there is a market, businesses will fill it. So not only do you get Agile Dogma (mouthing agile buzzwords and assuming they will always work without thinking), but you get Agile pushers, who are more interested in the advancement in the prominence of "Agile" than in any lasting contribution to the field.
My response to the "Give me a cookie-cutter soltuion" question is, well, long. I like to try to start a dialogue. After all, it takes a very long time to say "Life is pain highness. Anyone who will tell you otherwise is selling something" without losing a client.
In the end, the reality is that software development is hard, and the cream will rise to the top.
I wouldn't have it any other way.