Five years ago, Cem Kaner held up his cell phone.
Dr. Kaner pointed out that his cell phone was the sort of consumer device that testers were being asked to evaluate, and it had millions of lines of code. Meanwhile, the test techniques that are popular are still the techniques of the 1970's and 1980's - back when a big program was 5,000 lines of COBOL.
He pointed out that programmers had stepped up their game, with new languages, models of programming, and interfaces -- while, for the most part, testers had not.
This conversation became a part of a series of articles and presentations Cem gave, around the theme "The Ongoing Revolution in Software Testing
." The bottom line was that Cem thought it was time for a new generation of test architects to step up to the plate with new ideas.
Five years ago, Cem Kaner held up his cell phone
Most recently, I've seen a series of speeches, presentations, and talks that try to address this issue -- to move testing forward. We've had testing in production
, staged deployments
, A/B split frameworks
, and, most lately, the "Test is dead" meme. I've even done some work on that area myself, at my recent STPCon speech on how to reduce test cost on Monday.
To get a good feeling for Test is Dead, you can check out James Whittaker's talk "All this testing is getting in the way of quality" from STARWEST
, or Alberto Savoia's talk
from Google's Test Automation Conference, titled "Test is dead."
I think those talks are worth listening to, if at least to understand the arguments they are making. I'm not endorsing the ideas, but I do think they are worth an investment of your time, and, at 'free', well, the price is right.
One thing I'll give to those guys, they know how to entertain.
There's a fair bit to digest here. I'll try to take it one piece at a time.
Testing As a Career Field
When I look at Whitaker's talk, I tend to agree on observations, if not conclusions. Whittaker walks us through the salaries, respect, and promotion opportunities available to testers, and claims they are, in general, much lower than developers.
Uh, okay. I tend to agree.
Do we really think they should be, in general, equal to developers?
I mean, really? Programmers, especially at companies like Google, IBM, and Microsoft, need to have more specific training and more specific skills. They generally require a college degree, and specific years of experience coding in specific frameworks and languages.
More than that, the developers are building the thing. Without us, the product would be buggy, but without the programmers, the product wouldn't exist.
Should we really have a problem that programmers are paid a bit more and get the fancy chairs?
Honestly, I don't. Nor am I surprised that promotion opportunities are limited in test -- developers are more common, so there will tend to be more management slots. In a company founded by programmers, that built a test discipline later, we will tend to see programmers at the top. Shocking, I know.
The question I ask myself every morning is not "How can I make the most possible money?", but, instead, "Can I deliver a service I believe in, with integrity, and be compensated enough to be satisfied?"
That is a very different question. If I wanted the former, and I wanted it with very little risk, I should probably drop both programming and testing and pursue executive management, right?
Notice here I am, testing away.
Testing As Advancing Quality
The next argument that Whittaker makes is that the quality advances in the past decade or so have not been driven by testing. Continuous Integration, Testing in Production, cheap/fast deployments, cheap/fast builds, TDD, crowdsourced testing, none of these things were done by us, the software testers.
In other words, we are back to Dr. Kaner's cell phone.
He goes on to suggest some uncomfortable truths, such as, for example, this idea that test artifacts are worthless, what matters is the product, so stop writing test plans and bug reports.
If you listen very carefully to my speech at STPCon (part I
and part II
), it turns out I say a very similar thing - that we need to get issues to developers quickly, and minimize documentation. On this, I think we have substantive agreement.
So Test Is Dead
The general conclusion is that test as we know it is "going away."
This, however is the final road, and one that I think does not really apply to me.
To put it a different way: If your company makes it's money off of giving away free software and selling advertisements, then, perhaps, if your customers are forgiving, and if you can respond to defects quickly, you can be successful by hiring automated-test-writers (selenium, etc) and crowdsourcing your testing with a product like uTest.
The idea here is to have humans find bugs fast - and cheap - perhaps over the weekend - while the automated tools prevent major regressions.
And hey, that might just work for google, or for Yahoo, or Mozilla.
Would you want your bank, your credit card company, your automobile transmission, or the medical device company that created your pacemaker to use such a strategy?
To the rest of us strongly in the middle, this new paradigm of testing brings up some interesting questions.
Like the newspaper, radio, movies, television, cable, and now online viewing with NetFlix, these new disruptions threaten to replace the old.
Notice, however, that the old things didn't go away.
Sure, some of them did, but the ones that survived found ways to innovate and contribute. Print media, for example, is really struggling right now. The newspapers that are doing the best are local newspapers - small rural newspapers, for example, that cover news the big boys simple do not have. In that way, the print newspapers are still unique.
Can testers find unique ways to contribute?
I think so.
More to come.
But forget about me -- what do you think?