Does anybody else remember those heady days of the 1990's, back before "Agile" was fashionable, when a test cycle would take months?
Why, back then, we would 'do it right', re-testing and re-testing until all the features passed.
Except when we didn't.
Now by generalizing I'm sure to get things wrong -- by very definition generalities will have exceptions.
Still, more often than not, on those 'big' projects, we would still have fixes a day or two before go-live.
And we'd be
tired. And we wouldn't have time for a full re-test. And we were pretty sure that the changes were localized to one or two specific functions. So instead of a full re-tests, we'd just verify the fixes. I mean sure, these changes would have introduced errors, but we decided the risk was small.
I call this problem -- where you are just re-testing fixes, not the entire application,
tester fatigue. Of course, in many cases, just verifying fixes is called for because the risk is small, or the consequences for failure is small.
When people say "You need to fully test the system", I see bad thinking. I mean, what does even mean?
Complete testing is impossible, and
repeated tests often have limited value.
Instead, I suggest that for every build we'll want to take a hard look at the risks we are exposed to, the change itself, the consequences of failure, the cost of the fix in the event of failure, and so on.
Again, I'm not saying that 'just' verifying fixes is always bad -- instead I am referring to a mindset where the tester just wants to be done, and stops looking for problems, choosing to only focus on the known issues, often taking an overly-charitable, overly-cavalier approach to which issues to log or escalate. I'll call that
tester fatigue.
Fast Forward a decade or so
All of a sudden, instead of a months-long test cycle, we have to go from requirements to production
ready code in a month. Or two weeks. Or one week.
This creates an amazing about of pressure on the tester.
So, for example, say Sally Scrum-Tester gets a story on Thursday and the iteration is supposed to close the following Friday. Plus the team needs a week for regression testing. Basically that gives Sally a couple days to test the story.
Sally finds
problems. In a story with perhaps one-hundred customer-collaboration-defined story tests, Sally might find twenty-five that fail. In addition, she might find fifteen other problems not on the list, just due to her interaction; the kind of problems that can not be easily predicted.
She describes the problems in some system, talks to the developer,, waits a day, and gets a new build.
Now instead of forty failures there are fifteen, then ten, then five. You see how this goes. Sally is supposed to be regression-testing but the story is late.
Instead of re-running all the tests, she's just verifying fixes, mumbling under her breath "work ... work dang you ..."
Sally wants the software to work more than the developer does.
Is this good testing?
If the developer is the type who hands over code with a large number of bugs, probably not -- because he is apt to break things in build #4 that worked in build #2. By chasing down just the failures, Sally has blinders on to other problems.
What is the fix for tester fatigue?
When you find yourself in this situation, things aren't good. You need to get out.
The most obvious way I see to get out is to get a fresh set of eyes on the work. In other words, trade stories (or features) with another tester. They'll have a fresh set of eyes and you'll have a new, interesting problem to have.
Another option is to raise the issue at a team standup. Tester fatigue is one of a host of problems that can bring productivity down and lead to that "big bad failure in production." Nobody wants that. So raise the issue at a team meeting and see if the group can shift assignments around a bit. If "Sally can't do it, nobody can" perhaps the company can lighten your loan on other features. Alternatively, perhaps you can take a few hours and think about something else or work on a different project, just to cleanse your mind.
The important thing is recognizing that "something is up" and acting on it.
I'm not sure, but I suspect there are whole host of these risk-increasing, morale-and-productivity-sucking patterns out there. Another one that comes to mind is "Work Avoidance", a deeply human trait that can't be ignored or explained away with a process diagram.
I'll talk about that next time. You see, if you'll excuse me, I gotta go test ...