We all endeavor to improve how we work. The twin goals of increasing both the efficiency and effectiveness of our testing are always on the mind of any professional software tester. But does improved testing always lead to business benefit?
What is Quality Software?
To see if our improvement efforts are yielding positive results, we should at least know how to define quality software. Or less philosophically, what makes our software quality.
This is a challenging question even, or especially, for experienced software test professionals. I have mentored a number of intern-level software testers and I always try to ask them the question, “How do you know if software is quality software?”
In my experience, they always come up with intelligent answers around the twin themes “it does what it is supposed to” and “it doesn’t have any bugs that customers find.” Good answers. I challenge them to expand their thinking to always consider the people that the software affects.
At a lean startup conference I attended recently, Eric Ries said, “If you don’t know who the customer is, you don’t know what quality means.” So true.
But while the customer is a critical group of people affected by the software, a good tester needs to consider the developers and testers who will be required to work on future releases; the salesperson who has big opportunities riding on the delivery of certain features; and the CEO who needs to meet this quarter’s targets while building a stable and successful software product suite. All of these people, and many more, have different needs and expectations regarding what makes the software quality.
I want to encourage all of us to remember, even when we are deep in the details, that software quality is always defined by the people who are affected by the software both now and in the future.
Better testing, lower quality
If we keep a clear vision of what quality means for our software products, we can improve our testing efforts in a way that increases quality. Without the proper perspective, test improvement in the wrong area could actually distract the entire development team from what truly matters.
For instance, will logging 17 integer overflow errors improve the quality of the software? The temptation is to automatically say yes. But, depending on the people the software affects, the effort behind finding, logging, processing, resolving and retesting those 17 issues could be wasted effort that leads to an overall quality reduction because of the limited team resources that were consumed with no real quality benefit.
Better testing, higher costs
Similarly, if you can find clear ways to reduce defects in the software system that lead to 15% higher testing costs per release, is that an improvement?
Again, the answer to the question depends on the more fundamental question of what quality means.
Great testing, bad business
I hope you will remember to recognize the danger that great testing divorced from the true context may lead to bad business results.
You cannot know which tools in your testing improvement tool belt will lead you to truly positive results. Generally, in my opinion, if you are using only test metrics and general testing advice to determine what to improve, you will be attracted to the wrong improvements and end up getting sub-optimal or potentially negative business results.
As a community, let’s learn – and keep learning – what quality means for our unique software and make certain our testing improvements lead to positive results for all.
About the Author
Matt Kallman A graduate of the Seidman College of Business at Grand Valley State University in West Michigan, Matt brings a variety of software project and product experiences to provide his current clients with success in their software projects.
He is currently the Project Manager and QA Director for Carnevale Interactive, a design focused software development agency. He is based in their Grand Rapids, Michigan office. His work in software testing, test automation, process improvement and project management have benefited software projects ranging from web applications to compliance software for financial institutions.