Over the past year as the Director of Membership & Publications, I have had the opportunity to talk with and work with
some great individuals. As authors have submitted content for the STP magazine, newsletters, Local Interest Chapter and
conference presentations, my understanding of testing and quality has continued to expand as a result of all of this great
material. Jon Quigley and Kim Pries, frequent contributors and authors of our ST&QA magazine Six Sigma series, have
really impressed me with their deliberate approach to testing and their ability to communicate very complex concepts.
IIn conversations with Jon and Kim, I have discovered they are about to release a new book, Testing of Complex and Embedded
Systems. Knowing the quality of the articles they have shared with me over the past year,
Implementing Six Sigma: Improvement in Software Development,
Six Sigma, Time to Measure,
Six Sigma, Analyze This,
Six Sigma, Control in Software Development, and
Tracking Quality Variations with Six Sigma Methods,
I asked them to give us a sneak peek into their newest endeavor.
The book is scheduled to be released in December. If you would like to
pre-order you can do so at Amazon. Look for an upcoming
article from Jon and Kim in the ST&QA October edition, The Embedded Software Development Process. If you have any questions
for Jon and Kim send them forward to me at rhand@softwaretestpro.com.
About the Book
The goal of Testing of Complex and Embedded Systems is less to elucidate specific test activities and more to persuade the
discerning reader to move toward the "true path" of the test engineer. The book covers the following major topics:
- The benefits of improved testing
- What we call the "Basic Principle" of testing
- The true question
- Contradictory perspectives that can mislead test engineers and test case designers
- The use and abuse of noise
- How to perform "bad" tests
- Documenting test cases and other test documentation
- Test administration
- Advanced concepts
- Configuration management
- Software testing
- Systems testing
- Simulation and emulation
- Exit criteria
Many pages discuss the many contradictions and biases that occur during a test program, but the strongest concern lies
with the idea that we test to a standard in order to pass—that just simply is not the case. We severely test a product
in order to find out where it fails, not whether it passes some arbitrary set of requirements.
The book also describes the need for configuration management of the test program, both for hardware and software. In
all cases, we need part numbers, version numbers, release numbers, and descriptions of hardware and software components.
The section on performing "bad" tests is a bit tongue-in-cheek, but it also points out obvious flaws in test programs
that are often overlooked in the rush to bring products to the marketplace. Experience suggests poor planning contributes
significantly to test inefficacy. Product launch dates seldom seem to be impacted by all of the late up-front work and
verification and test activities are subject to compression or in some cases elimination, thereby putting the product
release at risk or, worse, releasing a defective product. Test planning and testing itself must start well in advance
of the last deliveries of the product from the development group.
Below is a scene from the Disney's Pixar's movie Ratatouille. The combative restaurant critic Anton Ego comes to
Gusteau's for dinner
1:
| |
Mustafa: [taking Ego's order] Do you know what you'd like this evening, sir?
Anton Ego: Yes, I think I do. After reading a lot of overheated puffery
about your new cook, you know what I'm craving? A little perspective. That's it. I'd like some
fresh, clear, well seasoned perspective. Can you suggest a good wine to go with that?
Mustafa: With what, sir?
Anton Ego: Perspective. Fresh out, I take it?
Mustafa: I am, uh...
Anton Ego: Very well. Since you're all out of perspective and no one else seems to have it in this
BLOODY TOWN, I'll make you a deal. You provide the food, I'll provide the perspective, which would
go nicely with a bottle of Cheval Blanc 1947.
|
A similar relationship exists between the product development groups and the test and verification personnel – with the test personnel playing the part of Anton Ego and the development group is Mustafa. Let's, not insist the test department provide perspective on the quality of the product.
Reading the tea leaves, or the results of the testing, to determine the suitability or capability of the product is also covered in the text. When is enough, enough? Chance termination is not a good way to complete the testing. Gathering snippets of information during each loop of testing helps make some reasonable predictions of the quality of the product.
1 http://www.imdb.com/title/tt0382932/quotes accessed 9/26/2010