Let's start with a basic assumption. In most instances when we as testing professionals are performing the activities of testing or preparing for testing we are not guessing in order to determine what we will do or how we will do it. We use our skill, judgment and expertise to make choices we feel will result in outcomes which are positive, effective and expedient. We employ specific tools and techniques based on our experiences because we think that the current challenge we are addressing can be solved by using an approach similar to one we have applied to similar situations we have encountered in the past. And, we therefore feel reasonably sure that the outcome will be similar in the current instance. We do this because we know that while past experience is no predictor of future success, it is an indicator. We can, with limitations rely on past experience to provide us minimally with a starting point from which to begin our activities. And we assign levels of confidence to those assumptions based on the perceived similarity between the experience being referenced and the one to which that experience is being applied.
So, in essence we are analyzing the current challenge for clues that we can use to mine our past experience libraries with so that we can choose what we perceive as a similar situation; We examine that past experience and determine what if any modifications are required, make adjustments and then we apply the solution now defined to the current situation. Evaluation tells us how close we got. When it doesn't work we ask why and examine with hindsight the problem space and how we defined it. If we're aware we realize that then reason for the less than optimal outcome is usually a failure of analysis. Reaching a particular conclusion with insufficient information or underestimating the importance of a particular fact or overestimating the similarities between experiences and simply choosing incorrectly. Simply put we made a mistake in analysis and applied the wrong tools and techniques to the problem at hand....
Sound familiar? It is the problem of the "silver bullet" approach to not only doing testing but really to doing anything. The idea that one process (to the exclusion of others) will address all problem spaces. This sort of approach ignores context, differences in degree vs differences in kind, and the need to monitor just to name a few things. Does that mean that there are no best practices? Of course not! The best practices are the tools, techniques and skills we apply coupled with analysis which helps us better define the problem space accurately the first time. We become better at analyzing with consistent, repeated application of all the tools at our disposal. In other words, experience and reflection make us more accurate.
So what does all this really mean? Our best practices are the sum of our shared experiences filtered through the prism of community feedback and wrapped in the collective wisdom of real practitioners correctly applied. Not some rolled up paradigm that allegedly will solve all of our problems. I propose a re-visioning of this construct - Let's stop talking about best practices because that seems too charged with political implications and begin the process of defining our collective practices and when and how to apply them. Let's define Right Practices! This is the model of a professional group applying professional skill and judgment to solve the problems we encounter in our daily professional practices and I think this makes a lot of sense..... I'd love to hear the thoughts of everyone else on this. Let's talk about this....
- Judy
Follow-up: Thanks Steven. Although I have not read those writings I am not surprised that others more prominent in the industry have expressed the same views. I would point out however that although people seem to raise this perspective from time to time, we continue to suffer from the problem of telling ourselves that we don't know what best practices are, let alone what our best practices are or should be or even how to define them. As such we still struggle essentially with issues of credibility and the opinion of some that anyone can do testing and do it correctly. Additionally, we are at risk for having non-testing professionals trying to glop their standards onto our profession resulting in certain groups promoting practices as best testing practices which are utter nonsense at best and irresponsible wastes of time and money at worst. We need to take and own this issue and get serious about definitions and practice and stop letting others how haven't got a clue do it for us....