My current team has been integrating exploratory testing techniques for a while now along with the traditional testing methods of manual testing as well as automation. Most of the testing communities that I have known seem to brush this testing technique off simply as “monkey testing” which doesn’t really provide any significant yield on bugs nor achieve high quality bugs. However, I have been observing how powerful this testing method can be from my own perspective as well as by seeing the type of bugs that some of my team mates find.
Many testers still underestimate the power of this technique and simply disregard it as another “ad-hoc” testing. They think that this is simply ad-hoc testing or monkey testing where the tester just randomly bangs and shakes the product until bugs are found. However, this technique is actually proven to find critical bugs, cool bugs, and most importantly the kind of bugs that customers would encounter as they use our products. Remember that customers are the ones who will “explore” your product features. So we should try to best mimic customers as best as we can.
The downside of exploratory testing is that it needs to be done in a strategic approach. A tester should plan on what testing techniques to use. This helps keep the testers in focus and helps them to apply the right techniques to get the most out of an exploratory testing session time.
The main techniques with exploratory testing are:
- Freestyle:
This is basically ad-hoc testing. Tester will test the product from black box perspective, which best simulates the product in the real world environment. Pros: Low cost. Anyone can do it. Encourage creativity. Cons: May not yield the critical bugs. Low quality bugs are typically found with lots of duplications. It is quite difficult to determine coverage. - Scenario Based:
Using a pre-defined end-to-end scenario, complete with actual reproducible steps and validation methods prior to testing. Testers can use these scenarios as the base, and figure out new end-to-end path that will achieve the same goals. Pros: Easy to trace the reproducible steps and to follow. Can determine test coverage. Cons: Pre-defined scenarios may actually not represent what the customer really wants. - Strategy Based:
A tester should have great understanding of the product, thoroughly understand the architecture and flow of the product features. They can then leverage this knowledge to explore deep into the nook and crannies of all the features of the product. Pros: Test techniques are proven to be effective. Testers just need to master them and apply them in the right situation. Testers are also pretty much free to use their own instinct and creativity to go about testing the product. Focus on learning. Cons: Newer testers on the team may not have the sufficient experience or knowledge of the product to be completely effective. It requires higher skills, and it can be time consuming. - Feedback Based:
This testing technique starts out much same as Freestyle testing. But as testing sessions are conducted, testers build up a history of test executions, areas covered, bugs identified, and code churns, etc. It is very similar to our session notes. Testing from historical data such as bug records, data from automations and scripts, application logs, customer reported issues, etc. Use these data as input so a tester can identify which areas may yield the most bugs. Pros: Knowledge is not lost. Make use of previous knowledge is very powerful. Cons: Sometimes it’s very difficult to make use of existing data effectively (especially if there are a lot). Testers may actually spend more time gathering and processing these data rather than actual testing.
In order to fully make exploratory work, knowledge sharing is key. It’s important to make use of historical data such as the type of bugs found in previous versions of the product; which features are buggy; customer-reported bugs; even knowing your own developers and their way of coding can also help uncover holes and bugs (humans tend to repeat the same mistakes).
Using these testing techniques combined with the exploring minds, the quality of the product is sure to increase.
About the Author
Chan Chaiyochlarb An experienced Software Engineer with broad experience in testing fundamentals, software development processes, web services, online advertisement business, and developing test automations. Particularly interested in opportunities which allow me to improve software quality, increase efficiency, and reduce time to market.