Most organizations wish for a quality product but paradoxically, very few give importance to testing. Instead they prefer fixing bugs as they appear in production. The general idea of other organizations is that quality of the application lies in the hands of the testers and it is the testers1 sole responsibility to deliver it “bug free”.
In both of the above cases, defects are identified further down the development pipeline. It is widely known that the later a defect is found and fixed the greater the cost. For instance, developers need to switch context in order to understand the logic before they fix the defect. This would take longer than identifying the same defect whilst the developers were working on that functionality.
This article highlights the significance of testing in a project. It also aims to show how the business can reduce cost to the project by early defect detection. This is achieved through “fast feedback”. I am focusing on a technique called “pairing2” in which people either with the same role or different roles, sit along side each other to work in a collaborative manner.
Based on my experiences on different projects I have highlighted some good practices to demonstrate how testing can influence different stages of application development in order to improve its quality. The techniques are meant to catch defects and improve quality as early in the development lifecycle as possible. I will primarily focus on Agile projects where teams are either onsite or co-located.
Testers Getting Together With Stakeholders
By participating in a project inception with stakeholders3, testers can get a better understanding of the domain and the system they are going to work on. This knowledge is vital to develop various testing strategies that could be implemented. For example, having a conversation with the stakeholders helps in identifying if the non-functional requirements (e.g. performance, security testing) need to be incorporated or how many testers would be needed in the team based on the overall team size.
One of the most important things a tester could do is ask questions. The questions are often of the “what if” format, thus exploring different possibilities of an outcome. Being in constant touch with the business on an insurance project helped me to create a test data suite. It comprised of complex combinations of the calculations that formed the core of the application. We used this suite as part of our data driven development.
A tester can highlight the risks associated with testability by giving specifics such as the issues with testing the various systems that are integrated with the application. Based on the forecasted risks and business priorities, the client is often able to make a call on the level of quality they are expecting in the system.
Once satisfied, the tester can closely collaborate with the stakeholders by demonstrating the finished features on a regular basis. The business thus gets a feeling of involvement, and sees the progress being made which raises mutual trust. The business can also use this opportunity to give feedback and make changes.
Testers Collaborating With Business Analysts
Testers can collaborate with business analysts to understand the application better at a functional level. It also helps testers to define the scope of testing. One of my clients had a large set of integration systems. I grasped the fact that we were touching only one component and all other components communicate through the existing message queues. So it made my life easy by just testing the incoming and outgoing messages of the component under development rather than testing the whole system.
Testers can help the business analysts to write the acceptance tests4 for the functionality before development. While writing, testers can give valuable inputs around boundary conditions or negative testing. This gives the developers a heads up to write automation tests. Coming up with what I am going to test beforehand, gives a heads up to the developers to look out for and write automation tests around them thus catching any defects earlier on in the lifecycle and in turn reducing the cost to the project.
Just before development begins for a feature, we usually deep dive into specifications. Sometimes gaps may arise in the functionality when testers pound on the requirements along with business analysts. While working with a telecommunications client, there was a requirement for displaying certain types of data on the front end. The obvious fact of filtering other data types, however, was not thought of until the testers pointed that out. So a filtering mechanism was developed before implementing the actual functionality. It would have proved expensive to fix this a couple of weeks down the line, or worse if all the data types would have appeared for a user in production.
Testers Pairing With The User Experience Team
I have worked on a few web-based projects where a team of people was working ahead of time on the different screens of the application before the actual functionality was being developed. This gave the business hands on experience of the look and feel of the application. In case they changed their mind, the screens could be easily modified without having the risk of modifying the code beneath.
It is useful for testers in this case to ensure that the front end of the application is intact in all browsers. The combination of browsers and operating systems gives plenty of space for broken interfaces. The business would sometimes ask us to support older versions of the browsers. Testers may have to identify whether the existing technology like JavaScript or flash runs well on them. They can work closely with the user experience team in fixing any issues before development of the functionality begins. Similarly we can check whether all the required elements and components exist on all the pages.
Some systems may have a workflow or a user journey that can also be verified by using the dummy buttons. I came across cases, where the links to the pages were either broken or re-directed to a wrong page. There was an instance when I was overwhelmed by the navigation links on the page and the breadcrumb navigation on top of the page. By giving the feedback we kept only the breadcrumb navigation thus improving the whole user experience.
Testers Pairing with Developers
Having a robust and thorough test coverage (both automated and manual) helps in improving the quality of the application by avoiding any regression defects. The team can move ahead rapidly to develop the application and also adapt to change requests.
It may not be necessary to automate all the requirements. Functionalities that are complex in nature, critical to business or repeatable, are some of the cases which can be automated. One of the challenges with automating all test cases is the rise of flaky tests, which are hard to fix. The team can have a mutual understanding of which part of the functionality needs to be automated versus being tested manually. Testers and developers can have this conversation at the beginning of developing a feature.
It is a common thought that testers need to write the acceptance tests. In order to write these tests, however, testers sometimes need to have a deeper understanding of the code, especially in non-web based projects, where mere clicking around a web page does not work. Thus pairing with the developers helps in this case along with effectively refactoring5 the test code.
Sometimes the intention and the implementation of a test differ. Looking out for such instances helps in improving the quality of the test suite. I have come across instances where the implementation of an automated test case was just an empty method. In another instance, two different automation tests were referring to the same method instead of their respective ones. Developers and testers pairing through the test code helps in identifying such cases.
Testers may also look out for false positives by feeding in the wrong input data to a test case and verifying whether the test fails. I once came across a test that passed even though it should not have. By pairing with the developer, we identified and fixed the test.
In an Agile world the team writes tests ahead of the functional code and runs them through a Continuous Integration6 (CI) build. Very often, when a build breaks because of a developer check-in, it is usually the tester who needs to dig into why the acceptance tests are not running any more. The idea is to have a shared responsibility of the tests between testers and developers by considering the test code as part of the application code.
Another area of discussion that a tester can have with the developer is to check that most of the automated test coverage exists through deeper layers of testing regarding unit, component and integration tests7 . This helps in faster CI build times ensuring faster feedback of the code quality.
In some cases it could help testers to sit with the developers and understand the implementation logic of the functionality. This may result in changing the thinking process of the testers. And they can come up with new ideas and possibilities of testing that task, leading to a higher probability of uncovering hidden defects.
For less technical testers, the developers can help them out by writing a quick script or mocking up a tool for testing purposes.
In one such case, some of the developers wrote a lightweight UI tool for me to test the message queues of a system. If not for the tool, it would have been a nightmare for me to hack the internal of the system to test the messages.
Testers Pairing with Database Analysts
In larger organizations we can find dedicated database analysts who are in “control” of their systems. Pairing with them helps to understand the database setup across different environments. This knowledge helps toensure that the configurations and settings of the test environment are more or less like the production environment.
There are instances where the database analysts write stored procedures and other subroutines that interact with the application. Testers can feel free to pair with the DB analysts to ensure that proper test coverage exists for these subroutines. Proper test coverage at this level would reduce the necessity to write a larger number of high level tests thus reducing the CI build time and eventually quicker feedback time.
The testers and DB analysts can mutually set up the test data required to run the tests. For example, the DB analysts received a fresh copy of production data copied onto the test environment. It made me feel more confident to test the application against such data. I also requested multiple combinations of data for a feature, which did not yet exist in production.
Testers Pairing With Operations
It is often surprising to see the outcome when testers get together with the build and deployment team to understand the deployment lifecycle. There are instances when you find out that there are defects in the deployment script itself. For instance the project artifact being automatically deployed is not the one intended. So you might be testing on a totally different revision of the build!
Testers can ensure that any DB changes including scripts and new table structures are being correctly deployed across environments and ways to automate the deployment. Once it became tricky when the code base, database, CSS, third-party tools and a CMS backend had to be deployed independently into a production environment.
Testers can work along with the operations team to also setup different test environments with the required settings and configurations. This would help the testers in catching any environmental defects before they get into production.
Testers Pairing With Testers
In a typical Agile project, we see developers pairing on developing functionality, but yet we see testers working in silos of each other. There were at least two large projects where I encouraged testers pairing on certain functionality. The outcome was that they started learning different testing techniques from each other by mere observation. They also started talking and sharing ideas about why they are doing what they are doing. In certain instances, they reminded each other about what else could be tested. This technique of testers pairing also helps in terms of knowledge sharing if there are multiple teams working on a project.
Thus by improving each other’s testing skills the team can eventually improve the overall quality of the application.
Testers Helping Out End Users
Sometimes the testers become a bit tunnel visioned with the application after testing it for a prolonged time. It would be very interesting for the testers to take part in the user acceptance testing to observe how the end users use the system. This would enable the testers to re-think if the application could be tested in a different way.
Quality is not just about an almost defect free application but also about improving the user experience. By observing the users, the testers can make notes on ways to improve the usability and hence customer satisfaction. For instance the development team might know that the logout link is at the bottom left hand corner of the screen. Users might be looking for a logout button on the top of the screen. So in this case we can either enhance the visibility of the link or place it at the top, or both.
Testers can also help and support the end users by providing them with any test data. Most of the user acceptance testing is done on a test environment. So testers can create or share for instance the test usernames and passwords with the users. Testers can come up with different high-level scenarios as guidance for the end users to navigate around the application. Users would thus come to know of the various new functionalities while still leaving enough space to let them explore the application by themselves.
On one of the projects I worked on, I created a feedback sheet with a legend to defect severity and category. This helped the users to categorically report any defects and enhancements that they came across. The categorical reporting helped us to quickly determine the order of priority for fixing them.
Building up such relations increases the trust level between the users and the development team.
Conclusion
I would agree that all of these testing efforts could be gradually implemented in any organization on an experimental basis rather than a big-bang approach. Many of the techniques explained would spread the testing mindset amongst the team. Members in other roles would thus become more aware of the quality of the application. Organizations other than software companies can also easily find analogies from the examples provided that could best fit them. I hope that at least a small percentage of teams will benefit from this knowledge sharing exercise.
1 – Tester can be anybody (developer, business analyst etc.) who can think and act like a tester
2 – http://www.extremeprogramming.org/rules/pair.html
3 – http://msdn.microsoft.com/en-us/library/ff182173.aspx
4 – http://www.extremeprogramming.org/rules/functionaltests.html
5 – http://www.refactoring.com/
6 – http://martinfowler.com/articles/continuousIntegration.html
7 – http://www.faqs.org/faqs/software-eng/testing-faq/section-14.html
About the Author
Srinivas Murthy Srinivas is a senior QA consultant at ThoughtWorks EU, where he worked on various Agile projects implementing Extreme Programming techniques. He is an Agile enabler and practitioner guiding teams and highlighting the values of testing. With an IT experience of over six years in various countries, he is very passionate about improving product quality and customer satisfaction.
Srinivas has done his Masters in Information Systems from Manchester Business School and holds Bachelors degree in engineering. He frequently blogs on http://srinivasmurty.blogspot.com/