It is my pleasure once again to present at the Software Test Professional Conference fall 2013. Actually, present is not the best word, facilitate is more accurate. I have the opportunity to facilitate a Lean Coffee session, which is a fairly newly structured, yet agenda-less meeting and discussion format. According to the Lean Coffee website: “Participants gather, build an agenda, and begin talking. Conversations are directed and productive because the agenda for the meeting was democratically generated.” This Lean Coffee session will focus on Mobile Testing in an Agile environment, but all topics are in scope as the group votes on what they want to discuss. From this Lean Coffee session participants will not only learn about the topics discussed but also be able to use this meeting format on their projects at home.
One thing that has been on my mind lately is barriers to entry for mobile test automation. I am not an SDET. In 15 years of QA experience I’ve tried multiple times to learn code; it didn’t stick. However, I was recently challenged by both my client and internal project leadership to implement test automation on my current agile mobile project. My first task was to request, and wait for, our sole iOS automation subject matter expert (SME) to grant us some of his valuable time. Once it became clear I was not going to get this scarce resource I had the choice to either make automation happen using my existing project resources or inform my stakeholders that I was unable to fulfill their request. Thankfully I was able to meet this request with the assistance of two other QA Engineers by utilizing a ground breaking process created by Dr. Leo Marvin at the turn of the Millennia: “Baby Steps“.
In my experience people often think of software automation as an all-or-nothing endeavor; as though they have to ignore other testing duties to focus full-time on an automation framework and infrastructure. At my first mobile job there were two well-meaning SDETs, one iOS and one Android. They proactively took it amongst themselves to come up with a system that all QA Engineers could use to get up to speed in mobile automation. After weeks of designing and writing they created a tutorial for this purpose. However, the unintended result was a tutorial with a huge barrier to entry. While I can see the potential benefit for a full featured automation tutorial, I’m a simple man, and I prefer baby steps. There is low-hanging automation fruit to be picked on every project. Below are examples of three low-hanging automation “fruits” that we picked and the process we used to implement them. The exact solutions might not translate directly to your project, but I guarantee that the baby steps will.
First Fruits – Jenkins and Continuous Integration:
First thing to do was to get our project connected to an instance of Jenkins. This free, and easy to install, little man will sit around and wait for a check-in and immediately build the project, giving us instant feedback on the build. Now we have a continuous integration system alerting us to build breaks–Baby Steps! Next step is to get Jenkins to step his game up a bit and provide a build verification test upon check-in.
Second Fruits – Build Verification Test:
We initially wanted to use Calabash, which is a Ruby based solution for automation that is very user friendly. However we were unable to use Calabash due to technical challenges specific to our project. Therefore we decided to use Apple’s UIAutomation framework, part of the Instruments suite of test tools included with Xcode. This solution has a higher learning curve than Ruby based solutions, but with that challenge comes more functionality. Using UIAutomation along with a little JavaScript knowledge that we learned online, we were able to quickly implement a basic BVT script that simply launched the application, logged inout, and closed the application. Now we have a daily automated BVT alerting us to broken backend services–Baby Steps! Next step, which we are currently working on, is to extend the basic BVT to add more functionality and additional logging.
Third Fruits – Unit Testing:
Where the first two low-hanging fruits build on each other, this third one is a bit different. It has been my experience that agile teams like to talk about unit testing but no one really likes to do it. I’ve heard many development teams, at many sprint retrospective meetings; challenge themselves and each other to write more unit tests. Often it is the case that developers will write unit tests but no one continues to execute them after the developer has moved on to coding another feature. I was surprised when I asked a Developer to teach me how to execute the unit tests in Xcode that it took about 5 seconds for all 43 tests to complete. In fact, some of the tests run so fast that Xcode does not know they were executed and reports them as “Not Complete.” It made sense to me that if I’m already running BVTs every day I could easily execute and report on the unit tests as well–Baby Steps! Next step here is for QA to write unit tests for new functionality. I’m pretty sure my Development team will have a parade in our honor once we are able to take unit test creation off their to-do list; or at least assist in the effort.
Again, these exact solutions might not be viable or even possible on your particular project. Regardless, here is your low-hanging fruit recipe. 1: Baby step your way toward an immature automation system; 2: Wow your Project Leadership with how much you can do with little resources; 3: Watch them give you more resources. Like me, you will suddenly find yourself as the “unintentional automation SME.”
Here is the email I was able to send to my project team when we finally completed the automation solution we initially designed:
“All:
After our initial launch of <ABC>, Project Leadership requested a few things of the QA team, one of which was more automation. This, along with a request from the <customer> team, kicked off a series of meetings that resulted in a decision to go ahead on the automation effort even though we did not have an iOS Automation SME resource. We created an end goal and baby steps to get there. Through a number of startstops, technical issues and assistance from others; I can finally say we are “There”.
- Jake has created a fairly robust script, using multiple files, that has functions for testing each functional area in <app>; we even found a few app bugs along the way. Ask him to walk you through it, you’ll be impressed.
- I have taken on the task of running the unit tests daily, along with the BVT, and it’s already alerted us to down services a few times. I can assure you that we are the only QA team at the Deloitte Digital Seattle studio that executes developer unit tests.
We have the most mature, and likely only, iOS daily automation process in the Seattle Office. We were able to achieve it while maintaining our quality of manual testing and even executing more Charter testing (hence Showstopper Salli), an effort also requested by Project Leadership.
Search Jira for the label ‘automation’ or type ‘charter’ to see specifics on those individual efforts.
Future Plans?
- Short term, monitor and investigate code coverage and minor bugs in both BVTs and Unit Tests
- Medium term, maintain the BVT script by keeping up with functionality as it is added and create a process for doing so
- Long term, QA responsible for writing unit tests
That’s that for now. Questions? Comments? Requests?
About the Author
Todd Shultz Over the past 15 years Todd has tested many types of software, in many industries, from web pages and insurance software to audio/video software and video games. Currently he is working on mobile applications, deep diving into this emerging market to identify the unique Quality Assurance challenges and how to mitigate them in an agile way. Todd is passionate about professional development; mentoring Quality Assurance Engineers who see QA as a career, and not just a job.