I have been working as an automation specialist in one form or another since the first hour of my first job. Since then, I have helped a number of organizations and consultants learn how to automate. Inevitably the automation solution revolves around their specific technique or tool. Now, as a consultant the tables are turned and I am the outsider.
As an outsider, when I start working with a new company I need to spend just as much time undoing and unteaching the work of previous consultants and their best practices as I do introducing my concepts and practices. The time I spend unraveling misnomers with my clients leads me to reveal the top six shocking automation truths. These truths are shocking not only because they are counter to the prevailing best practices in the industry but because they can cause me – the consultant – to lose a billable opportunity. A client who saves money but achieves their objectives is often the happiest client.
Truth One: You do not need to automate everything
If some automation is good, then a lot of automation must be better, right? Well, not always. Automation is expensive in terms of both creation and maintenance time. That time could have been used testing the application. Ask yourself why you are about to automate something. Will it make something long, short? Will it make something complex to test, simple? Will your coverage dramatically improve? If not, then you might be automating for the sake of automating which is not a good use of resources. The questions do not have to be captured in an algorithm– just have a conversation with your neighbor to see if it is worth the effort; it often is not. Selective automation is bad for the consultant but good for the client.
Truth Two: Going forwards in reverse is still going backwards.
Let me know if this sounds familiar. Your application has gone through a couple releases and there is a large chunk of application which has no automation wrapped around it. Again, automation is good so the plan is to flush out the automation of the existing stuff. Good plan, right? Sure, if you want to go backwards. When introducing automation to an existing project, ignore everything from that day back. Instead, focus on the new features; they are the ones that have the most risk in them — they are new! Automating the prior work is a recipe for both falling behind on new feature automation and inflating the automation budget. Good for the consultant, bad for the client.
Truth Three: The automation language does not have to be the same as the development language
I cannot count the number of times I have heard a variant of ‘our application is written in language x, so the automation shall be as well.’ If your automation team knows language x then there is not a problem, but if they know language y or no language at all, it quickly becomes a problem. Want to see someone flounder when starting automation? Throw them at Java or C# where they have to learn about objects. A more successful route is to start with something like Python or Ruby. Use the java or .net variants of them and you can even access the application natively. Of course, long-term training and hand-holding arrangements are good for the consultant, but wouldn’t you rather be self-sufficient?
Truth Four: There is not one tool to rule them all
Automation tools were not forged in the fiery depths of Mordor, they come from a wide variety of sources to solve an even wider variety of problems. You most likely will have to mix-and-match a variety of tools to get optimal results. Yes, some of the integrations between specific tools can be sexy, but what are you giving up in return? Consultants love the one tool myth as it allows them to tap into the marketing effort of that tool. I am strongly aligned with Selenium and some of the largest commercial interests associated with it, but not so much that I would recommend them if it wasn’t in the best interest of the client.
Truth Five: There is no ‘right’ or ‘wrong’ way to automate (though there are better and worse).
Within the first few minutes of most exploratory conversations around consulting, the potential client will say they want to learn about automation best practices. Here is the truth about best practices; they are a myth. Automation, like testing, is both heuristic and highly context sensitive. I could promise you best practices, but what you would end up with are custom practices that fit your unique problems. Those are not best practices, those are custom solutions. That said there are better and worse ways to do automation, but those form the groundwork of a custom solution. The term ‘best practices’ was likely a concoction of a consultant since everyone wants the best, but if you are getting a canned solution before they even know the problem it is anything but the best.
Truth Six: Your automation environment is not production.
If hitting your thumb with a hammer hurts, you stop doing it. Same thing applies for some problems in the automation space. If your automation is difficult because of a specific server configuration that makes it look like production, change it. I see this all the time in the Selenium world regarding HTTPS and certificates. Unless your application actually needs to be over HTTPS then there is no reason to do so. It is not production; these are not real users with real credit cards and personal information. By just disabling security, a difficult problem may have become non-existent. Now, the mercenary side of me says I’d be happy to spend a day developing a custom profile for your automation that has all the certificates and security settings done correctly, but clients likely have better things to do with their money.
Today’s applications are more complex than ever, and the automation of them follows in lockstep. This often requires the outside expertise of a third-party consultant. As an automation specialist and now consultant, these truths can be a bit shocking, but are important for both sides to realize, even if it means that the client may not need the consultant as extensively as they originally thought. There will be many future opportunities which may actually result in a bigger win for the consultant if he gains his client’s trust.
About the Author
Adam Goucher: Adam Goucher has been testing professionally for over 12 years at a range of organizations from national banks to startups. A large part of that time has been spent augmenting his exploratory testing with automation. After discovering Selenium during a capital budget freeze in 2006, it quickly became his tool of choice when automating web interfaces. As an active member of both the core development team and the larger Selenium community he has recently started to revitalize the IDE project in an effort to kickstart business adoption of it. Automation is not the only tool in his toolbox. As an experienced tester who believes in the principles of both the Context School of Testing and the Agile Manifesto he brings an open and modern view of testing to projects. His first book, Beautiful Testing was published in October 2009.