Right now one of the "louder voices in the room" for test automation is the "Agile" test automation voice. And by that I mean something very specific - that you create a large series of
clerical tests that go from acceptance level all the way to unit; that you can run those tests at the click of a button, and get a green or redbar (pass/fail) very, very quickly. If any component in the system is slow, you can mock or stub it out to get that fast feedback. A few people who talk about this idea admit that you may want to actually run the system end-to-end, for example by running the full, slow test suite overnight.
This kind of test automation can be very valuable, especially for software developers, but it isn't the only kind.
Earlier this week, Ben Simo and I were kicking ideas around and we came up with:
1 Any tool that eliminates manual, clerical work is test automation (
diff, loadrunner,
tasker)
2 Finite-State-Machine, Random-Number Generating Tests are test automation (In the
Harry Robinson sense, or, for that matter, the
Ben Simo sense)
3 "Big FOR loop with an external, trusted Oracle" test automation
4 Develop it twice, run it on both systems, and compare the results test automation (Create two, not-very-trustworthy oracles)
5 Monkey Test Automation. Press Random buttons, record the order, wait for a crash, and then dump the log.
6 P
airwise Test Generation tools are test automation.
7 Any test case generator (
perlclip) is test automation
This is a quick, sloppy list. What are some other forms of test automation? What am I missing?
It would be awful nice to come up with a consistent model for different types of testing tools. Failing that, I'd settle for a vision, or, to be honest, a "good start."
Elisabeth Hendrickson just announced the A
gile Alliance Functional Testing Tools Visioning Workshop, in Portland, in October. I think all of these could fall into that vision. If I could only cover the travel expense and lost income, i'd be there. As it is, I look forward to see what falls out.
If you can make it, I'd encourage any regular Creative Chaos readers to attend. In other words, if you like this blog, I think you'd like the workshop, and, I would hope, vice versa ...