Yesterday I started reading Agile Testing: A Practical Guide for Testers and Agile Teams
by Crispin and Gregory. Oh, it's a good book. I think the authors deserve serious applause and credit. They went out into the field and asked people what really works - and waited for multiple responses before weighing in. They had an extended review process, and they had hundreds of references. I'm impressed, and I recommend it.
But something happened when I started thumbing through those hundreds of references - I noticed that of 200 or so references, all but one were dated after the agile manifesto, which was penned in 2001. The sole exception was a paper about session based test management, written the year before
Don't get me wrong; telling the history of software testing is simply not the Goal of "Agile Testing" Book - it is about Agile Testing, something essentially born in 2001, and it does a good job of that.
Yet the Agile Manifesto says that we are "uncovering" new ways of developing software by doing it and helping others do it. To do that, shoudn't we have a balanced sense of history? Example:
Not too many years ago, I was a young student of mathematics and took a 400-level course called "History of Mathematics." In fact, I'm pleased to see it's still on the books
The course gave us a sense of the history of where math came from - from ol' Pythagoras
More importantly, /I/ learned a lot about how mathematicians think; how they model and solve problems - from direct proof, to proof by induction, to reducto ad absurdum. Reducto ad absurdum, for example, is really interesting: To prove something is true, assume it's false, and keep going until you find a contradiction.
But I won't bore you with math proofs; this is a testing blog.
So, if you wanted to read an article or take a course like that for software testing: A history of the ideas in the field --- where would you go?
time passes ...
Oh, sorry, that's a cricket.
Oh, perhaps, if you were an academic, you might find a survey of the testing literature on CiteSeer
If by some Miracle, you find one written in plain English, drop me an email. In the mean time, I'm not holding my breath. I've been thinking of developing a paper, talk, lightning talk, article, series of blog posts ... something
about the history of software testing to give the newbie some idea of the ground we are covering, so we don't have to have this same discussion of "should the testers be involved up front
" again and again and again.
So, let's see what we have totally and completely off the cuff:1958+
(Buddy Holly; Elvis)
Jerry Weinberg leads project Mercury Test Team (IBM), first independent test team1960's
(The Beatles; Star Trek)Computer Programming Fundamentals
, by Herbert Leeds and Jerry Weinberg, describes software testingPL/1 programming: A manual of style
, Weinberg publishes the triangle problem for the first time1970's -
(Lynrd Skynard, Thick Ties)
Time Sharing Systems. Birth of "It works on my machine."The Art of Software Testing
, Glenford Myers
- Equivalence Classes, Boundaries, Error Guessing, Cause/Effect Graphing (It's history, good and bad) - Functional to Unit Testing"Managing the development of Large Software Systems"
, Dr. Winston Royce.Cyclometric Complexity
, Thomas McCabe1980's -
(Swatch Watch) Software Testing Techniques
, Boris BeizerBlack-Box Testing
- Every software program can be expressed as a directed graph of blah blah blah blah
Test Cases; V-Model System/Integration/Unit
Code, Branch, Input Coverage Metrics
Really interesting stuff in Silicon Valley; tester-as-expert mythos (The Black Team)
"Rethinking Systems Analysis and Design", Jerry Weinberg - Iterative Development and Testing describedEarly 1990's -
Testing Computer Software (Kaner, et al)
Bug Tracking and Version Control become popular
- Version Control changes the popular meaning of regression testing
ASQ-CSQE (and Deming, and Drucker, and Juran)
"Software Testing, a craftman's approach" Jorgensen, Petri Nets and State TransitionsSTAR Conferences Start
, 1992Later 1990's -
)Los Altos Workshops on Software Testing
BeginWhen should a test be automated
, Brian Marick
Test Driven Development - Extreme Programming - Becomes Popular (Beck et al, xUnit)
Customer-Driven Acceptance Tests (The XP Crew)
Exploratory Testing (Bach)
Rapid Software Testing (Bach/Bolton)Test Automation Snake Oil
, James Bach
- The Minefield Problem
Performance Testing / Web Testing Takes off
"How to Break Software", James Whitaker, Quick Tests (Also, ESH popularized quick tests
about this time)Early 2000's -
(The West Wing
Session based test management (Bach)Keyword-Driven
(Coined 1999, Graham/Fewster, popularized by Linda Hayes/Worksoft Certify)Manfiesto For Agile Software Development
Continuous Integration. (More Agile-ness)
FIT/Fitness (Ward Cunningham)
"Lessons Learned In Software Testing" (Kaner/Bach/Pettichord)
WaTIR (Pettichord et al)
"Key Test Design Techniques", Lee Copeland. Unifies approaches to testing; popularizes the insurance problem as an alternative to the Triangle problemModel-Driven Testing
, Harry RobinsonSoftware Engineering Metrics, What do they measure and how do we know
, KanerLater 2000's - Selenium Mocks, Stubs and Fakes
Acceptance-Test Driven Development (Marcano
"Agile Testing", Crispin and Gregory
Faught questions the teaching value of the triangle problem
The Balanced Breakfast Strategy
Hopefully, by now, readers know that I am a "Throw stuff up against a wall and see what sticks" kind of person. This list is ugly, probably contains mistakes, and is just a start. It's tentative. It contains a history of the evolution of Agile/Context driven ideas. If your pet paper, book, or idea isn't on this list, is influential, and fits, leave a comment and it may get on the list.
The idea to is to get our comment juices going and start filling in the gaps; to help make a list that is good enough
and yet comprehensible
Then I'll turn every reference into a hyperlink, and it'll be a self-study guide. With a little more work, it might turn into an article or presentation.
What do you say; want to help me out?UPDATE:
For the purposes of this post, I'll consider functional, performance, and test management as "testing"; I may do a future, separate and distinct list for security testing or for regulated testing (FDA, MILSPEC, etc is out of scope.)