Yesterday I started reading
Agile Testing: A Practical Guide for Testers and Agile Teams by Crispin and Gregory. Oh, it's a good book. I think the authors deserve serious applause and credit. They went out into the field and asked people what really works - and waited for multiple responses before weighing in. They had an extended review process, and they had hundreds of references. I'm impressed, and I recommend it.
But something happened when I started thumbing through those hundreds of references - I noticed that of 200 or so references, all but one were dated after the agile manifesto, which was penned in 2001. The sole exception was a paper about session based test management, written
the year before.
Don't get me wrong; telling the history of software testing is simply not the Goal of "Agile Testing" Book - it is about Agile Testing, something essentially born in 2001, and it does a good job of that.
Yet the Agile Manifesto says that we are "uncovering" new ways of developing software by doing it and helping others do it. To do that, shoudn't we have a balanced sense of history?
Example: Not too many years ago, I was a young student of mathematics and took a 400-level course called "History of Mathematics." In fact, I'm pleased to see it's
still on the books.
The course gave us a sense of the history of where math came from - from ol'
Pythagoras through
NP-Complete.
More importantly, /I/ learned a lot about how mathematicians think; how they model and solve problems - from direct proof, to proof by induction, to reducto ad absurdum. Reducto ad absurdum, for example, is really interesting: To prove something is true, assume it's false, and keep going until you find a contradiction.
But I won't bore you with math proofs; this is a testing blog.
So, if you wanted to read an article or take a course like that for software testing: A history of the ideas in the field --- where would you go?
...
time passes ...
...
Chirp, Chirp.
...
Oh, sorry, that's a cricket.
Oh, perhaps, if you were an academic, you might find a survey of the testing literature on
CiteSeer.
If by some Miracle, you find one written in plain English, drop me an email. In the mean time, I'm not holding my breath. I've been thinking of developing a paper, talk, lightning talk, article, series of blog posts ...
something about the history of software testing to give the newbie some idea of the ground we are covering, so we don't have to have this same discussion of "
should the testers be involved up front" again and again and again.
So, let's see what we have totally and completely off the cuff:
1958+ (Buddy Holly; Elvis)
Jerry Weinberg leads project Mercury Test Team (IBM), first independent test team
1960's (The Beatles; Star Trek)
Computer Programming Fundamentals, by Herbert Leeds and Jerry Weinberg, describes software testing
PL/1 programming: A manual of style, Weinberg publishes the triangle problem for the first time
1970's - (Lynrd Skynard, Thick Ties)
Time Sharing Systems. Birth of "It works on my machine."
The Art of Software Testing, Glenford Myers
- Equivalence Classes, Boundaries, Error Guessing, Cause/Effect Graphing (It's history, good and bad) - Functional to Unit Testing
"Managing the development of Large Software Systems", Dr. Winston Royce.
Cyclometric Complexity, Thomas McCabe
1980's - (Swatch Watch)
Software Testing Techniques, Boris Beizer
Black-Box Testing Boris Beizer
- Every software program can be expressed as a directed graph of blah blah blah blah
Test Cases; V-Model System/Integration/Unit
Code, Branch, Input Coverage Metrics
Really interesting stuff in Silicon Valley; tester-as-expert mythos (The Black Team)
"Rethinking Systems Analysis and Design", Jerry Weinberg - Iterative Development and Testing described
Early 1990's - (Thin Ties)
Record/PlayBlack (Winrunner)
Testing Computer Software (Kaner, et al)
Bug Tracking and Version Control become popular
- Version Control changes the popular meaning of regression testing
ASQ-CSQE (and Deming, and Drucker, and Juran)
"Software Testing, a craftman's approach" Jorgensen, Petri Nets and State Transitions
STAR Conferences Start, 1992
Later 1990's - (
Friends)
Los Altos Workshops on Software Testing Begin
When should a test be automated, Brian Marick
Test Driven Development - Extreme Programming - Becomes Popular (Beck et al, xUnit)
Customer-Driven Acceptance Tests (The XP Crew)
Exploratory Testing (Bach)
Rapid Software Testing (Bach/Bolton)
Test Automation Snake Oil, James Bach
- The Minefield Problem
Performance Testing / Web Testing Takes off
Heuristics (Bach)
"How to Break Software", James Whitaker, Quick Tests (Also, ESH popularized
quick tests about this time)
Early 2000's - (
The West Wing)
Session based test management (Bach)
Keyword-Driven (Coined 1999, Graham/Fewster, popularized by Linda Hayes/Worksoft Certify)
Manfiesto For Agile Software Development Continuous Integration. (More Agile-ness)
FIT/Fitness (Ward Cunningham)
"Lessons Learned In Software Testing" (Kaner/Bach/Pettichord)
WaTIR (Pettichord et al)
ISTQB
Six Sigma
"Key Test Design Techniques", Lee Copeland. Unifies approaches to testing; popularizes the insurance problem as an alternative to the Triangle problem
Model-Driven Testing, Harry Robinson
Software Engineering Metrics, What do they measure and how do we know, Kaner
Later 2000's - Selenium Mocks, Stubs and Fakes.
Acceptance-Test Driven Development (
Marcano and
Hendrickson)
"Agile Testing", Crispin and Gregory
Faught questions the teaching value of the triangle problem
The Balanced Breakfast Strategy
Hopefully, by now, readers know that I am a "Throw stuff up against a wall and see what sticks" kind of person. This list is ugly, probably contains mistakes, and is just a start. It's tentative. It contains a history of the evolution of Agile/Context driven ideas. If your pet paper, book, or idea isn't on this list, is influential, and fits, leave a comment and it may get on the list.
The idea to is to get our comment juices going and start filling in the gaps; to help make a list that is
good enough and yet
comprehensible.
Then I'll turn every reference into a hyperlink, and it'll be a self-study guide. With a little more work, it might turn into an article or presentation.
What do you say; want to help me out?
UPDATE: For the purposes of this post, I'll consider functional, performance, and test management as "testing"; I may do a future, separate and distinct list for security testing or for regulated testing (FDA, MILSPEC, etc is out of scope.)