Between a blog response or two here and two podcasts on the subject, with Jon Bach's keynote from STPCon still to come, we've provided plenty of coverage of test is dead right here.

If you'll permit me, I would like to add one more little wrinkle to the mix.

Now any attempt to reduce a one-hour talk to a sentance or two is going to be lossy, especially one articulated by multiple people, in different ways, at different points in time.  Yet if I had to reduce the test is dead movement to a few sentences, I would say something like this, and have some confidence in my statement:

"Hey man.  All testers do is report conformance to spec (or lack thereof).  So why keep a whole army around?  We can have a few test automators writing automated GUI tests, hire a company to do crowdsourced exploratory testing (or rely on our users), and cut the fat."

What's wrong with that?

As I've said before, if you give away your software, if that software is funded by advertisers, if you can monitor production and rollback/fix/redeploy quickly, and you don't touch money - sure, its possible to be successful with a strategy like that.

Mostly.

Only every time I engage on this, something strikes me wrong.

The idea above is reductionist:  It reduces testing to conformance to spec.  

But what about?

The tester that notices something going on in the user interface is not quite right, and makes suggestions to improve the specification?

The tester with domain expertise, who notices that a change in Product X is inconsistent with the behavior of Product Y?

The tester who knows the product so well that she is pulled into tech support meetings, who find the problem, isolates it, and gives a test (sometimes executable) over to the product owners to prioritize or the developers to fix?

The tester than can explain to the programmers why the software does what it does now, due to a complex business process?

The tester that knows what is really going on, and can inform management about it in time to change plans to meet reality?

... and those are just five questions I thought of offhand, over my lunch hour.

There are a lot of ways that a traditional tester can add value in the process that "test is dead" seems to ignore.

Arguably, these these are not "traditional testing."  and yet, I still call it testing.

I still call it testing

During the STPCon panel discussion, I made the statement that one way to improve delivery speed is increasing code quality before it gets to test.  

A few people took exception to this, saying that testers can be all over the process, helping to improve the product design, the usability, the user interface, and exploring, exploring, exploring.  This is very much in line with my statement above, yet I like the term testing.  I'm not ready to give up on it yet.

I'm still thinking on it.

For the time being, though, I am convinced that there are opportunities to add value testers have that GUI-Automation/Crowdsourced testing is missing.  What am I missing?

I would like to hear from you.

Last time, coming back from the high of STPCon spring, I wrote about the fall conference (Oct 15-18 in Miami!) and the new, special, hands-on track we will have at the conference.  Within a few hours of writing my post, the folks at STP put the Call for Session Proposals up on the public website.

Let me be clear.

As it is, we are looking at a hands-on track for the conference, and it's going to be good.  My expectation is to take things up a notch, which, for a high calibre conference, is hard to do.

I'd like to do better.

My goal is to radically change the conference experience, and I would like your help.

In the past, my school, the Context-Driven School of Software Testing, has been accused of elitism, or an insiders club.

For this conference, I don't care what school you belong to if your ideas are good.  

Maybe you are a security guy, or a performance tester, or into the lean startup.  Maybe you have a magic javascripty tool that helps with a/b split testing.  Maybe you have a technique you use to test mobile applications, or a way to decide what tests you can skip.  Maybe you learned a cool scripting language that allows you to do really powerful repeatable stuff.  Maybe you ...

aww, why don't I just provide a big list of possible talk ideas?

If you've ever wanted to give a talk but didn't know what to say, consider these:

1) No Spec?  No Problem!  Quick Attacks for web-based software
2) Test Case Creating - And Doing - In Real-Time with (Technique)
3) A Funny Thing Happened on my bus ride on my iPhone, or: Mobile App Testing HandsOn
4) How to build a log-file analyzer in (language)
5) How to be a hero at work by building your own supper-happy fun Diff File Compare tool
6) Fitnesse/SpecFlow/DBFit/Take your pick tool install hackfast
7) Jenkins and other CI tools for fun and profit
8) Penetration Testing In About an Hour
9) Performance Testing with (Tool) - JMeter? SOASTA?  Telerik?  Cloudstorm?
10) Testing Cloud-Based Apps with (Technique)/(Platform)
11) Using the Cloud to test with (Technique/Platform)
12) Testing REST Apis with (SoapUI?)
13) From Bare Metal to running selenium tests in about an hour
14) Just enough (PHP/Rails/HTML5/Flex) to do (purpose)

Of course, the Heusser-Style Comedy Title is Optional; the only requirement is that any tool you demo must have a significantly valuable free version.  (Vendors will still have go through the traditional STP Route, which prevents pitch-talks and conflict of interest.)

Now if we have a massive outpouring of energy, we may even be able to schedule a few evening events. I mean, WereWolf was fun and all, but how about a HackAThon or a test competition?

What now?   

Well, I can push, I can prod, I can suggest, but as a community, if this is going to happen and be successful, I will need your help.

If you've ever wanted to give a talk at a test conference, and have it be wildly different, valuable, and awesome, well, the call has gone out.

The next step?

This is up to you.

We have just completed another STP conference in the city of New Orleans and from the feedback we received it was a great success! That is good news to us here at STP but what does that mean to the software industry?

When industry practitioners, consultants, and sponsors come together in person it is a great opportunity to discuss and determine what is happening in the industry. Can the same discussions happen online? I say no. You can have conversations and build relationships on-line but they are not the same as meeting people in person. I know the new media folks are typing away right now to rebut this statement, but no words on-line can convince me that in person discussions are the same. Good luck trying.

What I observed and took part in was a conference that created some binding and long term business relationships that are stronger due to a hand shake and an in-person discussion. I watched a number of attendees ask questions in the sessions and as a result ended up in a hallway conversation passing business cards and making promises to keep in touch.

In the sponsor hall where we held the welcome reception it was a buzz with discussions about new solutions and the importance of validating the industry through practices and industry certifications. Again, I understand there are strong feelings about practices and certifications but individuals can determine what is best for them and their career. The conversations went well past the official closing time and many were still engaged in conversation while the breakdown crew disassembled the hall because they were tired of waiting.

I observed a number of groups making their way down Bourbon Street (to network I’m sure) and I promise I will not share names or pictures out of professional courtesy. Only kidding kind of…

In many industries the speakers and sponsors often play very specific and limited roles during the conference. At STP it seems that speakers and sponsors contribute throughout the entire event. They attend the keynotes, sessions, formal and informal networking. It says something very positive about the dedication of the individuals in the industry.

The trending topic has to be mobile testing. As Matt Johnston of uTest shared in his keynote which was more like a “State of the Industry” address, mobile devices and the apps that are targeted toward them are on an exponential growth pattern. That trend will force organizations to face the reality of having a mobile testing strategy. As Matt and many others suggested, QA needs to get a seat at that table because every Board Room is discussing mobile. We here at STP will need to make sure we provide these presentations at our October conference in Miami.

Thank you to all of our keynote speakers, session presenters and sponsors. The great feedback is a reflection of your hard work in preparing and delivering a great experience for the attendees of the conference! We can't do it without you and it is apparent we make a good team.

As we dig through all of the evaluations and review the new mobile application we tested at this conference we will make the adjustments needed to continue the growth we are experiencing as a conference and association. The mobile app we tested during the conference received an unexpected value of having a conference full of testers test their app, for free! The company truly appreciated the feedback and fixed many of the bugs that were so graciously pointed out by the attendees. I extend their thanks to you!

If you haven’t been to a conference in a long time I suggest you try it out and get connected to people in person again. I guarantee it will energize you. I hope if you do choose to go to a conference you choose us here at STP. I believe we are the best in the business of delivering great events and our surveys definitely support my claim. But don’t take my word on it, come test us…

I’m here in New Orleans at STPCon, enjoying a post-conference-day high after day one, looking forward to day two during day two.  (I wrote this yesterday, but didn't get a chance to post until today.  Speaking of busy ...) 

From flying in to town, to working the weekend at an Association for Software Testing board meeting, to participating in the career workshop by Fiona Charles and Anne-Marie Charrett, then dinner, it’s been a whirlwind.

The workshop was wonderful; it included an exercise about personal values, presentations by a seasoned tester, test manager, coach/trainer, and a consultant, a discussion of what skills/qualities/learning opportunities exist for the roles, and a visioning exercise.  (I was invited to present as the consultant, for which I was honored.)

After the workshop, the group mostly hung out in the hallway until dinner; Ben Simo brought a box game that was a spacial challenge, and Eric Jacobson stopped by.  

C’mon, you know Eric Jacobson, he’s been on the podcast three times, is an active blogger, and keeps doing more and more in public about his work as a test manager in the Atlanta, Georgia, area.

We got into a heated discussion about software testing (imagine that!) and two of the things he said freaked me out. 

It turns out I agree with just about everything he said, but still ... it kind of freaked me out.

He started out by asking “Who Needs a Regression Test?”

Let's slow down a bit and talk about test automation and assumptions.

The Assumptions’ll Kill Ya

When I say test automation, I might mean something that drives a GUI, written after the code is developed by a team in a different context -- or I might mean that the developers take a spreadsheet-like specification of “Examples”, and, at the business-logic level, use a tool to demonstrate that the examples pass before handing the code to  testers to explore.  (Done well this can drastically improve code quality before it gets to ‘test.‘    It also creates some amount of regression-test automation you can run before any deploy -- nifty!)

Or I might mean unit testing, or a performance test, or a half dozen other things.

In the software testing world, unless I am very careful to spend a few hundred words setting the scene, it is likely when I use the phrase “Test Automation”, people will assume I mean something else.  Most recently and painfully, they think I means GUI Driving automation, when in fact I am talking about those business-logic level tests.

I’ve been struggling so often to communicate about this; I have decided to write a position paper on test automation -- but that is not this post.

Which brings to me to what nearly happened when Eric Jacobson asked “Who needs regression testing?”  

Eric meant on a maintenance team, where the team is maintaining a large (50? 60? 70? 100?) number of truly independent, small applications, where each one can be deployed independently.  In that world, ‘regression testing’ the application and testing the new feature are very similar - so why talk about it?  If that is true, then my concern that Kanban lacks a regression test ‘step’ is irrelevant, and Eric is right.

Which wasn't what I assumed he meant when he started talking.  The model in my head was different.  We had to talk through it. Eventually, I 'got' it.

It still kinda freaked me out.

The Bottom Line

Some people may not to able to make STPCon.  I get it.  The distance is too far, the conference budget is frozen, the whole team really needs to focus on the 1Q2012 release, whatever the case may be, I get it. 

What I am saying is to find the kind of people that stretch your imagination, that make you think, that make you want to be your best self.  You don’t need a conference for that; it might be user’s group, an online forum, or two people from work over beverages Thursday evening.

It may freak you out, but it will be a good kind of freak out.

Go find your own Eric Jacobson.

Older Posts

Newer Posts