Nearly a year ago I did a blog post that included a quote from the "Man in the Area" speech by Teddy Roosevelt.  That speech is about daring greatly; about stretching yourself.

This spring, after STPCon, I decided to try another stretch - to propose that the fall conference have a hands-on testing track.  

By hands-on, I meant just that: That the attendees would be actually doing software testing.  To get this track, I approached the decision makers at STP and made my pitch, along with Tim Peysar and Mark Tomlinson.

They went for it, but only if I would take responsibility to run.

No, problem.  I'm Matt Heusser, right?  I dare greatly.

I immediately went into recruiting mode for the conference, with planning mode, pitching mode, and a flurry of work.  I posted on it, pitched it, was out on twitter and the intarwebs.

After the CFP closed, can you guess how many people put in for sessions?

Zero.

What just happened?

There were a lot of things colliding here that made for zero responses.  First, a number of absolutely great speakers are taking themselves off the circuit in the fall, or a lot of reasons.  Some folks are sick of travelling, others have billable clients that need their attention, some just won't be on the right continent at the right time. Our intention was that all test software be web-based, or at least a quick download, and the CFP says "web only" - it is possible that some folks thought hands-on was a webinar, or we just confused.

But I don't think that explains it all.

I think hands-on sessions are hard.

Seriously, Hard.

First of all, you have the install problem; some people are going to need some hand-holding.  Unless you are careful, there goes the first half of your session.  Then you have to explain something valuable; the quickest part of your audience will know the basics, the slowest will never catch on.  You will have questions form the audience that you have to field and improvise.  You have to develop exercises that scale to a wide ability group and a wide number of platforms.

Like I said, hard.  This is not kid stuff.

But what does it say when of the forty, fifty, sixty, seventy, who knows how many applications to speak at a conference, all of them would rather show powerpoints than actually do testing?

What is that all about?

The Common Responses

After I realized the low turnout, I went around asking why, and what I could do to fix it.  Besides "oh, Matt, I won't be travelling the fall", there were some common responses.  One was "I'm kind of new to the speaking biz and I'm not up for it yet", which I think is fair.  Another was "Matt, come on, it's STPCon.  This is a powerpoint-presentation conference.  It's the wrong venue."

They Are All Powerpoint-Presentation Conferences

Oh, I know, the Bachs and Michael Boltons do exploratory exercises in their tutorials, and there are some workshops with simulation-like activities.  Yet by and large, most public conferences are presentation conferences. (I counted on fingers, then toes, then ran out.  All the public conferences I have been to were presentation conferences.)

I know all the analogies about moving the ship and intertia, that you need a very big wheel and it takes a long time and so on, but saying "I don't see STPCon as a hands-on conference so I didn't apply" pretty much guarantees that it will stay that way.

The Good News

The conference organizers extended the deadline, and I am back in the hunt for speakers.  While no one responded to a general CFP, they seem to be responding better to individual requests along with suggestions, offers of feedback, and so on.  

I think we will be able to save this track, but it is going to be tough. And if this track has a bad year, it is unlikely you will see the idea back any time soon.

The real test is going to be in the free market: Will anybody show up?

Because I have to reach out individually, I am spending a lot more time with each speaker.  We will have much more detailed back-and-forth - the program will build from one session to another, have better flow and less overlap.  We'll be including cutting-edge issues like mobile testing and in-browser performance testing, and ending the event with a double-track testing competition.

Again, I have to wonder, will anybody show up?

That choice is up to you.

STPCon fall will be Oct 16-18 in Miami, FL.

Don't worry, there is plenty more to come.

A couple of months back I published an article called "Two Ways to Argue."  My basic point was that there are two ways to frame an argument - from agreement or disagreement - and that can influence how the discussion plays out.  When I work with people whom I have some shared understanding, folks who may be inclined sto hear what I have to say, I'm likely to start from dissonance.  When I am dealing with a skeptical audience, I prefer to start where we agree, and walk the discussion toward a conclusion that may be counter-intuitive.

That's great, but how do you actually make the argument?  I mean, once you have a style down, what do you actually say?

Say, for example, you'd like your programmers to do more testing before the software gets to general exploratory test.   It doesn't really matter the issue - the point is that we need to convince someone of something. 

If you think about it, this happens all the time.  The obvious issues in software are when (you think) the code is not ready to deploy, or the schedule is unrealistic, or feature A1 is better than feature A2, or when one bug is more important than a new feature, or whatever. 

All the time, I tell ya.

Now someone is going to say that no, our role in test is to provide information for decision makers, and I agree to a great extent - yet I would hold that one way we testers can add value is to be Quality Advocates (QA) - to help people make good decisions.

Even if we are 'just' laying out the case impartially, we want to know what matters.

Here are a few things to think about.

Facts and Statistics - The SquareCalc interface is supposed to run every day.  If it falls behind an entire month, then transactions "fall off" the end and can not be recovered.  Work in progress transactions are valued at about $5 Million per day.  The interface hasn't run two weeks.

People often argue in testing that we need "hard numbers" to "prove" to the executives that we are doing good work.  I am skeptical of bug counts and ratios, but the example above is meaningful.

Anecdotes, Stories and Quotes - A friend of mine once took bids on three contractors to build a swimming pool.  The first two came out, took some measurements, and gave him a quote.  The third came out with a little stick that he pushed into the ground.  After twenty minutes, he stood up and said he would need to bring out a backhoe and some equipment, to determine what was under the ground - bed rock would need to be blasted, but it's possibly that he just needed to dig.  When my friend pointed out that he had two solid quotes on price and schedule from people who didn't need to do any more research, the salesman replied "right.  And that's why you are going to hire me; because those guys made a promise they have no idea if they can keep."

The Anecdote approach reveals a Truth we all can realize if we just think about it in a different way.  Analogies and Metaphors do this by connecting us to something more familiar and tangible  -- like building something real, instead of something in bits and bytes. You don't have to use analogy; many would argue that the factory analogy has done more harm than good for software development.  (I would argue that most of the folks pushing the factory not only don't get software, but that they often don't understand how a modern, Just In Time factory works either!)

Like I said, you don't have to use Anecdote. Shared experience, for example, can work wonders.  A few times, instead of arguing that we should do this or that, I have said something like "hmm. This feels a lot like project Foo.  Do we want another project Foo?" and that has encouraged changes in policy and behavior.

Exposition - Gasoline is a petroleum product, and limited in quantity. Eventually, it will run out.   For that matter, all plastic items are made of petroleum.  Our current American consumption lifestyle creates waste, which poisons the environment and uses up the world's oil reserves.  Burning gasoline and other fossil fuels also creates pollution and sends carbon into the atmosphere, adding to the greenhouse effect.  If we would burn less gasoline and use less petroleum, the world would be a better place. 

The format here is if/then.  If we do this thing, then this other thing will happen, and the system of forces around these things produces this negative outcome, therefore we should not do the first thing.

I call this an expository argument; it is an argument from reason.  Most beginning writers tend to fall back on exposition; it is as easy as making an if/then case.  (A mere human, I often find I am relying heavily on this style, especially in my writing.)

It is also weak, because there is no actual person in the story, nothing hard to grasp on to.

But think about how much more powerful the argument above would be if we talked about:

- How much fossil fuel does the world have left?
- How much are we using per day?
- When does that project that we will run out?
- How much trash does the typical American family throw out per week? Per year?
- How many families are there in the USA?  How much trash do they create per year?
- How much pollution is created per year?
- What does that mean in a tangible way?  What are the results we can count?

Adding a few statistics moves the argument from an editorial to a position.

Throw a story or two on top of that: An endangered species, a family that got their quality of life back by eliminating 20 hours a week spent driving, and now you've got a real argument.

When speaking on an issue, know your audience, as individuals and groups may respond better to one of the three approaches.  Each has its strengths and weaknesses, and the strongest argument is usually made by combining them.

When we try to persuade

Perhaps you don't have this problem, but I am writing this article for mere humans, who make arguments and, sometimes, find themselves losing.

Next time you make a case and something seems wrong, take a look at how your argument is laid out.  Could it benefit from a little less of one style and a little more of another? 

You might benefit from experimenting a little and being a little more conscious of how you argue.  Even if you don't benefit, I've found a kind of enjoyment in variety, in asking myself is there a number that would help make my argument?  If there is, how can I get it and how can I deliver it in a reasoned way, with integrity?

Then again, sometimes you can't win, and sometimes it is best to remain silent, and knowing those two things can be very important.

Yes, there is plenty more to come.

I can still remember the moment.

I was sitting in an IT leadership meeting.  The team was talking about the great variety of reporting tools we were using.  The analysts were using MS Access, the programmers SQL Station, the Business Intelligence folks were using the Business Object front-end GUI Web Tool, and some business unit was trying to purchase Crystal Reports in order to do it's own thing.

I remember the moment, when the director of software development slammed his fist on the desk and growled "We need to standardize on one reporting tool, and we need to do it now!"  (I believe he said something about "Amateur Hour", but I am not sure.  This is a true story, and I don't want to embellish.)

It was a long time ago; I was a bushy-tailed young engineer, only present at the leadership meeting to discuss the progress on the Quality Assurance sub-committee, where I was the chairperson.

I wanted to ask "Why?"

Seriously, why?

So we have a little bit of redundancy in tools.  So we need to cross-train some of our support folks, and have some inefficiencies when people transfer and need to learn a new skill.

Ok.  So what? 

We a few thousand a year really a critical priority for a 120+ person IT department?  Didn't we have bigger projects, more important projects, more bet-the-farm projects to focus on, and other opportunities to save more time and money to look at?

For that matter, why now?  Couldn't we defer the decision a week, a month, a year, two years?  What did it hurt to wait?

Of course, I didn't say any of this out loud.  Remember, I was a young, easily influenced developer, scared for my job.  If anything, my body language probably implied the same as everyone else "Those other people have got to get there stuff together."

Ten years later, I think I have learned a few things.

There is no Them

Blaming the other guy is a great posture to take;  you get to be righteously indignant at the other guy and you don't have to do anything!

By attributing blame to an outside group, you are also claiming the problem is not your own, and taking  no responsibility for it.  So blame is great, in that we get to feel good, but it is very bad, in that the problem will probably not get fixed -- or at least, we aren't going to make any positive influence toward a fix.

Ten years after the director of software development slammed his fist on the table, the company still has a pile of reporting tools.  Somehow, I am not surprised, but that's only the first insight.

Defer Decisions

Deferring the decision as long as possible isn't just my idea; it turns out to be one of the principles from the Toyota Production System (TPS), the manufacturing system credited for eating Detroit's lunch.  The lean software development elevates this to a core principle, and it is not just the lean community - Robert "Uncle Bob" Martin, a co-author of the Agile Manifesto, lists the Liskov Substitution Principle as part of his SOLID principles of object-oriented programming.  Liskov encourages programmers to create interfaces, which allows them to "switch out" major components, like the web server, the database, the operating system, etc.  

When UncleBob started writing Fitnesse, they did the simplest thing that could possibly work, saving files to the file system, not a database.  By designing interfaces with Liskov, the technical team could get a proof-of-concept GUI up fast and work on persistence later.

When the app was getting close to completion, UncleBob realized it was good enough.  They didn't need to implement a DB layer.  The application was open source; thanks to Liskov, if someone needed it, they could "just" implement it themselves.

This is the power of deferred decisions.

Another term for the deferred decision is an option.  We know about these in the financial markets; you pay a little bit more to buy an option now, to control costs in the future.

Options in the Enterprise

Estimates are hard.  Look, they are.  We don't know how long things will take, and if we take a week to go create a work breakdown structure and build some confidence with work in meetings and on paper, it's likely that our estimates at the end of the week will be about as accurate as they were at the beginning fo the week!

Of course we can do it better than that.  We could spent the week actually creating prototypes and proofs of concept, building real confidence that the work is possible -- but it management wants to control scope and schedule, to do anything even modestly ambitious, and oh and it's got to actually work, then, well, it's a craps shoot.

That is where options come in.

Consider the CEO who needs to know if the team can deliver the application by October first, because if the company doesn't make October first, it can't make the Christmas Shopping Season.  He also needs to know if he should purchase television advertisements on multiple channels, because prices will go up exponentially as the date approaches October.

He could purchase an option; pay a modest fee to the providers, which will allow him to buy airtime in the future at a locked-in low price.  

Changing our goal, from locking-in decisions early, to deferring them, can change the way we think about our work and the way we do things.  Again, this is not my idea; Chris Matts has been talking about Real Options in the Agile Community for years.

Except I suspect that answer won't satisfy the CEO.  He needs to know if you can do it, but he also needs to know that the answer will be yes.

We still have plenty of work to do.

More to come.

Note:  If your company produces software that you give away to users for free, with revenue funded by advertisers, this blog post might not be relevant.  My hope is that it is relevant to enough people to be interesting.  If that's you, read on ...

A few years ago, before "is dead" was cool, James Bach did a couple of insightful blog posts called Quality is Dead, part one and part two.  It's hard for me to compress such a complex subject into a quick summary line or two, so I encourage you to read the posts -- but in a nutshell, James was concerned that the ephemeral element of 'quality' was getting short shrift in our society.

I am concerned too.

I was concerned, years ago, when we had quality problems with our products, management decided to "invest some developer time".  Specifically, we were to work with tech support.  This does not mean we were bug fixing,  or figuring out what was wrong, just sitting down with Tech Support through a couple of calls, talking about their feelings about the product, building some empathy.

Perhaps that's not the most charitable interpretation of what happened, but it was how I felt at the time.

At the meeting after the sit-downs, our manager told us that we had "improved perceptions."  I replied that it would be nicer if we improved reality, ya know?

Then he said the dreaded words: Perception is reality.

And, in a sense, I suppose, he was right.  The way people perceive the software (or anything) impacts how they see the world.  If they perceive a problem, it is a problem, at least to them.  As far as that goes, I suppose, that's right.

... still ...

Eventually there comes a point where you start to believe the reality you are projecting -- and you run the risk of falling off the cliff you told everyone wasn't there.  When this happens, it may end in a courtroom or on C-SPAN - Enron, Worldcom, and the Tyco scandal all come to mind.  For all of these huge crises that destroyed the economy of entire cities, I suspect they usually started with someone arguing for something a little untrue, claiming "You've got put your best foot forward, after all."

Truth

You can fool yourself into believing the fuzzy front end of the project is done, and even that the coding is done.  It's a little harder these days with modern methods, but if the developers are in on the game, not so much.

Then the tester says it doesn't work.

"Percept your reality" all you want, if you release the thing, and it doesn't work, the music will stop.  The reality distortion field will break ... and your team, your company may find itself having stepped off a cliff.

Sometimes, testers stand in for Truth when no one else will. 

We have to.  We know that if we let this one slide, the Truth will come out, and someone will ask "Why didn't QA find those bugs?"

Other times, the truth can not be heard.  Perhaps the company will go out of business if we fail to ship, or someone further up the chain makes the decision to ship anyway. (Note: If you work on life-critical software, and refuse to accept "ship it anyway", and lose your job over it, look me up.  I'll try to help.)

And, when that happens, you can expect a story like this one.

BATS Markets

BATS Markets is the third largest US commodities trading outlet -- right after The New York Stock Exchange and NASDAQ.  It has planned to begin acting as a real exchange, both offering and trading stock on it's own board. 

The first stock BATS intended to offer was to be it's own Initial Public Offering.

Only when the stock was introduced on March 17th, there was a bug.  The symbol went from $16 to pennies in seconds.  BATS had to stop trading that day; by the end of the following week, the Board of directors removed the CEO of his title as chairman of the board, and was appointing and independent chair.  Forbes has the story and commentary.

A software 'glitch' that destroys your stock, and your exchange, on the day they go live?  That's kind of a big deal.

It's not just BATS

In another FORBES Story, a profile of Amazon, the author points out Amazon's performance metrics, including one that is very illustrative.  It seems that a one-tenth of a second hit to web site performance is correlated to a 1% drop in sales.  Brian Noggle, whose twitter handle is @QAHatesYou, broke the story for the test community and added some analysis on his blog.

It doesn't take a genius to figure this out.  At many companies, many of the ones we work for, as ephemeral as it is, Quality correlates directly to better outcomes and increased revenue.

You do not need ROI numbers to come to this conclusion.

Quality may be struggling, but if we can tell the whole story, I have to say, I think in some sectors, it has a fighting chance.

Oh, and one more thing: Has anybody else noticed that FORBES keeps coming up, again and again, with good coverage and analysis of breaking stories that are relevant to our community?

(Compared to what I was reading from them in 2011, it's a huge improvement.

Real coverage of testing relevant information in the mainstream media, and it's substantially improving over time?

Somebody pinch me.

A few weeks ago, a website you've probably never heard of called CareerBliss.com came up with a "top twenty happiest jobs" in America.  

Software Tester, or, more accurately, "Software Quality Assurance Engineer", took the number one spot.

No, the survey did not come out on April first, and no, I'm pretty sure it's not a joke.

Now you may have heard of this study, but if you have, I doubt it's because of CareerBliss - more likely is i because Forbes.com picked up the story and ran with it, and a large number of influential people read Forbes.

The reaction of the test community was, shall we say ... skeptical.

On an pure emotional level, I must say, I totally agree with the story.

Where else can you get paid to criticize someone else's work?

Where else can you say "it doesn't work", point the finger at someone else, and walk away?  (Not that we actually do this; I try to be helpful.  The point is that you can add value to a project by bringing out your inner critic.)

A handful of people can make a living but being professional movie critics or product reviewers, but for the rest of us, there is software testing.

I honestly don't find this surprising.  Think about what happens when the website is down.  Ops is scrambling to get the site up, developers are running around coding patches and worrying about the build or the merge, and we in test are going back to sleep.  The PM asks you what you are doing, and you get to reply "What?  The site. It's broken.  That is the status.  Do you want me to file a ticket or something?"

I kid, but not entirely.  When I think about the various ways I have had pressure forced on me (often in companies with no test role) and the service I help provide now, helping to create a safety net for the programmers, well, I feel a little pride.

But there's more

The article by Forbes has seen a fair bit of air time, including blog entries and a mention on SQAForums.  (I've never been active on the forums, but I have to say, some of the comments are entertaining, including "I don't know Jim, I think they need to switch to a better grade of crack.")

By friend Scott Barber has what I think of as probably the most insightful blog post of the bunch.

Earlier I wrote on an emotional level -- reacting to the headline.    Scott dug into the detail of the Forbes article and took issue with some of the characterization of what the "QA" (don't start) role is and how it is performed, ending with this comment:

Feel free to share your thoughts, but this strikes me as "not *even* wrong" to a degree that I can't seem to even reverse-engineer a single measurement dysfunction that could account for all the ways in which this article strikes me as "just not right."

He's got a point, and it's worth reading the Forbes article just to compare the status quo with what we testers actually do.  (If you agree with the article, please, leave me a comment.  Let's talk.  I'm sure there are more kinds of testers found under heaven than are dreampt of in my philosophy.)

But there's something else going on here, too.

Where did these numbers come from?

The CareerBliss data evaluates the key factors which affect work happiness, including: one’s relationship with their boss and co-workers, their work environment, job resources, compensation, growth opportunities, company culture, company reputation, their daily tasks, and job control over the work that they do on a daily basis. The data accounts for how an employee values each factor as well as how important that factor is to the employee’s overall happiness. Each review is given an average score indicating where the company places between one and five. All assessments are derived from February 1, 2011 through January 31, 2012. CareerBliss assessed a total of 100,467 employee reviews. A minimum of 50 employee reviews was required to be considered for CareerBliss’ 20 Happiest Jobs in America. Executive level jobs were excluded for this study.

This tells me that they interviewed random people who took surveys on the internet, asked each of them to rate their happiness on ten attributes from, say, one to five, then to rank those interests, and some weighting is given from the most important to the least.

That's great, now show me the data.

If we had the data, we could slice it six ways.  For example, if control over your own daily tasks is the most important you, you could ignore the "happiest" job and look for the one that offered you that single benefit.  Likewise, you could examine the average deviation for each job -- perhaps software QA has a lot of people who are ecstatic about their job, and a lot that are merely happy, and the average is 4.245.  (I would call that a 'biplolar' distribution.'  It also happens in conference talks when your session proposal is too inclusive, and you draw in folks who may be better served attending a different session. But I digress.)

If the data is bipolar, it is possible that outliers are throwing off the numbers.  Teachers, for example, made third on the unhappiest list, with a score of 3.595.  If there ever were a field I would suspect of a bipolar distribution, it would be teachers - there might a whole lot of 4.9's, a whole lot of 2.0's, and the result is the average represents, well ... no one.

The first part of the problem is this idea of an aggregate, of an average response.  With bipolar data, the aggregate is not representative.  

There are ways of dealing with this in the survey world - for example, you might measure the percentage of people who are all 5's across the board - your "true fans", and only count those.

Then again, perhaps that data really is bell-curve, the groupings are tight, and these numbers are representational.  I'm not expecting a scientifically valid survey, with geographic region factored in and margins of accuracy -- but it is possibly the data falls mostly in line.  (When we talk about average salary, though, that kind of information would be nice.)

The thing is, we don't know.  The data is closed; all we get are the averages.

And that is my point today: We need the data.  The raw data, the real thing.

Next time you go to a talk, or read a nice blog post, or check out something on slideshare, ask for the data.  The raw data; probably something in Excel or a database.  Slice and dice it five ways.  Interview the folks who worked on it. Find something insightful.  Take it up a notch.

As for me, I just sent a little email off to the good people at CareerBliss.com.

Wish me luck.


 

Older Posts