Last year, James Lyndsay introduced me to his “Nr1 diagram of testing” (© Workroom Productions): a deceivingly simple model that tries to capture the essence of testing.
The circle on the left represents our expectations – all the things we expect. This is the area where scripted tests or checklists come into play to tell us about the value that is present. The right hand circle represents the actual deliverable – what the software really does. This never completely matches what was originally expected, so this is where we use exploratory testing to find out about risk.
The diagram divides the world into four distinct regions:
- The overlap. These are the things we both expected and got.
- The region outside the circles. That is all we didn’t want, and didn’t receive in the end. A pretty infinite set, that is.
- The left-hand arc. This is what we expected to get, but didn’t receive. This means that the deliverable turned out less valuable than we had hoped.
- The right-hand arc. The software system under test does all kinds of things things we didn’t expect. There’s clearly some risk involved, here, and it’s up to us to discover just how much risk there is.
Simplicity is of course the main strength of such a model. It can certainly help to identify or classify things in our quest to quickly grasp the essence of things.
These four regions got me thinking. I’d like to expand on that. What about things that we expected and received, but do not prove value, for instance? Unneeded features – not too unrealistic. Or – even better – what about things that were not expected, but are there and actually do provide value in the end? Immediately the term “Collateral features” came to mind: no matter how hard we try to create designs for certain uses, people will always utilize them in their own way. These unintended uses can be strange sometimes, but some them are downright brilliant.
Take a look at Alec Brownstein. While most people use Google AdWords to promote their business (after all, that’s what it was designed for), Alec used it to get a job. He was trying to land a job as a copywriter with some of the top ad agencies in New York City. He assumed that the creative directors at these top agencies would “vanity google” their own name. So he bought AdWords of the names and put a special message in to each of them. The rest is history. He now works for Y&R, after a total investment of $6 (story here).
Collateral features also emerged in the microblogging world. Because Twitter provided no easy way to group tweets or add extra data, the twitter community came up with their own way: hashtags. A hashtag is similar to other web tags- it helps add tweets to a category. Hashtags weren’t an official feature, but they sure made their way into the daily twitter work lexicon of billions of people.
In an article in Forbes magazine called You Can’t Predict Who Will Change The World, Nassim Nicholas Taleb pointed out that things are all too often discovered by accident—but we don’t see that when we look at history in our rear-view mirrors. The technologies that run the world today (like the Internet, the computer and the laser) are not used in the way intended by those who invented them.
There will always be unforeseen usage of your software. Some prove risky, others do contain value. Some of these collateral features even replace the intended use to become main features and make their way in the ‘expected’ circle. Ultimately, your customers make the final call. They decide how to use your product or service. Not you, not your marketeers.
– “But no user would ever do that!”
– “Fair enough. Wanna bet?”
Excellent post. I’d not come across that diagram before, but it neatly illustrates the point. I’m fascinated by that right-hand arc, the unintended functionality. It can open up all sorts of exposures, and some potential benefits too. Conventional software testing has focussed on the left hand circle. In particular script based testing from the documented requirements is poorly equipped to uncover the collateral features, or latent functionality.
I wrote an article for Testing Experience about testing for latent functionality
http://clarotesting.com/page15.htm
I also blogged about how usability testing techniques can help.
http://clarotesting.wordpress.com/2010/03/27/usability-and-security/
Hi James, thanks for the comment. There’s a lot of stuff (good & bad) hidden in that right-hand arc, fascinating indeed. Interesting articles you got there, very thorough as well.
“Collateral features” – nice term! I’ll remember it the next time developers hand over a build with unexpected features.
Developers seem to enjoy this. And while very occasionally they end up useful, the negatives almost always outweigh the positives:
– unplanned, unscheduled features that must be tested, debugged, fixed
– never any documentation for these types of features
– the time spent on these “fun” features, is always at the expense of something else
– what seems like creative inspiration for the developer can often end up as a support nightmare
– it’s always interesting to hear the developers response to why they haven’t finished the expected features, but yet had time to add in unexpected features
Whenever I see a new feature magically appear, I write a bug report. During bug triage the developer can try to justify the feature if she truly believes it is worthwhile. Otherwise, it gets removed.
I use a three-circle diagram consisting of Described, Actual, and Imagined. Described and Imagined are both aspects of Expected but separating them is important, because it highlights the big problem we have with what’s in our head vs what we talk about and show each other.
Hi James, that model sounds interesting – do you have any articles/blog posts on that?
Indeed interesting model. But as you described how about unintended functionality that actually provides value?
But this model is not one that can help practically. Its more like representation of something abstract.
And for someone who has experience his own model will be applied.
Anyway intriguing 🙂
Sebi
http://www.testalways.com
Hi Eusebio. The best thing about these models is that they make people think & talk 🙂