I had to skip the previous weekend testing session involving Bing maps (which was a pity – I absolutely adore map applications), but today I was able to participate in EWT03. It was again an interesting experience. Not too many participants this time, which made for a more cozy atmosphere. Today’s testers were Marlena Compton, Anna Baik, Markus Gärtner and myself.
The Mission: You work in a small-medium company, and your manager has been asked to evaluate switching the company over to using Google calendar. He needs a quick assessment from you before his conference call this afternoon. Use the FCC CUTS VIDS touring heuristic to guide you.
We started firing off a bunch of questions at the boss, but he was in a hurry and kind of unresponsive – something about an important golf game he had to attend. He just wanted a quick assessment of the tool, he apparently wasn’t into answering today. So much for questioning – we were on our own pretty soon. I wonder if there’s any way to keep an imaginary boss from leaving a virtual meeting. Still not sure about that.
We decided to divide the coverage since we started late and there was less than half an hour of testing time left. We ended up cutting the FCC CUTS VIDS into – surprise! – FCC, CUTS and VIDS (yes, sometimes you just go for the obvious). I settled for the VIDS:
- Variability tour: look for things you can change in the application – and then you try to change them.
- Interopeability tour: what does this application interact with?
- Data tour: identify the major data elements of the application.
- Structure tour : find everything you can about what comprises the physical product (code, interfaces, hardware, files, etc…)
In the meanwhile the boss had magically reappeared to give us a quarter of an hour extra – he probably forgot his bag of golf clubs. But I quickly realised that time would still be way too short to look at all aspects and concentrated on variability and interoperability – and on a list of existing bugs as well, something I remembered from the first session. In my quest for things that can be changed in the application, I settled for the system date, not really a part of the application but still something that can change while you’re using it. Some interesting bugs ensued, including a very severe one in Skype group chat – not really part of the mission. I thought I had lost connection but it turned out that the order of the chat messages was totally messed up. It took a while before I realised what had happened and lost a good deal of time because of that. Later on, I switched to interoperability but I also lost my internet connection for for a while, which was annoying but in retrospect also pointed me to a synchronisation bug.
The debriefing, moderated by Anna, revealed some good points from the other participants. Marlena covered the FCC (feature, complexity, claims) part and did a feature tour first, which proved very helpful. She explored all functionalities to get more familiar with them before she went off to investigate other areas. She wondered how the rest of us could start testing without doing such a feature tour/exploration first. But in fact, I think we all started with a bit of general exploration. I know I did. It feels kind of natural, probing around the surface for a while before before diving into the abyss of detail. Concerning the splitting up of the heuristic among several testers, this turned out to be a bit distracting, confusing and counter-productive. There is quite some overlap between the different areas and sticking to only some of them might mean that you’re missing out on otherwise obvious problems. This led us to the use of sapience in testing, being able to picking the heuristics that are useful for the given features.
We didn’t get to brief the boss in the end, so you could say that our mission didn’t really succeed. But hey, some say that failure breeds success. Soon, things will be looking so bright we’ll have to wear shades.
Test Side Story 