Tag: Workshop

What happened at DEWT1 doesn’t just stay at DEWT1 (June 11, 2011)

A report on the first DEWT (Dutch Exploratory Workshop on Testing) on May 11, 2011 in Driebergen, NL

What started on twitter in november last year, culminated in a first major milestone last weekend: DEWT1, our first peer – and Exploratory – Workshop on Testing (yes, the D is for Dutch, but these Dutchmen happily accepted this Belgian foreign element in their midst). Michael Bolton added to the international character by agreeing to be our special guest for the weekend.

It turned out to be an inspiring and fun event. Here’s my write-up.

The venue

Hotel Bergsebossen, Driebergen, NL

The participants

People on DEWT-y, from left to right:

Jeroen Rosink, Ray Oei, Jeanne Hofmans, Michel Kraaij, Huib Schoots, Jean-Paul Varwijk, Ruud Cox, Zeger Van Hese, Michael Bolton

Peter “Simon” Schrijver (who was roaming the earth the Better Software conference at the time)  and Anna Danchenko could not attend

The pre-conference

We gathered on friday night as a warm-up to the conference. When Michael Bolton is around, this usually means getting lured into some tricky testing puzzles, and some beers to ease the pain of messing up. And yes, jokes too. And Talisker. After we discovered the versatility of the average Dutch hotel bouncer (half bouncer, half God ad-hoc bartender), we called it a night. A dream-ridden night it was, filled with newly learned terms, such as…

Shanghai (transitive verb) \ˈshaŋ-ˌhī, shaŋ-ˈhī\ (shanghaied / shanghaiing)

1 a : to put aboard a ship by force often with the help of liquor or a drug b : to put by force or threat of force into or as if into a place of detention

2 : to put by trickery into an undesirable position

The conference

Artful Testing

Speaking of which… during our last preparatory DEWT-meet-up, my fellow DEWTees shanghaied me into doing the first talk of the day, which they promptly called a keynote to make it sound like an invitation. I thankfully accepted though, since I wanted to get some feedback on my work-in-progress presentation. The link between art and testing has been consuming me for more than half a year now. I premiered my ideas on it at the second Writing About Testing (WAT) conference in Durango last month (if you haven’t done it already, you should check-out the great WAT write-ups from Marlena Compton, Alan Page and Markus Gärtner).

Ruud (who facilitated the morning sessions) kicked off the conference and invited me to take the proverbial stage. Based on the feedback from WAT, I made some modifications to the presentation and put it out here again for a second time. I don’t know if the subject was really fit for an early morning session, but I received some gratifying feedback that convinced me to pursue my efforts in this direction.

Transpections

Transpections (basically a way of learning and sharpening your ideas by putting yourself in someone else’s place in some kind of Socratic dialog) were on our DEWT wish list for quite some time already. We had been reading all sorts of interesting stuff on it (see James Bach’s post here, some Michael Bolton posts here and here, and Stephen J. Hill’s post here), so we asked Michael Bolton if he would be willing to give us a quick roundup on the subject. Michael agreed and made it into an interactive session, inviting us to pair up to gather information about transpections and then transpect on that. Meta-transpection for the win!

The information gathering exercise was enlightening, and brought up some good food for thought. Michael compared a transpection session with the play between a hammer and an anvil, where the hammer would be the initiator of the transpection, the anvil the person whom the initiator is transpecting with, and the metal the idea being shaped.

In the end, we didn’t get to try an actual transpection session, partly because I artfully exceeded my allotted time in the previous session. Oh well…  It was a valuable exercise nonetheless.

Lightning talks

After lunch there were some lightning talks to fight the afternoon dip:

  • Jeroen got started about the hierarchic “testing pyramid” model (testers / test coordinators / test managers) and how he wants to challenge that classical view
  • Huib followed, on “the power of knowing nothing”, about how starting with a (mentally) clean slate reduces the chances of being biased. “It’s not about the what, it’s about the “why”
  • I touched upon the topic of the Baader Meinhof phenomenon and how testers could leverage the effect by absorbing as much knowledge as possible, on several subjects (a blog about that has been sitting in my drafts since january 2010 – I’ll try to finish that)

Introducing exploratory testing in Dutch projects

Ray then presented an experience report on how he was able to introduce exploratory testing and session based test management in classic, T-Map-style projects, using the principles he learned from Rapid Software Testing. Discussion ensued on how to prove the benefits of RST, and what the major differences between the approaches are. But we ended up talking mostly about “release advice”, and what to do when you’re asked to give it. One take-away phrase for me: “it’s not declining, it’s empowering the product manager”.

Walking break & Positive Deviance

Although we finished the previous topic way ahead of schedule, everyone felt like the last discussion drained our energy (our staying up late the night before probably didn’t help either). Jeanne, who facilitated the afternoon sessions, had the brilliant idea to just go out for a walk in the “Utrechtse Heuvelrug” national park, which turned out to be a conference session in its own right: relaxing, fun and informative. A beautiful spot, too. There was a moment where I thought we were getting lost, but here’s another lesson: do not underestimate the power of nine explorers, without a map.

Back at the hotel, Michael talked about positive deviance and positive deviants (people whose uncommon but successful behaviors or strategies enable them to find better solutions to a problem than their peers, despite having no special resources or knowledge). He also showed us a video of Jasper Palmer, a patient transporter at the Albert Einstein hospital (and a positive deviant) who became famous for his “Palmer Method”, which is now a standard life-saving practice in a number of hospitals. A mighty fascinating topic, that I’ll be exploring more for sure.

Credibility

Ruud delivered the closing presentation, on credibility – the quality of being trusted and believed. The main issues Ruud addressed were: how do we – testers – build credibility, and how do we manage to maintain it? After all, trust is built slowly, but destroyed in seconds. Simple questions, but a very complex subject indeed. “Trust” and “credibility” are relations: you can be credible to some person at a certain moment in time, but totally incredible to another. Trying to build your credibility is not always something controllable. Sure, you can do your very best to improve your credibility on a personal level, but you don’t really have an influence on how people will perceive you. Ruud then explained how he tries to build credibility. He impressed me with the personal mnemonic he developed, and the matching artwork as a personal reminder to stick to these principles:

STYLE

  • Safety language
  • Two ears one mouth
  • Yes but
  • Lighten up a little
  • Empathy
I’m not going in detail here, because I specifically want Ruud to finish that blog post he’s been mulling over for ages now. So, yes Ruud, the pressure is on. You’ve got some great material – time to share it with the world!

DEWT1 ended with drinks, testing games and dinner. I ended the day way more energized than I started it, which is always a good sign (silly extroverts like me get fueled by events like this). DEWT1 rocked. It was informal, informative and entertaining. When is the next?

Advertisement

The winner takes it all (a 3-day workshop with Cem Kaner)

A review of the Cem Kaner workshop during the Cem Kaner Week, held in the last week of October 2009 in Nieuwegein, The Netherlands.

Last August I received a mail from a company called Immune-IT that promoted their upcoming “Cem Kaner week“. The concept: during the last week of october Cem Kaner would visit The Netherlands to lead a three day workshop and give a freely accessible presentation as well. Considering the cost of the workshop I settled for the free presentation, which I figured was also quite unique. Nieuwegein in The Netherlands is only a 160 km drive from my home (ok, you’d have to get past Antwerp, Breda and the Utrecht traffic jams, but still: worth the hassle). Later I heard that there was also a possibility to get a free workshop place by entering a contest on their website. Long story short: I participated, won the contest, checked the mail again (still said I won it) and jumped from joy. On October 26, I had a date with testing history.

I have always looked up to Cem Kaner, admired him for being one of the pioneers in the software testing field. He is a professor of computer sciences at Florida Institute of Technology and the (co-)author of truly great books like “Testing Computer Software” (alledgedly the best selling software testing book of all time) and “Lessons learned in Software Testing” (which inspired and influenced me big time). He founded the “context-driven school of software testing” (I’m on their side) and has all these excellent free courses available online for Black Box Software Testing. All this was rushing through my head while cruising through the lowlands. Later on, traffic came to a standstill, and this gave me even more time to get my imagination running… What if I don’t like him? What if he turns out to be a belligerent ghoul (gratuitous “The Smiths“-reference)? Well, none of all that. Cem is a nice person, almost humble in his ways. I like him. A friendly, human encyclopedia of black box software testing.

The size of the workshop was pretty small – only 12 people enrolled, which made it super-interactive. Immune IT had asked Cem to talk specifically about testing in difficult economic times. He did talk about that, briefly, but luckily he covered a whole range of other things as well. Some highlights:

  • Exploratory checklists and a truly hilarious piece of scientific proof that following scripts is a best practise (yes, you heard it right, in some cases there *are* best practises) for brain-damaged rats. In 1977, Cem incidentally stumbled upon proof that normal lab rats use ‘checklists’ to drive their behaviour, while brain-damaged ones resort to following the same script over and over, no matter the context. More on this can be found here.
  • Scripting and learning. Cem debunked some common myths surrounding scripted tests.
    #1. “Test scripts are ideal as ‘training wheels’ for new testers. After several months of following a wide range of scripts, the new tester will have learned by example a lot about the application domain, the program and how to test it”. So they say. But have you ever noticed what happens when you drive to a new place with a GPS that gives you a script like “Turn left at the next lights etc…?”. If you try to go there again, do you actually remember the route? You learn faster when exploring and discovering, rather than by ‘being told’.
    #2. “Test scripts specify all initial entry conditions”. But you cannot capture everything, there are so many variables that it becomes impossible (system state, program state, system config, system resources, other processes) to describe this upfront.
    #3.  “Test scripts specify the expected results”. But – same as above – scripts cannot specify all possible outcomes. There are simply too many variables involved. Our tests cannot address all possibilities.
    #4.  “Test scripts involve a comparison that machines or humans should make”. But what about confirmation bias? Or inattentional blindness? We ignore things based on their meaning, before we ever become aware of them.
  • An extensive part about exploratory testing from the godfather himself, it doesn’t get any better than that. He explained the renewed definition of exploratory testing (which I endorse but find a bit too long – I liked the old one better).  *taking a big breath* here goes:

Exploratory Software Testing:
– is a style of software testing
– that emphasizes the personal freedom and responsibility
– of the individual tester
– to continually optimize the value of her work
– by treating
     * test-related learning,
     * test design,
     * test execution, and
     * test result interpretation
– as mutually supportive activities
– that run in parallel throughout the project

  • An intriguing part on testing an investment modeling tool called VectorVest.
     
  • “Exploratory Test Automation”. Slides available here. I specifically liked his experiences at a phone manufacturing company called Telenova.  They were able to track down a stack overflow bug in one of their telephone sets that had been undetected even though testing achieved 100% statement and branch coverage in the relevant parts of the code. In the field, this would require a long sequence of calls to a continuously-active phone. They succeeded in isolating it through “Exploratory Test Automation”: they created a simulator that generated long chains of random events, emulating input to the system’s 100 phones. That eventually exposed the bug (and several others as well).
  • Cem is obviously not a great fan of automated tests, but he makes an exception for “Exploratory Test Automation” , “High-volume Test Automation” and “Extended Random Regression Testing”. Traditional test techniques tie us to a small number of tests. Extended random regression and long simulations exposes bugs the traditional techniques probably won’t find.

These three days were a real treat. I was able to talk and discuss with Cem Kaner personally – and learn from him, which was priceless.  I also met some other great people – passionate about testing and always prepared for an healthy argument. At the end Cem signed the copies of “Lessons learned in software testing” that we all got for free, which means I now got two of them. One to use and one to cherish.