Innovate & Renovate: Evolving Testing

Test Side Sorry

I know it’s been quiet here lately. A big Test Side Sorry for that. A lot has happened the last months, and many things occupy my mind and time.

Eurostar 2011

One of these is Eurostar, Europe’s largest testing conference which took place in Manchester in november. I had a great time meeting new people and catching up with others. Finally meeting people from twitter in person is one of the best side-effects from conferences I know. It feels like meeting old acquaintances, in a way. 

The test lab vibe was great, as always. I saw some great track sessions, and there were loads of things happening behind the scenes as well: video tapings, special and fun sessions that will be published the coming months. I even spotted a smoke machine or two on a Thin Lizzy soundtrack. The conferring didn’t stop at 6 pm – Manchester pubs were test-infected for a while.

At the gala awards dinner, in the magnificent setting of the Manchester Monastery, Julian Harty received the Testing Excellence award. Geoff Thompson also revealed next year’s programme chair.

Eurostar 2012

Rewind three weeks. Lorraine – Eurostar’s conference manager – calls me during my daily commute to inquire if I would be interested in becoming the programme chair for 2012. I barely manage to steer clear of a ditch. And I apologize to that old lady I almost ran into. Back to the call. I hesitate at first, ask some time to think it through. Seconds later I realize: “wait a minute, what was I thinking? Of course I’ll do it! Yes, there will be work involved. But it’s all good.

Eurostar will turn twenty in november 2012. I will be the host of this special edition, taking place from november 5-8 in Amsterdam, the Netherlands. Twenty years, that is quite something. No longer a teenager, and old enough to party.

My partners in crime for this conference are James Lyndsay, Julian Harty and Shmuel Gershon, and I can’t stress enough how honored I feel that they accepted to be on the team. Our first task was to come up with a theme. Since our aim is to craft a learning conference, focusing on innovation, renovation and creativity in testing, we decided on:

Innovate & Renovate: Evolving Testing

The call for submissions is now open, by the way. I look forward to receiving your ideas.

Home Swede Home – Øredev 2011

Last week I attended (and presented at) the Øredev conference in Malmö. Sigge Birgisson invited me to be part of a fully Context-driven test track, which I gratefully accepted. It turned out to be quite a memorable experience. Øredev was the first ever *developer* conference (be it with a testing twist) I attended, which gave the event a totally different vibe for me. Cosy, laid back and open-minded. Geeky too, in a good way:  they provided a cool conference app with a puzzle that could only be solved by obtaining other people’s codes. The side effect of that was that random people started addressing me with “Hi. Can I have your code?” moments before bolting off in their own space-time continuum. Speed dating for techies.

Another thing that really stood out were the graphic live-recordings by Heather and Nora from Imagethink. These talented ladies recorded every keynote live on stage, and made the beautifully looking artworks available as handouts later on. A brilliant idea.

As for the proceedings of the conference – here are some personal highlights:

Day 1

Day 1 had no real testing track, but there was enough fun to be had in other areas of the development spectrum. As the conference was centered around “the user” (Enter Userverse), it kicked off with “Only your mom wants to use your website”, an entertaining keynote by Alexis Ohanian, of Reddit and Hipmunk fame. Hey, the guy even spoke at TED about a whale called Mister Splashy Pants – top that! This time he told a compelling story about how the secret behind succesful websites is caring for your users. He told us that generally, the bar on websites is raised so low that it is really easy to stand out if you’re able to delight your user.

In “Collaboration by better understanding yourself”, Pat Kua stated that people have lots of in built reactions that hold us back from collaborating more effectively: power distance, physical distance, titles, even clothes. What could help us? Awareness, feedback, breaking the cycle, XP practices, courage. A good talk with good content, and some good book recommendations as well.

Johanna Rothman managed to keep me engaged for her whole talk about “Managing for collaboration”. She talked about how to manage the entire system for success, and how we should optimize and collaborate on the highest level, solving problems for the entire organization, not the project. I had the privilige of getting to know Johanna in her may 2011 PSL (Problem Solving Leadership) class, which she organizes together with Esther Derby and Jerry Weinberg. I knew she was a great storyteller, and she did not let us down: one gem was how she upset management by donating her entire bonus to her team and letting them decide who got what. 

Neal Ford closed off the day conference with “Abstraction distractions”, in which he dissected abstractions that have become so common that we started mistaking them for the real thing. An abstraction is a simplification of something much more complicated that is going on under the covers. As it turns out, a lot of computer programming consists of building abstractions. A file system, for instance, is a way to pretend that a hard drive isn’t really a bunch of spinning magnetic platters that can store bits at certain locations, but rather a hierarchical system of folders. And what’s that icon on a save button again? A floppy what? In addition, we shouldn’t name things that expose the underlying details. Users really don’t want save buttons, they just want their stuff to be saved. He also quoted Joel Spolsky’s Law of Leaky Abstractions: All non-trivial abstractions, to some degree, are leaky.

The day ended with drinks, dinner and some live jazz. I ended up talking testing (among other things) with Pradeep Soundarajan over dinner, when suddenly a late night evening session was announced: Copenhagen Suborbitals. At that moment, it reeked of a mediocre techno-act from the late nineties and I didn’t really feel like joining in. But Pradeep was curious enough and I decided to tag along. 

Flash forward one hour. Pradeep and I were literally blown away by a passionate tale of two Danes with a dream to build and launch their own manned rocket into space. Peter Madsen told a compelling and inspiring story about dreams, constraints, possibilities, enthusiasm, courage and rocket fuel.

Day 2

Day two was kicked off by Dan North who talked about “embracing uncertainty”. Fear – he said – leads to risk, risk leads to process, process leads to hate… and suffering and Gantt charts. Dan stressed that people would rather be wrong than uncertain, and that adding more process in times of uncertainty is wasteful and counter-productive. He also contrasted the original intentions of the agile manifesto in 2001, and what has become of that now. He stated that our ability to survive is directly related to handling the unexpected. We should embrace uncertainty, expect the unexpected and anticipate ignorance.

I decided to put up my basecamp in the “Test” room today, since this was context-driven testing day: six testing tracks covering a wide variety of topics. The only drawback was that the room looked like it was designed by an architect on acid: unfinished, an enigmatic door way up high in a wall, bare cables and sockets and a very short and high stage that forced you either to stand in front of the projection screen or to stay cemented in the same spot the whole time. Sound isolation was kind of peculiar too, although that only seemed to be  a problem when Americans were presenting nextdoors. But I’m nitpicking here: the whole Slagthuset venue was nice, and organization and technical team were super helpful, the whole day.

Pradeep Soundararajan‘s talk was titled “How I wish users knew how I help them through context driven testing”. Pradeep started by pointing out that he had the shortest abstract and the longest bio in the conference booklet. True. He seems to like long titles for his talks, too. In combination with his name, this probably makes him a nightmare to introduce at conferences. But in contrast with the title, his talk was short, crisp and funny. He was brave enough to do some live-demoing of his twitter-driven exploratory testing approach: looking for user feedback by searching in tweets with negative emoticons and profanities combined with the product or website name. I hadn’t read his blogpost before now, and it made me laugh out loud. I love the smell of profanities in the morning. Brilliant idea, that.

Next up was Shmuel Gershon, who shared an experience report of a 100% exploratory testing project, “Case Study on Team Leadership with Context-Driven Exploratory Tests”. He came well-prepared, all set to win our hearts with charisma, handouts and chocolats. He told us about how he took his team on a journey towards more context-driven testing and how he dealt with that as his role was also changing. He told us a story on test management, session based testing, recruiting even. He urged us to let people tell their stories, don’t start asking why, leaving them feeling that they have to justify themselves.

The ubiquitous Gojko Adzic (I suspect there are several clones making the rounds of conferences worldwide. Where /doesn’t/ he speak?) was his energetic self in his graveyard shift session called “Sleeping with the enemy”. Independent testing, he said, should be a thing from the past. Testers should engage with developers and business users, in order to create opportunities to accomplish things they cannot do otherwise. I like Gojko’s style, always direct and uncompromising, but always thoughtful. After Gojko’s presentation, a heated hallway discussion ensued in the so-called chalk-talk area. This embodies what conferences are all about: conferring. 

With “Diversity in team composition”, Henrik Andersson took the small stage trying to convince us that when assembling good teams, diversity rocks and uniformity, well, not so much. With some simple examples (“can I please ask everyone wearing black clothes to stand up. You are now a team”), he showed us that there’s much more to it than randomly throwing some people together.

Then it was Selena Delesie‘s turn to shine in the beamer lights. In “Focusing Testing on Business Needs”, she explained how to focus the testing effort on customer needs. She asked some pertinent questions; Are you valued in your team? How do you know?

The last presentation slot of the day in the testing track was for yours truly. In Artful Testing, I talked about how I think testing can benefit from the arts. From thoughtfully looking at it, to develop our thinking. From critical theory and the tools used by art critics, to become software critics. From artists, and how they look at the world – through artist personas). I also touched on the importance of context in evaluating art and software. I received some great reactions and feedback afterwards, and some good tips from Pradeep and Rikard as well.

After that, there was dinner, drinks and Øredev Open, where Pradeep was invited to present “The next generation software tester”. In theory. But you know how these things go. In theory, theory and practice are the same; in practice they are not: dinner took a bit longer than expected, drinks were abundant and so it happened that Pradeep took the stage for some Beer-Driven Exploratory Presenting. It was a great stand-up routine.

Ola Hylten joined in and Shmuel decided to whip out his box with tester games and puzzles. Time for some serious thinking, mixed with laughs. When the Øredev Open closed, we took ourselves and our silly games to the hotel bar where innocent passers-by quickened their pace.

Day 3

Day 3 in the test track started with “Agile testing: advanced topics” by Janet Gregory who highlighted five topics that had emerged since the release of “Agile Testing” by Lisa Crispin and herself. She mentioned feature acceptance (when you’re not able to deliver everything, focus on the features that matter), collaborative automation, large organizations, distributed teams, continuous learning.

Next up was my favorite Swedish philosopher (granted, I only know one), Rikard Edgren, who delivered did a spot-on and thought-provoking session called “Curing Our Binary Disease”. He stated that software testing is suffering from a binary disease: pass/fail addiction, coverage obsession, metrics tumor and sick test design techniques (sick as in “ill”, not “wicked” – my interpretation). Couldn’t agree more. He also mentioned his infamous “software potato”, which made for following legendary phrase: “A tester might not even know that he’s in the potato”. 

All this binary goodness got me thinking: Stay/Go? Focus/Defocus? Defocus it was. I chose to do a final round of the expo and do a quick Copenhagen visit to get some fresh air while it was still light out.

That concluded Øredev 2011. It was great to finally meet Selena, Sigge and Pradeep. And Robert Bergqvist as well. It was great catching up with others (Johanna, Shmuel, Henrik, Janet, David, Ola, Rikard,…). Next up: Eurostar in Manchester next week. A full-blooded tester conference that will rock as well. Let’s meet there.

Finding Porcini

A couple of weeks ago I found myself in southwestern France, a region which – at the time – was being struck by a unseen spell of global wetting. Summer had arrived three months early, people said. April and may had been exceptionally dry and warm; but in July, autumn was knocking on the door of our vacation home. Early autumn, I was told by our gentle host Philippe, goes hand in hand with a peculiar phenomenon: fungi frenzy/mushroom madness. All the locals get this strange misty-eyed look and head for the woods to hunt for the precious boletus edulis, more commonly known as  fungo porcino, porcine mushroom, cep and lovingly referred to as “the brown plague”, as it tends to halt the local economy.

When Philippe invited me to join him on an early morning quest for porcini, I gladly accepted. It sounded like a treasure hunt, as fresh porcini are sold for outrageous prices to local restaurants. And who doesn’t enjoy a good treasure hunt after breakfast? We left for the woods, armed only with wooden baskets, a knife and a sturdy 4×4.

Mushroom hunting, I found out quickly, is an art in its own right. Slowly and carefully, like an old sensei, Philippe unveiled his mushroom hunting mysteries. And as the mysteries disappeared, a neat set of categorized heuristics came trickling through:

The Mission

  • “Ready Zeger? So, we’re looking for cèpes, têtes de negres and chanterelles. Leave the others be. Some are poisonous, others just don’t taste nice”
  • “When we stop? When we’re finding more mushrooms than we can carry. Or when we’re not finding anything anymore.”

Classification

  • “Wait, you don’t know what to look for, do you? Come over here for a sec. This is a vintage cèpe. A fungho porcino. Like all other boletes, it has tubes extending downward from the underside of the cap, rather than gills. The pore surface of fruit body is whitish when young, but ages to a greenish-yellow when older.”
  • “Be careful there. Some mushrooms look very much like porcini. You should look under the hood. If it’s yellow, don’t touch ’em.”
  • “When you’re not really sure, scratch the bottom of the hood. When the scratch turns purple, don’t touch.”

Timing

  • “Why we’re heading out all of a sudden? Porcini tend to appear after summer peaks, depending on the weather. Usually they pop up about a week after a wet spell.”
  • “Oh no, that’s not true, Zeger. The fact that we picked loads of mushrooms here doesn’t mean I’m not coming back here tomorrow. These things grow fast. They often push overnight, you know.”

Location

  • “Location is everything. You should look at open spots in the woods, where the sun can actually reach the ground. Look, that should be a good place over there. Do you see the sunbeams peaking through the leafs? Let’s head over there.”
  • “When you find one, mind your step. Where there is one, there are many. You might crush some perfectly good ‘shrooms hidden under some leaves or grass.”
  • “Spotting porcini takes a trained and experienced eye. Here’s a pro-tip: look for where the leaves bunch up – perhaps they are being pushed up by a growing mushroom.”
  • “Don’t spend your time looking near ferns, man. Ferns grow on intensely acid soil, porcini don’t”
  • “Hey Zeger! You see this black beauty here? This is a tête de nègre, a particularly tasty and expensive kind of cep. Look for them near oak trees.”
  • “This here’s a chanterelle. If you spot two of them, follow the line that connects them; they always grow on a straight line.”

Picking technique

  • “Use a knife. Never ever pull mushrooms out of the ground. Cut them. If you damage their mycelium, they won’t grow again next year.”
  • “Remember: be gentle, cut them near the bottom of the stem in a straight line. Don’t break the hood.”
  • “Wait! Don’t cut the really small ones – they’ll be worth much more later on.”

I was soaking with sweat after a couple of hours of intense scouting. In a short timespan, Philippe managed to transform me into a die-hard mushroom hunter. A novice still, but I felt I was learning quickly. Philippe’s heuristics (not best practices, mind you) helped me discern the good from the bad, finding porcini hidden under leaves and cantharelles in a neat straight line. I even developed my own heuristics as I went along: I started looking alongside paths through the woods – plenty of chances for the sun to peep through the deck of leaves, and easier to spot since the vegetation is less dense.

Testing porcini

As I was wandering through the woods with eagle eyes and at a snail’s pace, it all felt strangely familiar. When Philippe said “Where there is one, there are many”, it struck my tester chord. Here I am, a tester, looking for mushrooms, which doesn’t seem to be all that different than looking for bugs. No wonder I liked it so much. I also realized that when I’m looking for bugs, I use these kind of heuristics all the time, but all too often I’m not very aware of them. Which is a pity, because used consciously, these heuristics (“a fallible method for solving a problem”) can be a really powerful tool to boost your exploratory testing efforts.

  • Start with a mission – make sure you – and your team – know what to look for, since our conception limits our perception. Michael Bolton often quotes Marshall McLuhan on this: “I wouldn’t have seen it if I hadn’t believed it”
  • Make sure you’ve got your classification right. If you’re only interested in a specific kind of bug, maybe you shouldn’t waste time reporting others. You could consider parking them somewhere, or keeping the reporting rather lightweight by MIPping (mention in passing) them. But try to stick to your main focus for the session. And if you find a nice-looking bug, is it really? Scratch it, it might turn purple
  • Timing – as in mushroom picking – is also a factor to be considered in bug hunting. Are there typical times at which the application is less stable? When is an ideal testing time, really? Again, this is largely dependent on context
  • Location, location, location. Personally, I use many heuristics to guide me where to test. Which areas are more vulnerable? When you find one, there tend to be many others, indeed. As leaves bunching up *might* indicate a pushing mushroom, seemingly insignificant facts might be a tell-tale sign for bugs nearby: the code that developers write after a wild night of partying might not be all that good, for example. Or they can just have a bad day. I was once told by an old native American medicine man that developers are human too
  • As some mushrooms are picked and not cut, our bag-o-techniques should enable us to deal with any situation. As Lee Copeland points out in A Practitioner’s Guide to Software Test Design: a tester should carry his techniques with him at all times, just like a handyman’s toolbox follows him around everywhere he goes. Apply a specific technique, use an particular approach when the situation calls for it.

For the record: I’m not a mushroom master, yet. I lack practice, experience and domain knowledge to attain mastery. I’m not a testing master either, as I’m in constant learning mode. For every good practice I know, in context, I am aware that there’s always another context that I need to get myself familiar with. That prospect may seem humbling and daunting to many, but I wouldn’t want it any other way. That’s Context-Driven Testing for ya.

(For more info, see The Seven Basic Principles of the Context-Driven School as a starting place. There’s a lot more where that came from).

What happened at DEWT1 doesn’t just stay at DEWT1 (June 11, 2011)

A report on the first DEWT (Dutch Exploratory Workshop on Testing) on May 11, 2011 in Driebergen, NL

What started on twitter in november last year, culminated in a first major milestone last weekend: DEWT1, our first peer – and Exploratory – Workshop on Testing (yes, the D is for Dutch, but these Dutchmen happily accepted this Belgian foreign element in their midst). Michael Bolton added to the international character by agreeing to be our special guest for the weekend.

It turned out to be an inspiring and fun event. Here’s my write-up.

The venue

Hotel Bergsebossen, Driebergen, NL

The participants

People on DEWT-y, from left to right:

Jeroen Rosink, Ray Oei, Jeanne Hofmans, Michel Kraaij, Huib Schoots, Jean-Paul Varwijk, Ruud Cox, Zeger Van Hese, Michael Bolton

Peter “Simon” Schrijver (who was roaming the earth the Better Software conference at the time)  and Anna Danchenko could not attend

The pre-conference

We gathered on friday night as a warm-up to the conference. When Michael Bolton is around, this usually means getting lured into some tricky testing puzzles, and some beers to ease the pain of messing up. And yes, jokes too. And Talisker. After we discovered the versatility of the average Dutch hotel bouncer (half bouncer, half God ad-hoc bartender), we called it a night. A dream-ridden night it was, filled with newly learned terms, such as…

Shanghai (transitive verb) \ˈshaŋ-ˌhī, shaŋ-ˈhī\ (shanghaied / shanghaiing)

1 a : to put aboard a ship by force often with the help of liquor or a drug b : to put by force or threat of force into or as if into a place of detention

2 : to put by trickery into an undesirable position

The conference

Artful Testing

Speaking of which… during our last preparatory DEWT-meet-up, my fellow DEWTees shanghaied me into doing the first talk of the day, which they promptly called a keynote to make it sound like an invitation. I thankfully accepted though, since I wanted to get some feedback on my work-in-progress presentation. The link between art and testing has been consuming me for more than half a year now. I premiered my ideas on it at the second Writing About Testing (WAT) conference in Durango last month (if you haven’t done it already, you should check-out the great WAT write-ups from Marlena Compton, Alan Page and Markus Gärtner).

Ruud (who facilitated the morning sessions) kicked off the conference and invited me to take the proverbial stage. Based on the feedback from WAT, I made some modifications to the presentation and put it out here again for a second time. I don’t know if the subject was really fit for an early morning session, but I received some gratifying feedback that convinced me to pursue my efforts in this direction.

Transpections

Transpections (basically a way of learning and sharpening your ideas by putting yourself in someone else’s place in some kind of Socratic dialog) were on our DEWT wish list for quite some time already. We had been reading all sorts of interesting stuff on it (see James Bach’s post here, some Michael Bolton posts here and here, and Stephen J. Hill’s post here), so we asked Michael Bolton if he would be willing to give us a quick roundup on the subject. Michael agreed and made it into an interactive session, inviting us to pair up to gather information about transpections and then transpect on that. Meta-transpection for the win!

The information gathering exercise was enlightening, and brought up some good food for thought. Michael compared a transpection session with the play between a hammer and an anvil, where the hammer would be the initiator of the transpection, the anvil the person whom the initiator is transpecting with, and the metal the idea being shaped.

In the end, we didn’t get to try an actual transpection session, partly because I artfully exceeded my allotted time in the previous session. Oh well…  It was a valuable exercise nonetheless.

Lightning talks

After lunch there were some lightning talks to fight the afternoon dip:

  • Jeroen got started about the hierarchic “testing pyramid” model (testers / test coordinators / test managers) and how he wants to challenge that classical view
  • Huib followed, on “the power of knowing nothing”, about how starting with a (mentally) clean slate reduces the chances of being biased. “It’s not about the what, it’s about the “why”
  • I touched upon the topic of the Baader Meinhof phenomenon and how testers could leverage the effect by absorbing as much knowledge as possible, on several subjects (a blog about that has been sitting in my drafts since january 2010 – I’ll try to finish that)

Introducing exploratory testing in Dutch projects

Ray then presented an experience report on how he was able to introduce exploratory testing and session based test management in classic, T-Map-style projects, using the principles he learned from Rapid Software Testing. Discussion ensued on how to prove the benefits of RST, and what the major differences between the approaches are. But we ended up talking mostly about “release advice”, and what to do when you’re asked to give it. One take-away phrase for me: “it’s not declining, it’s empowering the product manager”.

Walking break & Positive Deviance

Although we finished the previous topic way ahead of schedule, everyone felt like the last discussion drained our energy (our staying up late the night before probably didn’t help either). Jeanne, who facilitated the afternoon sessions, had the brilliant idea to just go out for a walk in the “Utrechtse Heuvelrug” national park, which turned out to be a conference session in its own right: relaxing, fun and informative. A beautiful spot, too. There was a moment where I thought we were getting lost, but here’s another lesson: do not underestimate the power of nine explorers, without a map.

Back at the hotel, Michael talked about positive deviance and positive deviants (people whose uncommon but successful behaviors or strategies enable them to find better solutions to a problem than their peers, despite having no special resources or knowledge). He also showed us a video of Jasper Palmer, a patient transporter at the Albert Einstein hospital (and a positive deviant) who became famous for his “Palmer Method”, which is now a standard life-saving practice in a number of hospitals. A mighty fascinating topic, that I’ll be exploring more for sure.

Credibility

Ruud delivered the closing presentation, on credibility – the quality of being trusted and believed. The main issues Ruud addressed were: how do we – testers – build credibility, and how do we manage to maintain it? After all, trust is built slowly, but destroyed in seconds. Simple questions, but a very complex subject indeed. “Trust” and “credibility” are relations: you can be credible to some person at a certain moment in time, but totally incredible to another. Trying to build your credibility is not always something controllable. Sure, you can do your very best to improve your credibility on a personal level, but you don’t really have an influence on how people will perceive you. Ruud then explained how he tries to build credibility. He impressed me with the personal mnemonic he developed, and the matching artwork as a personal reminder to stick to these principles:

STYLE

  • Safety language
  • Two ears one mouth
  • Yes but
  • Lighten up a little
  • Empathy
I’m not going in detail here, because I specifically want Ruud to finish that blog post he’s been mulling over for ages now. So, yes Ruud, the pressure is on. You’ve got some great material – time to share it with the world!

DEWT1 ended with drinks, testing games and dinner. I ended the day way more energized than I started it, which is always a good sign (silly extroverts like me get fueled by events like this). DEWT1 rocked. It was informal, informative and entertaining. When is the next?

On TED, James Randi and Software Skeptics

Source: xkcd.com

TEDx Brussels

In december I attended TEDx Brussels, a local spin-off  of the hugely succesful TED events. It turned out to be very worthwile and inspirational. The verdict: whole lotta hits, and a couple of misses.

One of these misses was a non-functioning wireless connection, leaving all these hip TEDsters bewildered, not knowing what to do with their hands, phones and Ipads. It was a priceless sight.

Another miss, Miss Lynne McTaggart, manifested herself late in the coffee-fueled afternoon, when she took the stage to talk about her Intention Experiment. She stated that the universe is connected by a vast quantum energy field and can be influenced by thought, enabling us to change our lives (and the world, while we’re at it).

Experiments gone bad

Now, I’m quite a film buff and the movies taught me that a little suspension of disbelief goes a long way, so I smiled, sat back and decided to enjoy the show. Lynne McTaggart started to describe some mind-over-matter experiments that she organized, including an effort to improve the water quality of Lake Biwa, Japan by concentrating on the lake together with a large group of people. She claimed she had scientifitic evidence that it really worked.

My tester genes started itching. Was she serious? The evidence will be next, I’m sure. Err… nope. She started talking about her healing powers instead. I was too confused to be amused. Where was the evidence? My comfy chair became less comfy all of a sudden. She called out for people with illnesses. A woman with a nasty bacterial infection stumbled forward.

“This can’t be happening”, I thought. I started looking around to check the reaction of others, but Lynne beat me to it and asked us to hold hands with our neighbors and concentrate on this poor woman’s illness while new-age music started oozing from the speakers.

That is where I drew the line. No holding of sweaty hands with strangers on a spacy soundtrack. I’m a tester, dammit. A skeptic. The music stopped and Lynne McTaggart quickly rushed off the stage, leaving everyone bewildered. What did just happen? No idea whether the audience beat the infection. She didn’t even bother to check. No way to verify her claims then and there. If only she would have focused on healing the TEDx wireless. That would have won everyone over.

James Randi is a tester!

On the way back home, I was still pondering what I just witnessed. This is exactly the kind of crookery that magician and professional skeptic James Randi has been battling against for years. Randi gained recognition in 1972 when he publicly challenged the claims of spoon bender extraordinaire Uri Geller. Later on, in 1996, Randi founded the JREF (James Randi Educational Foundation) with the mission of educating the general public on the dangers of accepting unproven claims, and to support research into paranormal claims in controlled scientific experimental conditions. They are also awarding critical thinking scholarships to college students nationwide. How cool is that?

I’ve always admired James Randi for his unrelenting skepticism and his ongoing battle against paranormal fakery. For me, he is a testing role-model, the original tester. Like Randi, I’d like to see myself as a professional (software) skeptic. This reminded me of something James Bach wrote a long time ago: testers should have a wary eye and a skeptical mind. James also stated that skepticism isn’t the rejection of belief, but rather the rejection of certainty. And that’s what testers should do: it’s natural that our clients want to become more confident in the system under test, but we should make clear to them that at no point we can promise absolute certainty. In fact, we should be the ones still doubting when every one else isn’t seeing any problems.

The one million dollar test

The JREF offers a one million dollar prize to anyone who can demonstrate a supernatural or paranormal ability under agreed-upon scientific testing criteria. They originally put out the challenge in 1968 for 100 dollar, and the prize money has been rising ever since. Randi has put out very stringent entry criteria for testing, as it should be: claims that cannot be tested experimentally are not eligible for the challenge. Needless to say that no one ever collected any prize money.

Can’t we learn from that?

I think we can. Let us not test the untestable, and make sure that we don’t start testing when we can’t create minimal conditions of testibility. When the results of our tests won’t tell us something valuable, why bother? We’re not dealing with telekinesis or bent spoons here, but with faulty software brimming with risks and emergent behaviors. We aren’t surrounded by ectoplasm, ghosts or faith healers, but by people that think we can “test quality into a product” and that it is possible to “test everything”. Sounds like testers shouldn’t only be software skeptics, but software myth busters as well.

Software myth busters? I can live with that. But that’s a different blog post altogether.

————-

EDIT:

Some footage of James Randi in action:

It’s… thinking thursday

The Challenge

A couple of minutes ago, Michael Bolton tweeted

Thinking Thursday. Test this sentence: “In successful agile development teams, every team member takes responsibility for quality.”

My initial reaction was: “A Michael Bolton challenge – where’s the catch?” This is actually a sentence that shows up regularly in agile literature. Heck, I even said it myself a couple of times. What I really wanted to say at the time, was probably something along the lines of “In agile development, producing quality software should be a team effort – lots of collaboration and communication. No blaming or finger-pointing individuals.”

I tweeted some replies, but soon realised that I would hit the 140 character limit head on.

The Test

But then I thought – why not give these kinds of agile creeds Weinberg’s “Mary had a little lamb”-workout, usually reserved for demistifying ambiguous requirements. I used it earlier: stress every word in turn and see where the ambiguities are.

  • In?
    Does this mean that outside agile development teams, no team members take responsibility?
  • Successful?
    Does this imply that in unsuccessful agile development teams, no one takes responsibility for quality, or that some individuals take the blame? Successful to whom, and compared to what? What is meant with “success”, really? On time, within budget? Satisfied customers? All of these combined?
  • Agile?
    What “Agile” definition are we talking about? Capital A, small a? A mindset, a methodology?And what about successful waterfall teams? Do some individuals take responsibility there? I would like to think that in successful teams, all team members would like a part of the praise. What about those other kinds of development teams out there?
  • Development teams?
    Are we talking about developers only here? What about the tester and product owner role? Or all the other roles that played an important part in developing the product? “In agile teams, testers *are* part of the development team”, you say? I agree, as are the product owners. But in that case, we should think about another label for the team.
  • Every?
    Really? *Every* team member? Can all team members be equally responsible for quality? As Michael Bolton contends, testers do not assure quality. Do testers hire the programmers? Fix problems in the code? Design the product? Set the schedule? Set the product scope? Decide which bugs to fix, write code?
  • Team member?
    What about people that played a part in successfully delivering the product, but that are not considered as core team members? Who are the people that make up the team? Is that defined up front? Aren’t those team boundaries pretty dynamic?
  • Takes Responsibility?
    Doesn’t *taking* responsibility sound a bit too negative? Isn’t “responsibility a two-sided sword? Receiving praise when the quality is applauded, taking the blame when quality turns out to be sub par?
  • Quality?
    Quality, to whom? Qualitative, compared to what? What is quality, anyway?

Is there a problem here?

Well… The sentence under scrutiny sounds comfortably familiar, and in that sense it was a good thing to think it through in a little more detail. It sure leaves a lot to interpretation. Some of the terms used in it are highly subjective or their definitions simply not generally agreed upon.

Back to twitter

Later on, in a response to a tweet from Shrini Kulkarni, Michael said that his purpose was “exploring what bugs me (and others) about it”.

Actually, nothing bugged me about it *before* the exercise, but now it dawned upon me that the wording of that good agile practice does not do the practice justice. It is too vague; it does need rephrasing.

How about a Frustrating Friday challenge: make this sentence fresh and ambiguity-free.

You could postpone it to Semantic Saturday, if you wish. Your call.

Whose problem is it, anyway?

My oldest daughter is suffering from a split lower lip for quite some time now. It appeared shortly after the first grim winter spell. It didn’t hurt, she said, so we didn’t pay a lot of attention to it. We treated it with a special ointment and a rather girlish chapstick. But after a while, it occured to us that the wound wasn’t healing nicely. What’s even worse: because of her constant fidgeting with the newly formed crust, it wasn’t healing at all.

So far, every attempt to stop her from doing that was met with total indifference. I couldn’t wrap my head around it. This would surely develop into a scar. In her face. Come on – didn’t she see that she was slowly mutilating herself?

A couple of days ago, I tried to tactfully tackle the issue once more during a toothbrushing session.

– “Time to brush those teeth, girl”

– “Okay, daddy”

I noticed some dried-up blood on her lip.

– “Did you remove the crust again? I told you not to do that”

She kept her cool.

– “I didn’t”

– “Don’t lie, honey…”

– “Well, not intentionally…”

I thought she was just testing my patience.

– “You’re pulling my leg, right?”

– “No daddy. Really. No. I mean… I can’t help it. And I don’t remember touching the crust.”

– “I told you, you should let it heal nicely. If you keep scratching your lip and removing the crust, it will become a scar. And scars aren’t pretty.”

She started brushing her teeth, gazing down at the sink. Total indifference. Again. I didn’t understand. How could she remain so calm under all this? I already had visions about her scarred face and classmates making fun of her, and she couldn’t care less! I was about to shift in daddy preaching mode (that it would make her ugly, that she would regret it big time later on, all that stuff) when, suddenly, she looked me straight in the eyes.

– “Why is that such a big deal, daddy?”

– “What do you mean? You don’t care about your face?”

I thought she was just provoking me. Instead, she gave me a brief look into her six-year old unspoiled mind, teaching me a valuable lesson in the process:

– “It’s no problem, really. I don’t see it.

– “You don’t see what?”

– “My face. I don’t see my face.”

She left me speechless for a while after that. I was stupefied. Of course! She doesn’t care, because she simply doesn’t see a problem.

Only afterwards, it occurred to me that these are the kinds of situations that are described in “Are your lights on?” (by Jerry Weinberg and Don Gause). A highly recommended and playful book on problem solving, by the way.

The authors describe a problem as “a difference between things as desired and things as perceived“.

When confronted with a problem, they advise us to:

  1. Identify the problem
  2. Determine the problem’s owner
  3. Identify where the problem came from
  4. Determine whether or not to solve it

The problem

What is the nature of the problem? A wound in her lip that is not healing. The constant fidgeting with the healing wound might cause an ugly scar or an infection. Or both.

The problem’s owner

Whose problem is that lip, anyway? My daughter’s? Her lip doesn’t hurt. And she perceives things differently: it never occured to me that she doesn’t see it constantly. She’s not the kind of girl that spends her time in front of a mirror, and if she does, it is only to admire that fancy new dress, her fairy make-up or that special Pippi Longstocking hairdo. But *never* her face. So, unless it starts hurting or until she hits puberty and starts seeing herself through the eyes of others, it will pretty much be the problem of her worried parents.

Where does the problem come from?

As the book also points out, the source of the problem most often originates within the person trying to solve the problem. Parents want their children to be healthy, beautiful, succesful and happy. Anything that threatens our children’s bliss worries us. In this case, we got nervous, made her nervous, possibly reinforcing the fidgeting behavior.

Should we solve the problem?

We want that wound to heal beautifully, for sure, but is this something we can really solve ourselves? Our daughter will only be motivated to adapt her behavior when *she* starts seeing it as *her* problem. Until then, and unless it develops into something more severe, we are perhaps better off by leaving the wound as is – let nature have its way.

Epilogue

“Children wish fathers looked but with their eyes;
fathers that children with their judgment looked;
and either may be wrong”
(William Shakespeare)

That day, my daughter made one thing crystal clear to me – I shouldn’t inflict my fears and worries on a six-year old who doesn’t yet care about her image in the mirror.

Rebel rebel – the Danish Alliance @ Eurostar 2010

Something way cool happened at Eurostar this year. A group of like-minded people got together after the conference to do a mini-CONFERence in a more intimate setting. They called themselves the Danish Alliance (or Oprørsalliancen, when they felt like badly pronouncing Danish words). The concept was based on the Rebel Alliance, started by Matt Heusser at StarEast last year. I had been thinking about a localized version of the Alliance before, but it was the ever energetic Shmuel Gershon who put his efforts into organizing the first Alliance on European soil. Of course, this little guerilla conference couldn’t have happened without the generous help of the Eurostar folks, who set us up with a superb meeting room. Need I say that they ROCK?

The ingredients were simple: 

  • A handful of passionate testers
  • A safe setting
  • Drinks
  • Pizza
  • Music
  • Chocolates & cookies

Throw all these together and stir gently. Observe.

Whatever happens, happens. There was no agenda, really. In this case we mingled first, talked and drank a bit until pizzas arrived. Major  epiphany: Denmark has pizzas that come in the size of a small wallaby. After that, there were some lighting talks, timed by quality gatetimekeeper Michael Bolton (who definitely should get into the timekeeping business whenever he gets out of the QA business). You can see (transcripted!) videos of the talks in Shmuel’s write-up of the event

‘Talks’ don’t have to be ‘talks’, per se. James Lyndsay did a call to action to test one of his new black box testing machines. Andy Glover (the Cartoon Tester) got us drawing abstract concepts. Dorothy Graham even gave us a Sound of Music flashback by singing about her favorite techniques. Anything goes.

Discussions continued until the wee hours. I thought it was wonderful. This is the kind of stuff that doesn’t regularly happen during the day at conferences. Sure, the Eurostar programme was great, again (and I’ll be writing more about that later), but the real conferring often happens outside the track sessions and tutorials. It feels great to connect with other people that are all driven by the same thing: a passion for their craft.

So thank you Shmuel Gershon, Jesper L Ottosen, Joris Meerts, Dorothy Graham, James LyndsayBart Knaack, Martin Jansson, Henrik AnderssonMichael Bolton, Andy Glover, John Stevenson, Rob LambertCarsten Feilberg, Ajay BalamurugadasMarkus GaertnerHenrik Emilsson, Julian Harty, Rob Sabourin, Rikard Edgren, Lynn McKee and Rob Lugton. The force will be with you, always.

Exploring Rapid Reporter

Eploring Rapid Reporter

This is my attempt at an exploratory essay.

I will be documenting my first exploration of a note-taking tool (Shmuel Gershon‘s free Rapid Reporter) to assist exploratory testing sessions.  That’s right, I’ll be exploring an explorer’s tool – sounds like this has the makings of a pretty meta experience.

I first heard about the tool when Shmuel gave the Rebel Alliance at Star East a sneak peek of what he was working on at the time. The Rebel Alliance was a very cool “conference within a conference” organised by Matt Heusser, with plenty of lightning talks and great discussions. You can see a video of his short talk here.

Up till now, I have been using David Gilbert’s TestExplorer, Jonathan Kohl’s Session Tester and James Bach’s Scan Tool to document my exploratory testing sessions (with miscellaneous results), so when he finally offered it to the public during the Blogstar competition (in which Shmuel was a well-deserved runner-up by the way), I knew I just had to give it a try. 

I started out by reading the user guide on the site, skimmed through the known issues list on the page and read through the short faq section. During this little theory intake, I deliberately didn’t start the tool (although I had to suppress the urge). I wanted to start using it for a first time in a proper testing session. 

Start

I fired the RapidReporter.exe, was asked to enter a name and a charter, all pretty straightforward.

*Edit: Actually, I entered a session name when prompted for a name. Later, when looking in the .csv-file, I realised that it was supposed to be my own name, the name of the reporter. It confused me, so perhaps the naming was ambiguous? Would “Reporter” be a better option? (Made a NOTE about it).  

I was ready to test a piece of newly coded software. The cool thing was that RapidExplorer stayed on top, tucked away in a place that didn’t bother visibility: no alt+Tab switching, no interrupting my flow. Note-taking went smoothly. Switching the note type by the up/down arrows was intuitive and quick.

After a short while I noticed a blue progress bar ehm… progressing and it surprised me. That’s a nice feature, it gives you a visual clue of the session’s progress, but without the actual distraction and pressure of a detailed timer. The blue bar also implied a default duration, since I hadn’t indicated any preferred session length before starting. I wondered why the default duration wasn’t asked along with the name and charter, and figured that was probably because most people use the same timebox for their sessions anyway (60-90-120 minutes, whatever). So what is the duration of the current session? It isn’t indicated on the main bar. Is there a menu of some kind? The most intuitive thing would be a right-click, I’d say. Bingo! Three menu items there: Time until end, Open working folder and About. But where do I see the duration of this very session? The only time-related entry is the first one, but this only breaks down in a menu to select either ’60-90-120 min until end’ or to ‘stop the session’. Selected menu items in comparable products often have a selection checkmark or bullet next to them (NOTE).

I opened some menu items in the application under test and noticed an inconstistency, so taking a screenshot was in order. I didn’t remember reading about hotkeys, but I tried some anyway (CTRL+S, ALT+S, S – which just entered an ‘S’ in Rapid Reporter, of course). I was afraid I would lose the expanded menu if I clicked outside the application. I eventually ended up clicking the S button, but – as I expected – the screenshot didn’t show the expanded menu items. Apparently, there is no keyboard shorcut yet, which makes it impossible to capture these kinds of phenomena (NOTE). For all the other screenshots I ended up taking, clicking the S did the trick just fine. The captures were saved as time-stamped jpegs in the startup folder. A huge improvement, since I was used to take the screenshot, paste it in MSPaint, name it & save it. By the time I got to the application, I often forgot what I was actually doing in the first place. Darned short-time memory.

I got interrupted by a phone call and wanted to pause the timer. Mmm. No pause option to be seen. I tried hitting the spacebar to pause – don’t know where that came from. Media players or games, maybe? There *was* a the stop option in the menu, so I tried that. But this actually stopped the timer and reset the progress bar to zero. Is there an option to pause and “restart” where I paused it? I can think of some occasions in which that would be useful (NOTE).

I was still pondering the timer reset, when I noticed that the progress bar started progressing again. Is the “stopped” state really a stopped state? Tried it again, but now stopping really stopped it. Maybe I started a session without knowing that first time? Maybe something intermittent? (INVESTIGATE LATER)

When time was up, this was clearly indicated by the flashing stopwatch. It was still possible to add keep working with it even when time is up, which seems like a logical and practical thing to do. I can think of many situations in which you want to continue for a bit, clean up some loose ends.

I started a new session, worked with that for a while. I was really getting used to this new kind of notetaking. I wanted to extend the length of the session, and wondered what would happen if I changed it to 120 minutes. I expected the progress bar to redraw itself (keeping in mind the time spent and time still to go), since my current session was 60 minutes. But it didn’t. Did the duration really change? Possibly, but no visual clue to confirm that (NOTE).

I was curious about what was actually being recorded during the session and opened the csv-file in the startup folder. All my notes were there, time-stamped and delimited with commas. All nice and powerful.

Stop

I stopped the session while it was still ongoing (by using the cross in the upper right corner) and got the message that an error occured when writing  the note into a file. Yes, the csv-file was still open and although I didn’t enter a note since opening the file, the application attempts to write something into the file when closing. It would be nice if the excel closed automatically instead of throwing an error when nothing changed (NOTE).

I checked the screenshots taken during the session (by clicking “S”) and the Rapid Reporter bar is in the screenshots. At first I thought this could be annoying for a tester, but then I figured that it would always be in a non-disturbing place anyway. It can also convey useful information: how far you were in the session, what the note was at the time, etc. 

I decided to try taking a screenshot the regular way (using Printscreen), and it turns out that the Rapid Reporter bar is *not* in the picture. Strange. Seems like an inconsistency, but I might not have all information here. I’ll talk to Shmuel about it (NOTE).   

Conclusion

I decided to stop my first exploration of Rapid Reporter here. It was a pretty shallow tour of some basic functionalities – I didn’t even look into the reporting possibilities yet, I’ll certainly do that next time. This little write-up just documents my first two hours of exploring a new tool. I wanted to learn about it first, not necessarily find bugs.

To round it up: I must say I’m impressed. It’s easy to use, and non-intrusive indeed. When using other Session Based Test Management tools, I often ended up spending more time in the note-taking tools than in the application under test, which is a bit of a drag. Rapid Reporter puts all focus on the software being explored. It supports exploration, while allowing testers to stay in their flow. This is important for me anyway: everytime someone interrupts my flow, God kills a kitten. Ok, not really, but you catch my drift.

Of course, I stumbled upon some issues, but they didn’t annoy me, they didn’t make me stop using it. I didn’t expect perfection – after all, the readme file explicitly mentions that it’s still buggy. I would have gotten *really*suspicious if it would have claimed it was bug-free. So its current state is perfectly okay for me. After all, it is brand new. It’s work-in-progress. The best thing Shmuel could do, was to unleash it upon an international cohort of testers. And he did. It turned out to be a great gift to the testing community.

Will definitely use again. A+++

Why you shouldn’t miss EuroSTAR 2010

10 reasons why you shouldn’t miss Eurostar 2010

Two weeks from now you will find me in trendy Copenhagen, proud home of the world’s best restaurant (Noma) and Hans Christian Andersen’s Little Mermaid. But the real reason for my trek up north is not sightseeing or spending money on Tørret kammusling og biodynamiske gryn: Copenhagen is also the host of the 18th edition of the annual EuroSTAR testing conference.

If you’re not yet familiar with Europe’s biggest software testing conference, you should definitely check them out. If you’re still hesitating about attending, there’s no need to. If you’re thinking of going, do go. Here’s why:

  • First and foremost: the Content.
    • The programme committee assembled a promising line-up, centered around the main theme “sharing the passion”. Track session categories include test management, exploratory testing, virtualisation, techniques, Scrum, inspiration, education, Lean, MBT, people and automation, among many others. L’embarras du choix.
    • Keynotes, anyone? Antony Marcano, Rob Sabourin, Bob Galen, Dino Patti and Stuart Reid. Recent history teaches us that wherever Stuart lays his hat, controversy and discussion automagically appear. I’m confident that his keynote “When Passion Obscures The Facts: The Case for Evidence-Based Testing” will be no different. 
    • I especially look forward to the tutorials. Rob Sabourin will run a full-day tutorial on “Just In Time Testing – Effective Testing Strategies“. Michael Bolton will be doing a half-day tutorial on Test Framing (read his blogpost that coins test framing here). But that’s not all. There’s Lee Copeland too. And many, many more. L’embarras du choix, revisité.   
  • Test Lab.
    James Lyndsay and Bart Knaack will run their on-site Test Lab for the second consecutive year. They will be assisted by Henrik Emilsson and Martin Jansson, 2/3 of that restless online blogging collective called The Test Eye. The other 1/3 is Rikard Edgren, who is part of the programme committee this year – I guess you could say that Eurostar is TestEye-infected. From what I experienced last year, the test lab is a really unique experience. Live testing at a testing conference! Theory put into practice, and maybe some weekend testing sessions, testing dojos or katas. Anything goes, really.
  • Inspiration.
    Hearing all these different viewpoints, new ideas, talking with the experts, engaging in discussions… It’s a savory buffet full of food for thought. Attending conferences is intellectually stimulating, and you’ll probably learn more during these couple of days than you do during most ‘regular’ training courses. I see EuroSTAR as a multi-dimensional training course that as such deserves to be on every company’s training calendar.
  • Get Primed.
    Any problems you are facing at work – you just might see them differently when you get back. Things you hear at the conference and  people you talk to often trigger other ways of thinking. Conferences tend to broaden your perspective on things.
  • Reach out to the testing community.
    This year’s theme is ‘Sharing the passion’, which should make it easy to meet like-minded people who share the same interests. In his 2009 book “The Element”, Ken Robinson calls this “finding your tribe”: connecting with people who share the same passions and commitment (your “tribe”) helps in finding and developing your “element” (which is the place where passion and skill meet). Members of a passionate community tend to stimulate each other to explore the real extent of their talents. Whenever tribes gather in the same place, the opportunities for mutual inspiration can become intense.
  • Meet Testing Tweeps.
    Twitter has been doing brilliant things for testers already, community-wise. It’s a great way to interact with testers worldwide on a daily basis. It has also proven to be a very useful, fun and informative way to cover conferences, especially for the people missing out (watch that #esconfs hashtag for some conference goodness). If you’re on twitter, EuroSTAR will also be a good opportunity to meet numerous testing tweeps in person and to take your twitter-conversations with them to the next level.
  • Hallway/Bar discussions.
    In an earlier blog post, C is for Conferring, I mentioned that conferences are for conferring, and that the most interesting things often happen in the hallways, in between sessions. Or in the bar. Or somewhere totally unexpected. Make sure there are blank spots in your busy schedule to invite serendipity.
  • One word: Copenhagen.
    ‘Nuff said. But did I mention that the place of action is the Bella center? Last year, the Bella center hosted the first sustainable, international political summit – the United Nations Climate Change Conference (COP15) – attracting over 33,000 people. No worries, Eurostar Conferences assured me that Eurostar 2010 will be more succesful than its flunked climatic counterpart.
  • Interactive Panel Session.
    On wednesday morning, Lee Copeland will facilitate a Hot Topics Panel Session. The expert panel is there to address *your* burning issues, so if you want to ask the EuroSTAR Panel a question, you can do so via facebook. Yes, the Social Network goes testing.
  • It’s fun!
    By focusing on all the content, the learning and networking, I almost forgot to mention that above all, it’s fun. All of the above takes place in a fun and relaxed athmosphere.Fun sessions for the weary testers are foreseen as well (I’ve been told that the supertesters are something to look out for).

This concludes *my* list. Rob Lambert wrote about attending EuroSTAR too, in his post EuroSTAR will rock. Eurostar Conferences has also listed their Top 10 Reasons To Attend EuroSTAR 2010! And if you need to make a case for attending the conference, the 10 ways to convince your boss to send you to EuroSTAR 2010 article may be able to help you with that.

I hope to see you in Copenhagen. I’m @TestSideStory, by the way. I’ll be roaming the hallways – feel free to come and talk to me. I’ll be the one with that thorny rose clenched in the teeth.

Short service announcement: tomorrow, October 16, programme committee member Peter Morgan will present a webinar especially for first time attendees. It is called “Getting The Most Out Of EuroSTAR“. More info and a link to register for the webinar can be found here.