Agile Testing Days 2010 – Day 2 (Defect limbo and stunt hamsters)

Agile Testing Days 2010 – Day 2 (Defect limbo and stunt hamsters)

Advertisements

October 5

After a full day of playful tutorials on monday, it was back to business on tuesday. The actual conference kicked off with an interesting keynote by Lisa Crispin. Lisa is an author/agile tester extraordinaire/donkey afficionado with Belgian roots – a winning combination if you ask me. The title of her talk was “Agile defect management” and was all about finding a suitable approach to manage and track defects in agile projects. Lisa used the limbo analogy for defects, stating that we should strive to lower the bar on defects. I liked the analogy – but had a hard time dissociating from all the alcoholic connotations of a classic limbo-fest, where the bar is generally lowered until the first drunkard ruptures his anterior cruciate ligaments. I think it’s time to groom my unconscious backlog a little.

In the agile/lean world, using a defect tracking system (DTS) is generally seen as wasteful, since agile teams strive for ‘zero defects’. Instead of filing bugs in a DTS, they prefer to fix the problem immediately by creating an automated test for that defect and adding it to the unit test suite.

I particularly liked Lisa’s “tabula rasa” idea: try to start your project *without* a defect tracking system and see what you need as you progress. Set some rules like “no more than 10 bugs at the same time” and fix the important bugs immediately. You could even use a combination of a DTS and defect cards on a board. Use the DTS for defects in production and defect cards on the story board for defects in development.

The next track I attended was “Incremental Scenario Testing: Beyond Exploratory Testing” from Matthias Ratert. He started off by explaining that they performed exploratory testing in their project, that it was helpful for about 2-3 test sessions , but that it became increasingly difficult for the testers to come up with new and creative test ideas and that too many areas of the complex system remained untested.

My exploratory tester heart was bleeding at first because they dismissed exploratory testing so quickly. When I heard that they were using unskilled, untrained and outsourced labor in the form of students, without experience and/or motivation to continue in this line of work, it all made sense. No wonder the exploratory testing yielded sub-par results.

In order to cope with the testers’ lack of imagination and sense of coverage, they developed a tool (the IST-tool) to do Incremental Scenario Testing (IST). The tool was used to automatically generate test scenarios as a starting point, which were composed of preconditions, states and events. It was tweakable by all kinds of different parameters to suit different contexts. The testers would still have the freedom to test in an exploratory fashion withing these predefined areas, without stating expected results. The tool could be configured so that important areas would appear more often in the testers’ selected scenarios.

The tool as such sounded like a good solution to  generate different scenarios, to spread and divide work to have a better coverage, but in my opinion will not solve their initial problem: they were still letting unskilled and unmotivated people perform exploratory testing, which is especially known to be a highly skilled, brain-engaged activity. Replacing them was apparently no option because of budgetary reasons. But why not try to train them into first-class ET-guerilleros first?

For the last morning session I chose to attend “One small change to code, one giant leap towards testability” by the lively and ubiquitous Brett Schuchert (presenter of two track sessions and the open space facilitator on thursday, how much more omnipresent can you get?).  His topic was mainly technical – how to design for testability. He used the example of a dice game in which the rolling of the dice was a factor that was beyond our control. In order to make the design testable, we should use dependency injection: create two loaded dice with a predictable result, and feed them into the game.

Schuchert’s inner showmaster came to surface when he threw some jeopardy-style quotes at us to illustrate the importance of Test Driven Development – a design practice rather than a testing practice:

The answer is 66%. What was the question?
“What is the chance that a one-line defect fix will introduce another defect?” (Jerry Weinberg)

After a copious lunch, ‘Fearless Change’-author Linda Rising inspired the big auditorium with her keynote “Deception and Estimation: How We Fool Ourselves“. She defined deception as consciously or unconsciously leading another or yourself to believe something that is not true. Her main message was that we constantly deceive ourselves and others. She illustrated her point with the typical marrying couple at the altar. Although current studies indicate that chances for a marriage to succeed are only 50-50, this knowledge doesn’t keep anyone from getting married. I didn’t know these odds when I decided to get married. I like to think it wouldn’t have made a difference – I’ve always liked to defy statistics. 

Us humans, we are a strange lot. We are hardwired to be optimistic. We see what we want to see, so we unconsciously filter out the things we dislike. After all, we fear what we cannot control. And it’s here that estimations come into play. Our hardwiring biases our estimations – we constantly overestimate our ability to do things: coding, testing, everything. And do we ever learn from our mistakes? But we shouldn’t be too overwhelmed by this, there’s hope: achieving good enough estimates isn’t totally impossible, if we just take small enough steps, experiment, learn from failures as well as successes.



Software Testing Club busybee Rob Lambert provided some very good food for thought in his talk “Structures Kill Testing Creativity“. I don’t know how deliberate it was, but he did this really cool thing of standing at the door outside the room and greeting people as they came in. It certainly made me feel welcome from the beginning. He was also the first person up till then that I saw using Prezi. A kindred spirit! I sat back and enjoyed the show. The main point of his presentation that in order to foster creativity, we need *some* structure (he used the example of a sonnet which has *some* predefined rules to it), but that imposing excessive structure upon people and teams will suffocate creativity (my wording, not his). Rob defined creativity through the equation:

Expertise + Motivation + Imagination = Creativity

Rob then tried the Purdue creativity test on his audience – he asked us to draw the person sitting next to us in a mere 30 seconds, which led to some hilarious results (you can check some of the drawings on his blog – the speed-portrait of Lisa Crispin by the artist formerly known as Ruud Cox is pretty mindblowing). The point of the exercise was to show that when we have to share creative ideas, we constantly self-edit. We feel shy and embarrassed, even more so if the environment doesn’t feel safe to us. True. It struck me that almost everyone was apologizing for the bad portraying afterwards. Excessively structured environments don’t make good breeding ground for creativity.

Rob Lambert told a couple of personal stories about people who actively pursued creative environments for the better. Marlena Compton moved to Australia to work at Atlassian, a cool company without excessive structures in place. He talked about Trish Koo, who works at Campaign Monitor, a company that is apparently all about people. He mentioned Pradeep Soundararajan, who started using a videocamera to film testing in progress to make the testing language more portable and universal.

© Quality Tree Software Inc.

Dynamic? Peppy? Is there a stronger word than energetic? If so, it would describe the keynote by compelling storyteller Elisabeth Hendrickson. The title of her talk rang a little bell: “Lessons Learned from 100+ Simulated Agile Transitions“. The main subject was indeed the infamous WordCount experiment, which she has done numerous times, including her tutorial on day 1 of the conference (you can find my write-up for that here). Because of non-disclosure agreements, she couldn’t use actual pictures, so she used stunt hamsters to illustrate her point. Throughout her talk, it was nice to see that our WordCount group from the day before was no bunch of forgettable dilettantes. It was déjà-vu all over the place:

  • Round 1. The computer is bored (check)
  • Round 2. Chaos (double check)
  • Round 3. Structure (triple check, at least the beginning of structure)
  • Round 4. Running on all cylinders (quadruple check)

 The lessons learned: teams that struggle, typically

  • Hold on tight to rules, silos and work areas
  • Have everything in progress and get nothing done
  • Sit in meetings constantly instead of creating visibility
  • Fail to engage effectively with the customer

Teams that succeed, generally:

  • Drive development with customer-provided examples
  • Seek customer feedback early and often
  • Re-shape their physical environment
  • Re-invent key-agile engineering practises like ATDD and CI

The ressemblance to what our team went through the day before was striking. I was still pondering that throughout dinner, when it hit me that it would be my turn to perform the next day. By that time, the stage in the dining hall was taken over by overly enthusiastic improv actors, but I wasn’t really in the mood for that kind of entertainment. I obeyed the voice in my head. Must. Prepare. Prezi. 

To be continued… Day 3

Agile Testing Days 2010 – Day 1 (Agile transitions)

Agile Testing Days 2010 – Day 1

After a great experience at the Agile Testing Days last year, I decided to answer their call for papers early. By the time the full program was announced (somewhere in april), I had almost forgotten that I participated. So it was a pleasant surprise to see my name listed among all those great speakers. I decided to break out of my comfort zone for once and in the last minute I “prezi-fied” my existing presentation. Confidently stressed, I flew east to Berlin to be part of what proved to be a wonderfully memorable conference. 

October 3

It was sunday October 3, which meant I arrived on the 20th anniversary of the German unification. The last time I had been in the city centre, Berlin was still a divided city. I was 16, and overwhelmed by the contrast between the neon-lit Ku’damm and the clean but spookily deserted East. Going through checkpoint Charlie to the East – and happily back again, while others desperately wanted to but couldn’t – still ranks among the most awkward moments in my otherwise pretty uneventful youth. Sure, the Alexanderplatz, Ishtar gate and Pergamon museum impressed me, but why a country would deliberately lock up its people was totally beyond my 16-year-old self.

So, with a few hours of daylight left, I headed to some sites that I still remembered from the days of yore. The Brandenburger Tor was now the backdrop for big festivities: music, beer, bratwurst and parachute commandos executing a perfect landing at Helmut Kohl’s feet at the Reichstag. No concrete walls to be seen. Unter den Linden completely opened up again. It felt great. Sometimes nostalgia isn’t what it used to be.

October 4

© Stephan Kämper

The morning of tutorial day, the Seminaris Hotel conference lobby was buzzing with coffee machines and activity. I had enrolled for Elisabeth Hendrickson‘s “Agile transitions” tutorial, which turned out to be an excellent choice. Eight people were taking part in the WordCount experiment, of which Elisabeth recounts an earlier experience here. After a round of introductions, we divided roles within the WordCount company: tester – developer – product manager – interoffice mail courier (snail mail only) – computer (yes, computers have feelings too) or observer. Strangely enough, I felt this natural urge to be a tester. I didn’t resist it, why should I? Elisabeth then proceeded to explain the rules. We would play a first round in which we had to stick to a set of fixed work agreements, like working in silos, formal handoffs and communicating only through the interoffice mail courier. The goal of the game was basically to make our customer happy by delivering features and thus earning money in the process.

We didn’t make our customer happy, that first round. On the contrary – confusion, chaos and frustration ensued. Testers belting out test cases, feeding them to the computer, getting back ambiguous results. Developers stressed out, struggling to understand the legacy code. Our product manager became hysterical because the customer kept harassing him for a demo and no-one was responding to his messages. The mail courier was bored, our computer felt pretty abandoned too. It all felt wonderfully unagile.

In round 2 we were allowed to change our work agreements any way we wanted, which sounded like music to our agile ears! We co-located immediately and fired our mail courier. We organised a big kickoff-meeting in which the customer would explain requirements and walk us through the application. We already visualised the money flowing in. In theory, theory and practice are the same. In practice – not so much. We spent a whole round discussing how we would work. We lost track of time. There were no new features, and no money. We felt pretty silly.

Round 3 was slightly better. We were able to fix some serious bugs and our first new features were developed, tested and working. But just when we thought we were on a roll, our customer coughed up some examples that she really wanted to pass too. They didn’t. 

Pressure was on in round 4, which was going to be the last one of the day. Would we make history by not delivering at all? Well, no. We actually reinvented ATDD, by letting the customer’s examples drive our development. This resulted in accepted features, and some money to go with that. We managed to develop, test and demo some additional functionalities too. A not-so-epic win, but a win nontheless. Wordcount was still in business. If there would have been a round 5, I’m pretty sure WordCount Inc. would have made a glorious entrance at the Nasdaq stock exchange.

Elisabeth did a great job facilitating the discussions in between rounds and playing a pretty realistic customer. All the participants made for a very enjoyable day too. The day really flew by and ended with a great speaker’s dinner at the borders of the Schlachtensee. A Canadian, an American, a German and a Belgian decided to walk back to the hotel instead of taking the bus. It sounds like the beginning of a bad joke, but that refreshing 5km walk through the green suburbs was actually the perfect closure of a terrific day. And without a map, I might add. As the rapid Canadian pointed out later: documentation is overrated.

A Eurostar interview

A Eurostar interview

A while ago, there was this little announcement on the Eurostar blog:

“As a new addition to the EuroSTAR community, we will be interviewing prominent testers from across the globe”

I thought that was pretty cool. There is lots to learn from experienced people. It’s nice to hear all these different takes on the sofware testing craft. They already published interviews with Isabel Evans, Mats Grindal, Tim Koomen, Michael Bolton, Martin Pol and Anne Mette Hass. Interesting stuff.

Several months later, I received an email from Kevin Byrne from the Qualtech/Eurostar team asking if I would be interested in doing an interview with them on testing (and other things as well). It took me a while to properly connect the term “prominent tester” with my own name. But I was honoured of course, so I accepted their offer.

And there it is. They even call me a ‘prominent Belgian tester’ in the introduction, which made me smile because it reminded me of the phrase “being big in Belgium” – often used interchangeably with being “big in Japan”, meaning as much as “totally unimportant”.

In the 1992 movie Singles, Matt Dillon plays in a band that claims to be “big in Belgium” – subtext: “what a bunch of forgettable losers”. Similarly, the legendary rock group Spinal Tap (the 1984 mockumentary This is Spinal Tap is hilarious, by the way) ended up being big in Japan, which basically meant “pathetically uncool and ridiculed at home”.

But I digress. I might not be all too prominent, but I am a Belgian tester allright. Here’s the interview:

http://www.eurostarconferences.com/blog/2010/5/18/an-interview-with-zeger-van-hese.aspx

C is for Conferring

On conferences and how to keep it cheap

The blogosphere is pregnant with conference posts these days. And good ones too I might add.
  • In his blogpost conferences on the cheap, Matt Heusser gives some helpful tips on keeping the cost of attending testing conferences down in these difficult times.
  • Matt also created a conference wiki where everyone can add any kind of testing event or conference that is taking place. A great source of information, that is still expanding as we speak. If you know of any noteworthy events that are not yet listed, feel free to add them. 
  • Very recently, the Software Testing Club spawned an interesting testing conference discussion whether or not people are paying for conferences out of their own pocket. 

I think being at conferences is great fun. You meet peers. You talk and share ideas. And get ideas, as well. Occasionally, you attend a track that makes you go “what was that all about?!”, but in general I get tremendously inspired. Not only from the track sessions – most of the time the hallway discussions provide some decent food for thought as well. Of course these benefits aren’t always very measurable, but conferences are doing their very best to provide attendees with ways to convince their managers and prove the added value. Eurostar even provides a justification kit and a Conference Evaluation & ROI Worksheet for reporting on the ROI back home.

But there’s always other ways: 

  • You can get in for free as a speaker, but in that case there’s some effort involved. Writing abstracts, papers, making a presentation. Can be quite a hassle – tit for tat. And you have to get selected of course.
  • Some conferences give away conference tickets in lotteries. Slim chance, but you never know.
  • Step out of the dark side and show your face to the community
    – be a video star. Are you ready for your close-up, <insert name here>?

Eurostar 2009 – a week to remember

A write-up of the Eurostar 2009-conference in Stockholm

I absolutely *love* Stockholm in wintertime. Pepparkakor, glögg, gravad lax… and Eurostar too. People keep telling me that I would probably love it even more in summertime, but I’ll always associate those dark days with Eurostar. I presented my first Eurostar track there in 2007 – nothing but good memories – and I was selected this year as well. The Eurostar line-up is always pretty impressive, so it can be both intimidating and exciting to be a part of that. It’s just a matter of keeping the intimidation level below the excitement level, I guess. As a boyscout, good old Baden Powell always told me to “be prepared”. Now sometimes I wouldn’t recognize a life lesson if it punched me in the face, but here’s one that I did remember. So I found myself writing a paper and assembling a presentation during those hot holiday nights in Southwestern France. You just gotta love those early deadlines!

November 29

After an uneventful flight from Brussels to Arlanda, set foot on Swedish soil. Met up with fellow Belgian Mieke Gevers, a member of this year’s program committee and in charge of the track chairs as well. I helped her carry some excess bagagge that turned out to contain presents for the trackchairs – you can’t go wrong with Belgian chocolates and “jenever“. We took the Arlanda express (easy and quick) to Stockholm C and a cab to the Rica Talk hotel.

November 30 – Tutorial day

On monday I attended a full-day tutorial by Michael Bolton called “Exploratory Testing Masterclass” (slides available here). Two years ago I attended his tutorial on Rapid Software Testing, which I found very valuable. Michael Bolton is an engaging speaker and teacher who invites you to think, rather than just sit and absorb theoretical matter. There were lots of exercises, including one on factoring (identifying dimensions of interest in a product). We were asked to identify all dimensions of a wineglass that may be relevant to testing it, using the “San Francisco Depot” – heuristic (Structure, Functions, Data, Platform, Operations, Time) – not new to me but always worth repeating. A lot of mnemonic wizardry to be found here. What about that handy mnemonic for oracles – HICCUPPS/F (History, Image, Comparable product, Claims, User expectation, Product, Purpose, Statutes, Familiar problems) – never again say that you don’t know why something should be considered a bug. Care to take a ride on that test reporting heuristic called MCOASTER? Well I’ll see your CRUSSPIC STMPL, and raise it with a FCC CUTS VIDS (Mike Kelly’s application touring heuristic). Mnemomania!

Of course, there were plenty of other impressions that kept lingering for a while.

  • A quote by Jerry Weinberg: “A tester is someone who knows things can be different” – true.
  • “If it ain’t exploratory, it’s avoidatory” – made me laugh. 
  • “A good tester doesn’t just ask “Pass or Fail?”. A good tester asks “Is there a problem here?”.
  • CHECks are CHange detECtors, testing is exploring.
  • A complete debunking of some boundary value analysis truisms: it is generally accepted that the behaviour at boundaries is more likely to show erratic behaviour, but how do we know these boundaries? The actual boundaries in a system may not be the ones we are told about. That’s why we must explore.
  • Testing is “storytelling” – I liked that take on testing:

“You must tell a story about the product, about how it failed, and how it might fail – in ways that matter to your various clients. But you must also also tell a story about testing, how you configured, operated and observed it – about what you haven’t tested, yet… or won’t test, at all – and about why what you did was good enough.”

The end of the session was foreseen at 5 PM. The discussions kept going on until 5.45 PM. I think that says it all. Later that evening, an international amalgam of testers set out to explore the possibilities of finding food in Gamla Stan. Eventually we found an Indian restaurant using that good old I.NEWTON heuristic (Indian, Nearby, Edible, Welcoming, Tasty, Open, Not-too-expensive). The end of a great day. Had some nice conversations with Rikard Edgren, Tone Molyneux, Ray Arell and  John Watkins (my trackchair) as well.

December 1

The second day started with a tutorial as well, be it a half-day one: Managing Exploratory Testing by Jonathan Kohl. Of course there were a lot of similarities with the first tutorial, but this was more of a hands-on session, where we could put Michael Bolton’s concepts from the day before into practise. There was some theory about coverage models – SF Depot anyone? We ended up describing a whole bunch of characteristics of a table that we had never associated with an ordinary table before. Practical and fun. Certainly an eye-opener.

At that point I was still trying to get a hold of the person I was supposed to trackchair on wednesday. Originally I would be trackchairing my colleague Wim De Mey’s track about regression testing in a migration project, but Wim had to cancel his presentation at the very last moment because of unfortunate familial circumstances. A replacement was found in the person of Mika Katara, from Finland – but no sign of him, yet. Oh well, time for a quick lunch, a tour of the expo and the actual kick-off of the conference.  Dorothy Graham opened the 17th Eurostar conference in style. She introduced the program committee (Tone and Mieke made sure Isabel Evans was also represented by carrying an air-filled balloon with a face drawn on it – I’m not sure if Isabel would be too happy with the analogy 🙂 ) and set the scene for the first keynote speaker.

Lee Copeland started this very first talk of the conference about nine of the most important innovations in software testing: the context-driven school, test-first development, really good books, open source tools, session-based test management, testing workshops, freedom of the press, virtualization and “testing in the cloud”. Strange that he sees the context-driven school as an innovation – as far as I know it was founded in 1999; the first book that explicitly named it was already published in 2001. I agree with the freedom of the press thing. Testing blogs are appearing everywhere (guilty, your honour), twitter is on the rise. Lee is apparently not a fan of twitter. Neither was I – I always thought of it as encouraging the spreading of triviality, but I’m actually starting to come back from that. I noticed that a lot of people within the testing community are using it to share their ideas, give advice or call for help. And it gives a great deal of extra coverage to an event like this (see twitter.com/esconfs), so maybe I’ll give it a try. Later. 

The rest of the afternoon consisted of a series of  short 20-minute tracks, which is mostly just enough to launch some provoking ideas, but not really ideal for a lot of content. Johan Jonasson talked about how he managed to save a project with the introduction of a structured exploratory testing approach. This track would have benefited from a 45 minute timeslot – there was no time to go into detail, which I found a pity. Next up was Julian Harty, who explained the concept of “trinity testing”: short session of around 90 minutes per feature, where the feature owner, the developer and the test engineer work interactively through the software to share knowledge and ideas. Pretty interesting, since I also found out later that “the trinity test” was also the name of the very first nuclear test ever conducted, marking the very start of the nuclear age. Julian is probably aware of this – I didn’t hear him mentioning it, though.
Geoff Thompson then talked about reporting – “If only we could make them listen!”. Well actually, it’s more the communicator’s job to make sure he gets heard. It was a great talk – he was able to slip in the Challenger disaster and the Heathrow terminal 5 debacle as examples of how important messages were apparently not deemed important enough, with horrendous results. Knowing your recipients is key, and knowing what information they want as well. Noteworthy: a lot of people are color-blind. If you absolutely want to make sure that everyone understands your reports, shouldn’t you avoid the reds and greens?
Besides being a sapient testing evangelist, Michael Bolton is also a human quote machine. He did this cross between stand-up routine and  political televangelism called “Burning Issues of the Day” (available here). A lot of wisecracks and eye-openers, the funniest moment at Eurostar for me. He was even able to win a bet by slipping in a quote about agilists and sex:

“The agilistas did not discover pairing or test-first programming. They’re like teenagers who’ve just discovered sex. It IS great, but calm down”.

The last speaker of the day was the same as the first one. Jonathan Kohl talked about how our urge to be “Agile” can distract us from our mission to deliver software that our customers value, while supporting our team. Agile can distract from skill development too. The term “Agile” has become big business, and lost a great deal of it’s significance. So let’s stop worrying about whether what we do is “agile” or not, and go back to calling it “software development”. As far as I’m concerned, he hit the nail on head. I wouldn’t have minded him talking about this a little longer.

The day ended with drinks in the expo and my attempt at playing a memory game at one of the stands. I kept failing epically. While I was trying to get asleep I found the ideal excuse: my head was already full of things to remember – no room for these trivial button sequences.

December 2

Right before the first keynote of the day I finally met Mika, whom I was supposed to be trackchairing in the afternoon. He was invited as a backup speaker on friday to speak on wednesday, was able to make it, but had to leave immediately after his talk. A true case of hit-and-run guerilla presenting at Eurostar! Naomi Karten then delivered an interesting keynote about “changing how you manage and communicate change”. Her talk was built around the Satir change model. There’s an initial status quo, then a foreign change-inducing element causing a ‘POW’, then chaos, after that an adjustment and in the end a new status quo. When people are confronted with change, they are experiencing a loss of control, and they often react to that in an emotional way. Important: listen, be empathic, regularly communicate the status of the change, even when there is nothing to report. She also used a quote that I well certainly use myself when feeling cornered:  

Hofstadter’s Law: It always take longer than you expect, even when you take into account Hofstadter’s Law”

By then it was time to pay a visit to the Test Lab that was set up by James Lyndsay and Bart Knaack. It was a Eurostar first, and I am actually wondering now why it took so long to have some actual “testing” going on at a testing conference. The software they were running was Open EMR, an open source patient management and appointment book system. What made it even more interesting for me is that I have been testing and working with a similar (not open source, though) system for a long time, so I more or less know what to expect (or what actual users of the software would expect). I paired up with Rikard for a while and found a whole bunch of issues by merely touring the application – we noted them for later reference. It is always nice to pair with fellow testers to see what they focus on, and what their reasoning is. The state of the software under test was something else. It showed some pretty alarming behaviour, and it was far from intuitive or user-friendly.

By then it was time for Eurostar veteran Erik Boelen, speaking at Eurostar for the fifth time already. I’ve known Erik for some time now, and his talks are always entertaining and relaxing in a way. “The power of risk” was his view on how to use a risk-based test strategy that “makes people talk”, like Läkerol. His main message was (apart from the implicit one that testing can be fun *and* will rule the world) that they defined all the risks and used them as entry paths for exploratory testing. For the highest and medium risks they documented their test cases, and for low risks they just reported the results.

After lunch I introduced Mika Katanen (from the university of Tampere in Finland) and his talk about Automatic GUI test generation for smartphone applications. I am totally new to model-based testing and I was impressed with the brief demo he showed. His track went well, and there were a lot of people approaching him for a chat at the end. I do hope that he was able to catch his plane on time. Parallel with this track, Shrini Kulkarni held his talk about software metrics which I was unable to attend. People said it was good – I hope I will be able to see him speak some place else in the future.

Remembering the memory game disaster from the day before, I decided to unfocus for a while – my mind was getting stuck again. I teamed up with some CTG colleagues plus a wildcard named Tom and enrolled ourselves for the quiz that was supposed to take place in the evening. We aptly named ourselves “The Handsome Oracles”, but it wasn’t meant to be. The quiz was canceled later on, so we weren’t able to put the money where our mouth was. We also worked out some testing limericks for the limerick competition – we didn’t win. I thought they were good, but that’s probably just another example of parents not recognizing the ugliness of their own babies. There’s a good joke and an interesting analogy about that hereGitte Ottosen ended the day with a talk about combining agile and maturity models which was chosen best presentation last year in The Hague. I had the impression she was a little nervous – which is completely understandable. I was telling to myself that delivering a keynote for a full auditorium like that sure looked like a daunting task – until I suddenly realised that I would be standing in that same room tomorrow. My unfocused mind started wandering off.

While the temperatures were taking a dive, the Handsome Oracles went into town for dinner. I returned a bit earlier than the rest to rehearse my talk and to get a good night’s sleep while the (by then just plain) Oracles went barhopping. Haha! Life’s good, but not fair at all. 

December 3

The last day of the conference, and people started looking weary. Ray Arell gave us a good wake-up call with his keynote on moving to an agile environment, based on his experiences at Intel. Ray’s a great speaker (and a fun guy too – I might add). He described his hits and misses; the ‘misses’ are often the most interesting parts of experience reports. Lot’s of good advice and some nice puns (Wagile, FRagile, Scrumfalls).

I stayed in the agile track in the big auditorium where John Watkins presented some material from his book on agile testing, aptly named “Agile Testing”. John had gathered case study material from twenty agile projects and proposed agile methods for small, medium, large, off-site, and even off-shore projects. Intriguing, but upon hearing the idea of “agile best practises”, my context-driven genes started to play up.

John was also my great trackchair and introduced me as “Filmstar, Rockstar, Tester!” At least, that was his own juicy summary after I mentioned to him that I had worked as a movie distributor before and had also played in a rock band. Granted, I also admitted playing a zombie once – a serious case of method acting. Anyway, his introduction loosened the audience a bit and I was able to present my track “A lucky shot at agile?” without any problems. I wanted to tell a testing story and I think it went well. I felt at ease (those wireless microphones are really great) and there were many questions afterwards. During the rest of the day people I didn’t know came up to me to congratulate me with the presentation, which was nice. I took a long lunch and had a walk around the expo. I went back to the Test Lab to report the bugs that we found earlier. I didn’t succeed in entering them all, which made me feel kind of guilty – I wished that I would have spent more time there. But I had a hard time choosing. It’s a pity that test labbing also meant skipping tracks as well.

The last regular talk of the conference was held by Rikard Edgren, who is also a Eurostar regular. I had seen his presentation on testing creativity (“Where testing creativity grows”) in 2007 and I liked it a lot, since it is also a subject that is dear to me. There’s far too many people that think that testing is not a creative or challenging activity. This time he talked about  “More and better test ideas“. He promoted the use of oneliners as test ideas – a brief statement of something that should be tested. These test ideas can then be used as a basis for test cases, or as a guideline for other types of testing, or even discarded when there irrelevant or when there is simply not enough time. I think Rikard’s subjects will always be a bit polarizing due to their innovative nature – you either like them or you don’t. I am a believer and it was a good way for me to finish the conference.

I missed the first part of the Test Lab result presentation since they changed the timing and I totally forgot about that. But I got the most important statistics. Over two and a half days, more than 50 bugs were found. My first reaction was: “Only 56? Man, there’s hundreds of them hiding in there”, but then I realised that people had been testing in the lab only for short periods, in between tracks, just as I did. I wonder what would have happened if hundreds of testers had a go at it, all at the same time. Bugfest!

After a short panel discussion with John Fodeh (next year’s programme chair), Geoff thompson, Tobias Fors and Nathalie Van Delft it was time for the award ceremony. Naomi Karten received the Best Tutorial Award and the European Testing Excellence Award went to Anne Mette Hass. In the meanwhile I was dozing off in my not-so-comfy chair – these 4 days of conferencing were finally getting to me. A friendly woman on the stage was mentioning someting about a longlist of papers, and a shortlist, and a final selection of three, containing two Dutch and one Belgian paper. Now wait a minute… how many Belgians sent in a paper? 1…2… before I could make the math, my name was announced as winner of the ENEA Best Paper Award. Two talks at Eurostar, two papers, two awards… what are the odds of that? I was absolutely flabbergasted. That’s actually three in a row for my company CTG, since Bert Jagers won the award last year in The Hague. The pressure is on for next year :-).

I spent the rest of the evening in the hotel bar, where all the testers with an early flight on friday morning were flocking. We ended the day singing an eclectic mix of Irish traditionals, Dylan, early Springsteen and – of course! – Abba, accompanied by a non-certified tester, who plays a mean mandolin. I love Stockholm in wintertime. It was a good Eurostar. Yes sirree.

Agile testing days 2009 – Berlin

A write-up of the Agile testing days 2009 in Berlin.

In october I attented the Agile Testing Days in Berlin. The program committee assembled a really great line-up (see the 2009 programme here). Here is my write-up of the event. A late one, I admit, but it certainly was worth writing about. So without any further ado, here goes… 

October 11

I arrived in Berlin late sunday evening. During the frantic cab ride through the green outskirts of Berlin I had my first conversation with a genuine Berliner. While giving me a quick ‘Berlin for dummies’ round-up, he managed to distract my attention just enough so I didn’t really notice all the near-collisions. I made it to the Seminaris Campushotel Berlin in one piece. Lovely venue, by the way. 

October 12

(c) 2009 Crispin & Gregory

First day of the conference. Quick registration, coffee and off to the first floor where I attended a full-day tutorial by Lisa Crispin, “Using the Agile Testing Quadrants to Cover Your Testing Needs”. There were four other tutorials going on that morning, by Elisabeth Hendrickson, Isabel Evans & Stuart Reid, Tom Gilb and Tom & Mary poppendieck. A great line-up, which made it really hard to choose. But since I had bought and already briefly skimmed through Lisa’s (and Janet Gregory’s) excellent book “Agile Testing: A Practical Guide for Testers and Agile Teams“, I decided to settle with the agile testing quadrants. The day went by really quick, which is always a good sign. The theory wasn’t new, but there were some revealing thought-exercises, like listing all your practises in the quadrants. Visualizing them often makes it very clear if things are missing. She also told a funny anecdote on how her team ‘materialises’ remote team members during meetings and pairing sessions:  

“My team set up a rolling cart for each remote team member, with a laptop, webcam, Skype and mic. My webcam displays on the laptop, and my team members roll ‘me’ around to whoever I’m pairing with, or to meetings (rolling through the halls saying hi to people is fun!) I can control the webcam to look for people.” 

October 13

With a indecent amount of coffee in our systems, the actual conference kicked off with a keynote by Lisa Crispin, “Are Agile Testers Different?”. An interesting keynote, based on ideas that are also described in her book: 

  • In agile projects, the lines between the different roles are blurred. 
  • Testers also need to change their mindsets (seek new ways to improve, be proactive, collaborate) if they want to contribute in an agile team.
  • The value that agile testers add to the team (through continuous feedback, direct communication, simplicity, responding to change, enjoyment).

After that I attended a talk by Ulrich Freyer-Hirtz, about “The Agility GPS”, described by the author as a ‘systematic approach for position fixing of agile projects’, a method to assess your team’s agility. The idea behind it was quite interesting and he already put a lot of work into the model, but it remains unclear to me why anyone would really want to know how ‘agile’ they are, especially considering the fact that the underlying agility model is different for every team/company. The author argued that it could be useful for self-assessment or to unmask alibi-agilistas. The agility GPS is really focused on the agile values, principles and practises and would tell you things like: “you are scoring low on this principle, you need more code reviews”. Interesting, but strange nontheless. According to me, agile practises are mostly context-driven. Apply the practises that give you the quickest return and that work best for you, and stick to them. Discard those that distract and do not add real value. 

Next track was ‘How to develop a common sense of done” by Alexander Schwartz. His main message was that the combination of branching and ‘quality gates’ can be a good way to improve the common sense of „DONE“, and that this will also help testers in getting integrated into agile teams. The most interesting part of the presentation for me was Mayank Gupta’s ‘Done thinking grid’ (read his Scrum Alliance article on a definition of done here). He also mentioned the use of a physical merge token to coordinate all publish merges in the trunk. They used what they called a “merge frog” (the German original Mördsch Frosch sounds pretty scary and is too much of a tongue twister for me) – a merge would only be allowed if a developer had put the merge frog on her/his desk first. 

Next up was a keynote of Elisabeth Hendrickson called “Agile testing, uncertainty, Risk, and Why It all Works”. I had seen some talks of her before on google video, but this ‘live performance’ made it crystal clear: you can’t beat the real thing. She’s really charismatic, a great speaker with a very clear and interesting message. She talked about the four big sources of techical risk (ambiguity, dependencies, assumptions and capacity), the seven key testing practises in agile (ATDD, TDD, exploratory testing, automated system tests, automated unit tests, collective test ownership and continuous integration) and how these practises help mitigate the afore mentioned risks. Simple but sweet. 

We were already deep in the afternoon, but that dreaded mid-afternoon dip didn’t stand a chance. The next track I attended was “Agile Quality Management – Axiom or Oxymoron?” by David Evans, in which he described a number of agile conundrums, some quality principles and a framework. The conundrums that were listed could be interpreted as oxymorons (sentences that combine contradictory terms) as well as axioms ( propositions that are not proven or demonstrated but considered to be self-evident), e.g.:  

  • “Developers know the acceptance tests”. Oxymoron: “They will only write code to make the tests pass!”. Axiom: “They won’t write code that makes the tests fail”.
  • “No test plan”.  Oxymoron: “Failing to plan is planning to fail! How will we know what to test?”. Axiom: “The test plan is implied in the product backlog: everything we build, we test”.
  • “No Test Manager (or Test Management Tool)”. Oxymoron: “So how can we possibly manage testing?”. Axiom: “Testers == team; tests == specifications; they don’t need separate management”.

The framework he described after that listed some interesting items as well. He mentioned ‘applying balloon patterns’, which intrigued me. I liked the metaphor. “Start valid but empty” (an empty balloon is still a balloon, complete the form of a solution before adding a function), “Rubber before air” (don’t deliver functionality that cannot be tested). And a great quote to finish off: “Delaying testing is just incurring quality debt”). 

After that I checked in on “Testify – One-button Test-Driven Development tooling & setup” by Mike Scott from – again – SQS. This was actually the third SQS speaker of the day (after  Ulrich Freyer-Hirtz and David Evans) – they seem to be pretty active in those agile trenches. Mike gave us a quick overview of a tool called Testify, an agile TDD Toolset installer and project generator. The speed with which he was able to set up a new project from scratch and to start writing some unit tests was pretty impressive. 

The keynote from Tom Gilb that ended the first day was bizarre, to say the least. The talk was supposed to be about Agile Inspections, but he talked about old-school inspections, how to perform them (“specifications must be unambigious, testable and must not contain design”) and how these inspections should be primarily used to refuse requirements being handed over to testing because of poor quality. The slides he used to prove his point formed one gigantic style inferno. My eyes started hurting from all these different styles and fonts, overloaded slides and texts being cut off randomly. I still have a hard time understanding why this specific talk was chosen for a keynote at an agile conference, where things are specifically NOT about finding as much defects in the requirements as possible so they can be thrown over the wall again. Agile teams prefer involving the whole team in requirement discussions to filter out all hidden assumptions. I guess this goes to show that you can not just put “agile” in front of some practises and “agilize” the hell out of them. 

When we descended back to the ground floor we were greeted by conference organizer José Diaz – in Lederhosen. The main hall had in the meanwhile been transformed into a genuine Bayerisches Oktoberfest, complete with Hendl, Schweinsbraten, Sauerkraut, Haxn, Würstl und Brezn. And big one-liter-glasses of beer. There was live music, regularly interrupted by the ‘Ein Prosit’-mantra. The whole Bierstube-atmosphere – ok, maybe it was just the beer – really made people talk. It was all great fun and I met and talked to some great people. I even won a prize in a  tombola: a one-year subscription to the Testing Experience magazine

October 14

 The last day was kicked off with a keynote by the godmother of Lean development, Mary Poppendieck: “The One Thing You Need to Know … About Software Development”. She started off by stating that complexity is the enemy of software development. She then gave an overview of ways to divide and conquer complexity, providing a whole lot of software development history in the process. Her natural presentation style made it a really enjoyable talk. More about her presentation can be found here and here.

What followed was – for me anyway – the best session of the conference: Declan Whelan with “Building a learning culture on your Agile team”. His track was stuffed with food for thought, real little gems: pointers, quotes, interesting movies and games – it left me anxious to go discover all those interesting books and websites. The highlights:

  • A quote by Shunryu Suzuki: “in the beginner’s mind there are many possibilities, in the expert’s mind there are few”
  • Bits about Peter Senge’s “The Fifth Discipline” (must read that one!)
  • Virginia Satir‘s change model
  • The principle of Shu-Ha-Ri, a martial arts concept that describes the stages of learning to mastery
  • A moving little video of Gever Tulley talking about “Tinkering School“.

“Tinkering School is a place where kids can pick up sticks and hammers and other dangerous objects, and be trusted. Trusted not to hurt themselves, and trusted not to hurt others. Tinkering School doesn’t follow a set curriculum. And there are no tests. We’re not trying to teach anybody any specific thing. When the kids arrive they’re confronted with lots of stuff, wood and nails and rope and wheels, and lots of tools, real tools… And within that context, we can offer the kids time. Something that seems in short supply in their over-scheduled lives. Our goal is to ensure that they leave with a better sense of how to make things than when they arrived, and the deep internal realization that you can figure things out by fooling around. Nothing ever turns out as planned … ever. And the kids soon learn that all projects go awry – and become at ease with the idea that every step in a project is a step closer to sweet success, or gleeful calamity. We start from doodles and sketches. And sometimes we make real plans. And sometimes we just start building. Building is at the heart of the experience. Hands on, deeply immersed and fully committed to the problem at hand. Robin and I, acting as collaborators, keep the landscape of the projects tilted towards completion. Success is in the doing. And failures are celebrated and analyzed. Problems become puzzles and obstacles disappear.”

Eric Jimmink also stepped up to the challenge of presenting a morning session after a beerfest, with “Promoting the use of a quality standard”. Main ideas I remembered: you should have a Definition of Done on different levels – for tasks, stories, sprints and releases. Revisit the DoD regularly in the sprint retrospectives. Although he looked a bit tired, he managed to get his message across using some excerpts of his and Anko Tijman’s book “Testen 2.0”, a great Dutch book on agile testing that was launched a year ago at Eurostar 2008. But although I am a native Dutch speaker, I find it hard to read books on testing in Dutch. I have always felt that English is the most natural choice within the testing community. Many of these translated terms just don’t sound right. Anyway, that’s probably just my silly Belgian self – I’m pretty sure all those Dutch testers out there don’t mind.

After lunch, Stuart Reid talked about skills needed in agile teams. His keynote “Investing in individuals and interactions” focused on the first statement of the agile manifesto. He showed a formula to calculate the job satisfaction (MPS=Motivating Potential Score) from Hackman & Oldham, which was interesting.  There also was this nice analogy about pairing, where you normally have the roles of “driver” and “navigator”. The driver is the one who is learning and the navigator has the expertise and can transfer the knowledge. But which do you think is the safest option when flying an airplane?  A senior pilot flying, the apprentice watching? or the other way around? Actually it’s the latter. The young pilot wouldn’t dare to criticize the older pilot when he sees a mistake, while the older pilot will be much more alert when he lets the younger pilot take control.

The last track I attended was “Agile practices in a traditional environment” from Markus Gärtner. He presented an experience report of how they started using some agile practises (test-driven development, exploratory testing, agile planning, improved communication) without actually using the term agile. They also used the testing quadrants to visualise where the current approach was lacking – similar to the exercise we actually did in Lisa Crispin’s tutorial two day earlier. This helped them to move their efforts more in the direction of business-facing automated tests, with the additional risk of neglecting the technology-facing tests. He seemed pretty nervous when starting his talk, but there was no reason to be – he had a great story to tell and he obviously knows what he is talking about.

By then it was time for me to leave for the airport. I missed the panel discussion that ended the conference, though I heard that many of the keynote speakers had already left. For a first time conference, the event was really well organised, cosy and well thought-out. A very nice and familiar atmosphere between attendees and speakers as well. Next year’s call for papers is open. Do send your abstracts to José, and maybe you’ll get a chance to see him in Lederhosen as well.