The busy month of May offered me another opportunity to practice sketchnoting: the wonderful Let’s Test conference in Runö – Stockholm. The conference was, as always, fantastic. A good number of the sessions were experiential workshops that invite participation rather than sketchnoting (although I might make summary notes on them later on), but there was still enough goodness to get drawing.
I am growing fond of the live noting of conference talks – with the unspoken rule of posting it on twitter before the last attendee leaves the room. I very much like the live time-pressure aspect of it since it doesn’t allow my evil perfectionist side to take over. My DEWT-friend and colleague Ruud Cox has gently tried to nudge me into two-stage sketchnoting or even non-live sketchnoting (books, recorded talks). As my conference calendar is fairly empty for the rest of the year, maybe I’ll have a go at that.
One of my personal goals for this year is to become better at sketchnoting. And becoming better – for me – means: working at expanding my visual library and – more importantly – practice, practice, practice!
Last week I attended StarEast in Orlando, and testing conferences happen to provide a wonderful opportunity to get some of that much needed practice in.
Armed with a Confidant notebook (a new brand of notebooks by Baron Fig) and a set of sharpie pens, I went to work. Apart from the Selenium and Webdriver tutorial by Alan Richardson (hands-on coding does not match well with equally hands-on drawing), my keynote and the one preceding it (duh!) and a presentation about game testing of which I missed the first ten minutes, I took notes for every single talk I attended.
I like taking notes this way – the timed challenge provides me with much needed focus (no zoning out here), and the notes seem to stick way longer. As a bonus: even if the content does not particularly appeal to me, the added note taking practice still makes it worthwile.
The software testing profession looks like a steadily maturing profession from the outside. After all, there are certifications schemes like ISTQB, CAT, IREB and QAMP (the one to rule them all), standards (ISO 29119) and companies reaching TMM (test maturity model) levels that – just like a Spinal Tap guitar amplifier – one day might even go up to 11. The number of employees that companies send off to get certified in a mere three days is soaring, and new certification programs are being created as we speak. Quick and easy. Multiple choice exams for the win!
The reality, however, is that the field of software testing is torn between different “schools” of testing. You could see these schools as determined and persistent patterns of belief, speech and behaviour. This means that different people – all calling themselves “test professionals” – have vastly different ideas of what testing is all about. Even something elementary as the definition of testing varies from “demonstration of fitness for purpose” to “questioning a product in order to evaluate it”, depending on who you talk with (for more info on the schools of software testing, I heartily recommend Brett Pettichord’s presentation on the subject).
And so it happens that different people think differently about “good” or “mature” software testing. I, for one, don’t believe in tester certification programs, at least not in the format they are in now and the way they are being used in the testing profession. The current business model is mainly designed to get as many people as possible certified within the shortest timeframe. Its prime focus is on certifiability, not on tester skill, and certainly not on the advancement of the craft. Advancement comes from sharing, rather than shielding.
Rapid Software Testing (RST)
So what are the options for a tester on a quest for knowledge and self-improvement? What is a budding tester to do?
I think there are valuable alternatives for people who are serious about becoming a world-class tester. One of these is Rapid Software Testing (RST), a 3-day hands-on course designed by James Bach and Michael Bolton.
Actually, calling this “a course” doesn’t do it justice. RST is at the same time a methodology, a mind-set and a skill set about how to do excellent software testing in a way that is very fast, inexpensive, credible and accountable. It is a highly experiential workshop with lessons that stick.
How is RST different?
During RST you spend much of the time actually testing, working on exercises, puzzles, thought experiments and scenarios—some computer-based, some not. The goal of the course is to teach you how to test anything expertly, under extreme time pressure and conditions of uncertainty, in a way that will stand up to scrutiny.
The philosophy presented in this class is not like traditional approaches to testing, which ignore the thinking part of testing and instead focus on narrow definitions for testing terms while advocating never-ending paperwork. Products have become too complex for that, time is too short, and testers are too expensive. Rapid testing uses a cyclic approach and heuristic methods to constantly re-optimize testing to fit the needs of your clients.
What’s in it for you?
The ability to test something rapidly and skilfully is critical. There is a growing need to test quickly, effectively, and with little information available. Testers are expected to provide quick feedback. These short feedback loops make for more efficient and higher quality development
Exploratory testing is at the heart of RST. It combines test design, test execution, test result interpretation, and learning into a seamless process that finds a lot of problems quickly. Experienced testers will find out how to articulate those intellectual processes of testing that they already practice intuitively, while new testers will find lots of hands-on testing exercises that help them gain critical experience
RST teaches you how to think critically and ask important questions. The art of questioning is key in testing, and a very important skill for any consultant
RST will provide you with tools to do excellent testing
RST will heighten your awareness
Bold claim bottom-line:
RST will make you a better tester.
RST comes to Belgium
Co-learning and Z-sharp are proud to announce that from 30 September – 2 October, Michael Bolton will visit Belgium to deliver the first ever RST course on Belgian soil, giving you the opportunity to experience this unique course in person. More info can be found here, or feel free to contact us for more info.
Brace yourself for an mind-opening experience that will energize and boost your mind.
Early tuesday, and my dreams were filled with empty auditoria and keynote speakers stuck in airports. A first reality check eases my mind a bit. The people I dreamt about are already checked in, and ready to rumble. As am I. I skip breakfast to be on site as early as possible, but halfway there I realize I left my phone in the hotel room. I hurry back and when I finally get to the premises, James and Julian beat me to it. While we help the tutorial speakers get all settled for the morning, the registration area is again being flashmobbed by testers. Delegates are now flowing in at a steady pace, but I notice remarkably few hiccups. Sure, there is the occasional delegate who is worried about his tutorial enrollment, but Siobhan seems to have a firm grip on payments and registrations. Siobhan handles all the adminstrative stuff throughout the year, a job that can never be underestimated. Rumor has it that she can even make Chuck Norris comply with Eurostar’s presentation materials deadline (which we couldn’t verify this year since his submission “I sit down in stand-up meetings” didn’t make the cut). The Eurostar team deals with the rush-hour queues swiftly, and before I can say “Morning coffee, anyone?”, the AM sessions are kicked off.
Photo by Randy Rice
Like yesterday, I wander around the now quiet and peaceful venue and do a temperature reading in the different tutorial rooms. Fiona Charles has everyone in her room up on their feet, milling around and fully engaged in her “Right-sizing Test Documentation”. Paul Gerrard is testing the room capacity boundaries in his totally sold out “How to Create a Test Strategy”. Randy Rice – all the way from Oklahoma City, Oklahoma – is mighty popular with his “Free and Cheap Test Tools”. Michael D. Kelly – all the way from Indianapolis, Indiana – is spreading his wisdom about managing exploratory testing for a full house as well in his “Session Based Test Management”. When I check in with Alan Richardson’s and Simon Stewart’s Selenium Clinic, I see a screens full of code and two presenters on fire. All is good.
Again I feel sorry that I cannot sit in – imagine all the learning I’m mssing! – but as it turns out there is plenty of learning to be done elsewhere: rendez-vous at 11 AM in the main auditorium for a run-through of the conference opening. Finally, the auditorium is made Shrek-free and we can admire the beautiful – blueish – theatre. While the main idea of the run-through was a rehearsal of the opening remarks, we spend an hour test-driving four yellow tourist bikes through various aisles of the auditorium, checking where to brake and how to park – testing the risky bits first, so to speak. Daragh and Paul of the Eurostar team were able to secure these bikes last-minute as we are going to use them in the most vital part of the conference opening: the committee entrance. While James, Julian and Shmuel get in touch with their inner 13 year-old and do little uphill races on the blue carpet, ringing frantically, I try to make my slideset work on Alan Page’s nifty little Surface RT. Hungry? Not really. No stress, no sirree.
Half past one. The official conference opening is upon us. The auditorium is packed, music is blasting out of the speakers and the four of us are in the – by now deserted – registration area, using our dubious trial bike skills to balance our yellow monsters in place. Lorraine gives us a go and we make our way to the front of the auditorium, all the time aware of the bizarreness of the situation: low visibility, a steep decline, funny brakes, loud music and a suit don’t make for a fluent biking experience. This concludes our very own “Men in Black”/”Boys are back in town” moment – and I’m glad we make it in one piece. We park the bikes and I climb onto the stage for the opening remarks. It is there that I have my first aha-moment. When asking how many people are attending the conference for the first time, I expect someting like 10-20% but see a *lot* of hands going up. I am a little thrown by that and I would like to see some official numbers to be sure what happened this year.
Photo by Wired Magazine, 2012
I go ahead and explain the conference theme (Innovate/Renovate), telling the story of Ferran Adrià, the former chef from El Bulli, the best restaurant in the world for some time. Adrià started his career as a dishwater but managed to change the world of gastronomy by bringing elements of other disciplines into cooking: chemistry, psychology, physics. He expanded his cooking toolbox (his new toys were lyophilizers, liquid nitrogen, candy floss machines,…) and started investigating how the presentation of a dish influenced the perception of taste (did you know that strawberry mousse is perceived to be ten percent sweeter when served on a white plate compared to a black one?). My main message? In testing, it’s probably the easiest easy to say that innovation is not your job, rather something for those crazy boys and girls in R&D. But it’s not – it’s everyone’s job. Innovation is not just about products: it is also about business practices, processes, tools – it lies in everything we do. Everyone can be an innovator – testers too.
When the committee takes the stage for a personal address, expressing their wishes and hopes for the conference, I can actually stand back for a while and be amazed by the size of the whole endeavour. So many people here, in difficult economic times, all undergoing geographical and financial inconvenience to be here to learn and share experiences – this is great, and humbling at the same time. It is super tuesday alright.
Photo by Rik Marselis
Time now for Alan Page to step into the light, with his opening keynote “Test Innovation for Everyone” (a link to the presentation can be found here). It turns out to be a great talk, in which he points out that innovation is all about ideas, which makes test innovation mainly about test ideas. We innovate to solve problems – but are we solving the right problems? Try a lot, but keep checking whether you are doing the right thing. Alan’s talk is also book recommendation hour: “Where Good Ideas Come From” (Steven Johnson), “The Wisdom of Crowds” (James Surowiecki), “The Lean Startup” (Eric Ries), “Jimbo – Adventures in Paradise” (Gary Panter), “Brain Rules” (John Medina), “The Myths of Innovation” (Scott Berkun), “They All Laughed” (Ira Flatow), “Steal Like an Artist” (Austin Kleon), “The 5 Elements of effective thinking” (Edward Burger, Michael Starbird) and “The Innovator’s Dilemma” (Clayton Christensen). Some of these are already acquired as we speak.
Next up are the first track sessions of the conference. Finally, time to watch the sessions we have been debating way back in march. I switch to track-hopping mode again and sample bits of as many tracks as possible. The afternoon flies by way too fast, and before I know, we are all inside the auditorium again for Alan Richardson‘s eclectic closing keynote, “Unconventional Influences”. I am very much looking forward to his talk – I chose this talk from his long list of possible subjects because I am all for bringing elements from other fields into our testing practice. And I am happy to see that Alan completely nails it with references to Dr. Seuss, H.P. Lovercraft’s “The Call of Cthulhu”, the “Fortean Times” magazine and ghostbusting. He tells a story about how he never allowed his testing to be limited by other people’s attitudes to testing or company mandates. He thinks that is an excuse that people embrace to stop them having to identify their beliefs about testing and challenge themselves to become better testers. Slides of his talk can be found here. He also wrote an unconventional paper to go with it, link here.
Tuesday evening is the traditional Expo drinks moment, and I take the opportunity to mingle and talk with as many people as possible. I drop by the Improve boot, where my very pregnant DEWT colleague Jeanne Hofmans hands me the first issue of “Quality Level Management – Managing quality in outsourcing”, a book she wrote together with Erwin Pasmans. Once I make it to the Test Lab, I see that Shmuel brought his infamous box with testing games and waste the rest of the expo drink time trying to solve puzzles together with great testers – and beer. How awesome is that? Totally sucked into puzzle solving, I fail to notice that the expo is closing, and together with Shmuel (still limping like a hunchback) I spend a good 30 minutes deciding whether we should ride our yellow bikes through the torrential rain or take a taxi. Taxi lines lines outside the RAI are rather discouraging, so we end up getting soaked on the bikes. Back at the Novotel, the Test Lab people set up a portable Test Lab in the bar, but even the most die-hard TestLabber gets hungry after a while. And so it happens that a group of twentysomething people find themselves in a great tapas restaurant for an evening of comido, bebidas y alegría. The athmosphere was so relaxed that I am tempted to stay up until the wee hours, but I am being a good chair and take the taxi back – mañana es otro día.
Here are Esther Gons‘ graphical recordings of this day:
En route from the novotel to the RAI on this dark and rainy monday morning, I am pleasantly surprised to see “Eurostar Conference” signposted everywhere (next to Shrek the Musical, where it belongs) – who knows I might have ended up at a conference far, far away. It is only 7 AM, but the RAI is buzzing with activity. The registration desk is fully (wo)man(n)ed, and last minute checks are being done. Not too long before the hounds will be released, and the eurostar crew is very much “in the zone”. I decide not to disturb them too much and go for a little orientation walk around the venue. The main auditorium remains closed this day (secret Shrek stuff, I assume) and the expo is still being built, but I am able to get a good idea of the venue layout. Loungy sitting areas, roomy but cosy session rooms – this place has the right vibe.
By the time I get back to the desk, the first tutorial speakers are already registering. Part of my job today is to show them to their rooms, make sure they get all settled and have everything they need. Dorothy Graham (“Managing Successful Test Automation”) and Janet Gregory (“Transitioning to Agile Testing”) are our first teachers on site, soon followed by Bob van de Burgt & Iris Pinkster O’Riordan (“Lessons Learned in Test Management”) and Rikard Edgren (“Exploratory Test Design”). I walk them up to their campsite for the day and when I get back down, the registration area looks like it is being flash-mobbed by multiple nationalities – minus the dancing. The desk is invaded with people waiting to get a name tag, conference bag and that very slick looking 20th-anniversary running shirt. They get a complimentary friendly word and welcome from the Eurostar crew, some advice and orientation, and off they are to learning heaven. By now I realize that I somehow missed Michael Bolton (“Critical thinking for testers”) entering the venue, but Siobhan reassures me that he /is/ in the house. I go upstairs again to say hi, and notice that Michael’s room is nearly full. Wait, is it 8:30 already? The last queues are cleared, and we have a lift off.
Standing by the registration desk, I notice keynote speakers Alan Page and Simon Stewart check in. As they are not teaching/speaking today, and they probably won’t be hanging around the venue the whole time, I make sure to remind them of the speaker’s drink taking place this evening in a bar behind the RAI. Apparently, I wasn’t the first:
The rest of the morning I spend wandering from room to room, sitting in for short periods of time, to catch the vibe and to see whether the delegates are enjoying themselves. This is a strange change of perspective. During the previous years, I was always in these tutorials myself, focused on learning. Now, I am checking the reactions of people and not really paying attention to what is being told. I am a lousy multi-tasker. Not that I don’t pick up stuff – I vividly recall Rikard walking around without shoes, talking about software potatoes. I catch Bob and Iris talking about a thin (cheese) slicing test management method (mmm… cheese), and Michael debriefing one of his many exercises and quoting Jerry Weinberg (*). I hear Dorothy highlighting and explaining stories from her award-winning book (“Experiences of Test Automation“), and Janet is scaling her normally much more intimate workshop to a way bigger audience – and she seems to handle that with style and grace.
The rest of the RAI is empty at this time of day, which makes for a strange contrast with the invasion of only an hour earlier. During the coffee breaks, the friendly chatter reappears, and I observe and talk. Finally, I bump into Shmuel, who is limping like someone with bad shoes who had a very long walk in Amsterdam on a rainy sunday. It adds to his overall funniness, although I think he already is funny enough as is. We discuss the subject of yellow tourist bikes to go back and forth between the Novotel and the RAI – bikes that will play a role in our conference opening as well.
Right before lunch, as I am welcoming delegates into the lunch restaurant, fellow committee member James Lyndsay enters the venue. He is sporting a big bag that mostly contains a stylish – and heavy – kilt. Guaranteed gala dinner goodness. During lunch, James is mentioning his upcoming gig with the London Bulgarian Choir – the friday after the conference. And I thought /I/ had a hectic schedule! When the tutorials start up again and the restaurant is emptying, Shmuel and I act as a (poor man’s excuse for a ) choir while James rehearses one of his deep-voiced solos, leaving the waiters and RAI staff wondering what they just witnessed. The RAI restaurant has good acoustics, actually. Rumor has it that James’ voice is still haunting the premises.
The afternoon flows smoothly, I feel, and around 4 pm I follow Lorraine’s recommendation to use the last couple of hours of the afternoon to relax a bit before things get really hectic tomorrow. The weather is crisp and clear, so I go for a walk around the area. Not too long though, since the speaker drinks kick off at 6.
At the drink, which turns out to take place in a cosy Austrian log cabin, I meet Julian Harty who just flew in from Nairobi with a short detour to the UK – to switch from summer to winter clothes. On thursday evening, he has to move on to go speak at the (equally fantastic) Oredev conference, which unfortunately takes place in the same week this year. Speaking of hectic schedules, I am convinced that Julian’s travel arrangements would make Kofi Annan look lazy. It is good to have our committee finally complete, and on site. Actually, this is the first time that the four of us together meet face to face, as Shmuel was skyping in while the rest of us were meeting in Galway in march.
As the committee is hosting the drinks, we do our best to make everyone feel welcome. Shmuel goes full reversed paparazzo and has his picture taken with everyone present – behavior that he will continue to exhibit throughout the whole conference, which makes me wonder: doesn’t that make his photo albums mighty Shmuel-centric? The athmosphere is really relaxed, and I am glad to see things turning out so nicely. I am now able to put faces to submissions, and voices to pictures. Michael D. Kelly – very thrilled to finally meet him, by the way – looked so young that I didn’t even recognize him at first. I don’t know why I had imagined him older, must be his reputation preceeding him.
My evening ended in a tasty Indonesian restaurant, where a large group of testers was diving into a rice table as we entered. I was invited to eat (heaps of) spicy leftovers from other people (thank you Rob Lambert and John Stevenson for feeding the hungry and the impatient – your good Samaritanism is highly appreciated). Again, occasions like this work wonders in putting faces to twitter handles: be warned, @GeirGulbrandsen and @Kristoffer_Nord, you’re no longer safe from me.
After dinner : plenty of rest for the wicked. Tomorrow, there’s an official conference-opening to be done.
… to be continued
——————————–
(*) Which comes as no surprise, as Jerry Weinberg – Patron Saint of thinking testers – is very much worth quoting. And reading, even more so. If you haven’t read any of his books, I encourage you to do so.
It has been a while since my last blog post, and being the programme chair for Europe’s biggest software testing conference probably had something to do with that. Now that the twentieth edition of Eurostar is over and the whole event is still very much in my system, I figured it is about time to revive Ye Olde TestSideStory blog.
The Eurostar office, Galway
The whole year leading up to this moment was one big trip into testing conference wonderland. I learned loads about conference-making (I’m pretending that this is a dictionary entry somewhere) in the small and the large. Selecting a committee, a theme, keynotes, tutorials, assembling a balanced programme out of 400+ submissions – these things in itself already were quite a challenge. This, combined with a steady flow of related side-activities proved to occupy the better part of my free time. Luckily, the Eurostar team in Galway (Ireland) made this into a very enjoyable and fluent experience. I had the privilige of visiting the Galway office a couple of times in the past year, and the team has a great energy that gets things going (and a love for Belgian chocolates and all things Guinness). Props to my employer CTG as well, for giving me the opportunity to spend time preparing the conference.
Working with my committee (Julian Harty – James Lyndsay – Shmuel Gershon) throughout the year was certainly a highlight. I have fond memories of our lengthy skype sessions, discussing about anything in the testing conference realm – we even managed to find some emerging behavior in skype chat in the process. In hindsight, I was particularly impressed with Julian’s pragmatism and fresh ideas, James’ note-taking fu in the face of a truckload of submissions, and Shmuel’s contagious enthusiasm.
The last weeks, pressure had been building gradually: seeing the early bird subscriptions take off, hearing about testlab preparations, tutorials filling up… Later on, a couple of speakers opted out and needed replacement – things were getting more real every week.
Rainy Amsterdam – Sunday November 4
After some uneventful aquaplaning all the way from Belgium, I met up with Israeli-Brazilian superstar (and programme committee member extraordinaire) Shmuel Gershon. Originally there was a visit planned to the RAI to get acquainted with the venue layout, but since Eurostar happened to coincide with Shrek The musical (Ogres in the main auditorium! Fionas mindmapping a test strategy!), this was no longer possible. We decided to dive headfirst into the city of Amsterdam, to explore. Some observations:
A couple of hours in Amsterdam can spawn more rain than six days in Ireland
Torrential rain will soak up even the sturdiest shoes
The Anne Frank house has bigger lines than the newly opened Amsterdam Apple Store
From now on, if the map and the territory disagree, I’m believing the territory
Serendipitous wandering can make you end up in one of the finer Indian Restaurants in Amsterdam
The finer Indian bread is very kosher – but expensive
Two men with identical bright blue Novotel umbrellas look funny (I guess people expected a Gene Kelly dance routine)
When arriving back at the Novotel, soaked to the bone, a bunch of testers had already gathered for an informal meetup in the bar. I was planning to change into dry clothes first, but got engaged in conversation and totally forgot about it. Sometimes you have to plan as you go along.
Conference pre-opening (photo by Huib Schoots)
While my shoes were drying slowly, I spent the rest of the evening chatting with new friends (Cyril Boucher, Jeanne Peng, Erkki Pöyhönen) and catching up with old ones (John Stevenson, Michael Bolton, Huib Schoots, Jean-Paul Varwijk, Rikard Edgren, Shmuel). John in particular was on fire that evening, quoting book titles like some kind of human reading tip generator. The two that I managed to note down are “The click moment” and “Everything is obvious“. The rest got lost in a pre-conference haze.
Later on I ran into the Eurostar crew as well. They had been on site since friday, unpacking stuff and basically building everything from scratch. They expanded their team for the conference, and it was nice meeting new faces there too. They all looked happy and confident, which was kind of reassuring to see: the logistic side is under control. Chatting with them also made me realize that things were about to be kicked off for real.
Are those nerves I feel? Anyway, time for bed – appointment at the RAI at 7 am.
This is the transcript/elaboration of a lightning talk I did at the Context-Driven theme night organised by TestNet in cooperation with our Dutch Exploratory Workshop on Testing (DEWT).
The hardest part
There comes a moment in the career of a context-driven tester where he is bound to have a sobering epiphany: for every situation where he knows the right approach, for every situation where he knows the perfect tools for the job, he comes to realize that there are numerous contexts where that approach isn’t the most appropriate one, where his ‘best practices’ are not usable. Or maybe they *are* usable, but they lead to suboptimal results.
Maybe I shouldn’t generalize. This is how it happened to me at least, and that was the moment when I became aware of the main principle of Context-Driven Testing: how you approach things is driven by the context of your project, not by your process. That is also the main difference with the common methodologies that try to replicate the same process over multiple contexts.
I admit that was a source of frustration for me. Going context-driven is certainly not taking the easy road – it would be much easier to implement the same processes everywhere I go. But none of that. Luckily, it happens to be the most exciting and challenging road I know.
Knowledge and awareness
What can we do to arm us against all these different changing contexts we find ourselves in? Gather knowledge, get experience, learn. Make your tester toolkit – with all your techniques and tips and tricks in there – as big as possible . That way you’ll be able to pick the right approach at the right moment.
Talk to fellow testers as much as possible. Grab every opportunity to network with them. Twitter is a blessing for these things – it has rocked my world and continues to do so. Conferences are hotspots for fascinating people, national and international alike. Events like this Testnet theme night are golden, they really are. Learn and read continuously. About testing, for sure, but also about other disciplines. I think there are many lessons to be learned from eg. psychology, sociology, philosophy and science. Some of the sharpest testers I know are physicists and philosophers!
Try out new ideas, new stuff, new approaches that you haven’t tried before. Apart from it being more fun, Marie Pasinski (staff neurologist at the Massachusetts General Hospital) recently wrote that studies have shown that engaging in novel, stimulating activities promotes the growth of new neurons in the hippocampus (1). Putting this to work in your everyday life can be as simple as trying out a new recipe, taking a different route to work, reading up on the newest technology trends, or meeting new people.
Widen your awareness. Keep eyes and ears open, at all times. Absorb everything, as if you were a sponge. Are you familiar with the phenomenon where you happen upon some obscure piece of information – often an unfamiliar word or name – and soon afterwards encounter the same subject again, often repeatedly? It sure has happened to me, and it has a name as well: the Baader-Meinhof phenomenon. However strange it may seem, it is not that illogical. It just proves that our brains are fantastic pattern recognition engines. This is a characteristic that is highly useful for learning and I think we can use this to our advantage. The more we are aware of things surrounding us, the more we absorb knowledge, the higher the chance that it will keep lingering in our subconscious, and the likelier that a piece of knowledge will surface – and will stick in memory – when we need it.
The importance of knowledge in the context-driven community cannot be overestimated. That is the reason why there are so many initiatives for sharing that knowledge: free coaching by experts, peer workshops (DEWT, anyone?), tester meet-ups, weekend testing, Pair / Learn / Present, blogs, lots and lots of course materials available online…
The context-driven community is a very open community that focuses on sharing. If you are curious and want to know more, do get in touch. We’re more than willing to help where we can.
(1) Marie Pasinski – Beautiful Brain Beautiful You, 2011
One month ago, my oldest daughter (6) started taking on rope skipping. The last time I had seen her practising, two weeks ago, she was still having trouble getting the rope neatly over (and under) herself, but yesterday she was able to complete several jumps in one go in a fluent movement. It was the first time I had seen her do that, so I was pretty impressed.
She was clearly in learning mode. I sat down to observe her more closely.
– “Wow, where did you learn all that?”
– “I’ve been watching older girls do that in school, daddy. Watch”.
She started jumping and counting out loud.
– “One, two, three, four, five, six, …”
She tripped on the rope.
– “Woohoo! Six!”
– “You go, girl!”
– “Again! One, two, three, four, five, nooooo…”
– “Five is good”.
– “No, daddy, five is not good. Again!”
She repeated the process a couple of times. She jumped seven (“Yes!”), four (“Nooo!”), five (“Pfff!”), six (“Yippie!”). I started noticing a pattern. It struck me that she alternated frustration with joy, and she let it depend on the number of jumps. Time for some questioning.
– “Why are you happy with anything above or equal to six, but unhappy with anything lower?”
– “It has to be at least six, daddy”.
– “Why six?”
She seemed really annoyed that I didn’t see her point. She thought I was pulling her leg.
– “Because I’m _six_ years old, daddy. Didn’t you know? What else could it be?”.
I was totally flabbergasted. She managed to impose some totally arbitrary pass/fail criteria on herself. Where did that come from? I thought that using pass/fail tests actually sabotages kids’ natural learning processes? But this appeared to come out of herself. No-one told her that she had to make at least six.
I wondered – maybe she just chose her age as a starting point, just to set some initial learning goals for herself? Was she planning on raising the bar later on when reaching six would have become too easy? Unfortunately I didn’t have the chance to follow-up on that – lunchtime!
Flash forward to work. All this reminded me of commonly defined pass/fail criteria such as
“90% of all tests must pass”
Really?
In “Are your lights on?”, Jerry Weinberg uses the well-known “Mary had a little lamb” nursery rhyme to show how a seemingly straightforward statement is prey to multiple interpretations, depending on which word you emphasize. An invaluable heuristic when looking at requirements. Why not try that on the familiar pass/fail criterium stated above?
– “90%”? What if the tests that would have revealed some serious errors happen to be in that 10% you so confidently dismissed? Why not 89 or 91?
– “All”? You know “all” possible tests that can be performed? Are they all documented? Some of them might still be residing in your head. What if in the meanwhile we performed some more important tests that revealed serious risks? Are these tests part of “All”?
– “Tests”? Do you only count scripted tests, or do you also take exploratory ones into account? What about important usability issues some users might have found? Or acceptance test checklists? Or automated checks?
– “Must”? What if not all 90% passes? Does this mean your solution is without value? The customer might value other things than you do. Is it up to you to decide how much value is in there?
– “Pass” ? What about behavior that is totally acceptable for your client, but that we find annoying? Pass or fail? What about tests that pass all steps, but that reveal important problems as a side-effect? Sometimes a test’s pass/fail decision is not binary.
My daughter went to school this morning and – for the first time – took her own jump rope with her. I wonder how many % of her rope jump cases will pass this time.
We are testers. We communicate, collaborate and report. We are the headlights of a project. We light the way, break the news, deliver messages – and they’re not always pretty. How do we make sure our point is taken? It all boils down to adapting our style of delivery depending on the content and the audience. Context is indeed everything.
The delivery of a speech is mostly done through the nonverbal channel (whereas the content itself is verbal). This includes all speech elements other than the words themselves: eye contact, voice, articulation, gestures, facial expressions, body language – even appearance. Using all of these effectively requires timing and practice, lots of practice. An additional problem is that there is a wealth of delivery styles to choose from. How do we get the message across in the best possible way?
A really smart person once said: “Smart people learn from their mistakes, but REALLY smart people learn from other people’s mistakes.”
I think that’s true. In the video below, councillor Phil Davison – a candidate for the position of Republican treasurer for Stark County, Ohio – delivers a passionate nomination speech in front of a receptive audience. His delivery style is – well… – pretty peculiar.
How Feynman’s take on naming things is applicable to testing
Feynman’ s father Melville played a big role in shaping little Richard’s way of thinking. He used to read bedtime stories from the Encyclopedia Britannica.
“See this Tyrannosaurus Rex over here? It says here that this thing was 25 feet high, and the head was six feet across. That means that if it stood in our front yard, it would be high enough to put his head through the window, but not quite because the head is a little bit too wide, it would break the window as it came by”.
He always tried to translate things into some kind of reality, so little Richard would be able to figure out what it really meant, what it was really saying.
Melville would also take his kid for long walks in the Catskill mountains, telling him about nature and explaining that in order to really *know* something, you should start observing and noticing instead of merely naming (a thing most of his classmates seemed to do):
“You can know the name of a bird in all the languages of the world, but when you’re finished, you’ll know absolutely nothing whatever about the bird… So let’s look at the bird and see what it’s doing — that’s what counts. I learned very early the difference between knowing the name of something and knowing something.”
From “The pleasure of finding things out” (1981)
I think the above quote illustrates a phenomenon that occurs all too often in software testing: the nominal fallacy. Basically, this means applying a label or name to something and thinking you have explained it.
What about boundary value testing (or domain testing), for instance?
“First, we identify the boundaries, then we identify tests at each boundary. For example, one test each for >, =, <, using the first value in the > range, the value that is equal to the boundary, and the first value in the < range”.
A pretty straightforward and effective technique, right? We think we master it, until we realise that most textbooks are only talking about known and visible boundaries. What about the boundaries that are not known, not even by the developers that wrote the code? Most of the time, the software’s complexity stretches far beyond our imagination.
In case you need convincing: let’s revisit the ParkCalc exercise that took place a couple of months ago (here‘s a good write-up of that event by Selena Delesie). Matt Heusser put out a challenge to test ParkCalc, a small app on the Grand Rapids Airport website to calculate parking fees. The site provided entry and leaving times, date pickers and five different kinds of parking to choose from. Quickly, an instant test-flashmob formed via twitter. Testers from all over the globe jumped on the bandwagon. James Bach joined in and challenged everyone to beat his highest total parking fee. Extreme testing ensued. What followed was a good exercise in factoring, investigation and on-the-fly test design. And it happened to illustrate the complexity of boundary value analysis as well.
To get an even better idea of this complexity, there’s always Cem Kaner‘s online course on domain testing. Do we know boundary value analysis because we know its most basic definition?
I’m not trying to open Pandora’s box here, but these nominal fallacies also apply to a testing certification that mainly focuses on definitions. Naming things isn’t enough. As Feynman put it: knowing something requires practice and observation. The benefit? No need to memorize anything anymore when real understanding kicks in.
Of course, all this isn’t new. Half of the starr-cross’d lovers were already aware of this, way back in the late sixteenth century:
“What’s in a name? That which we call a rose
By any other name would smell as sweet.”
En route from the novotel to the RAI on this dark and rainy monday morning, I am pleasantly surprised to see “Eurostar Conference” signposted everywhere (next to Shrek the Musical, where it belongs) – who knows I might have ended up at a conference far, far away. It is only 7 AM, but the RAI is buzzing with activity. The registration desk is fully (wo)man(n)ed, and last minute checks are being done. Not too long before the hounds will be released, and the eurostar crew is very much “in the zone”. I decide not to disturb them too much and go for a little orientation walk around the venue. The main auditorium remains closed this day (secret Shrek stuff, I assume) and the expo is still being built, but I am able to get a good idea of the venue layout. Loungy sitting areas, roomy but cosy session rooms – this place has the right vibe.
in Test Management”) and Rikard Edgren (“Exploratory Test Design”). I walk them up to their campsite for the day and when I get back down, the registration area looks like it is being flash-mobbed by multiple nationalities – minus the dancing. The desk is invaded with people waiting to get a name tag, conference bag and that very slick looking 20th-anniversary running shirt. They get a complimentary friendly word and welcome from the Eurostar crew, some advice and orientation, and off they are to learning heaven. By now I realize that I somehow missed Michael Bolton (“Critical thinking for testers”) entering the venue, but Siobhan reassures me that he /is/ in the house. I go upstairs again to say hi, and notice that Michael’s room is nearly full. Wait, is it 8:30 already? The last queues are cleared, and we have a lift off.
The rest of the RAI is empty at this time of day, which makes for a strange contrast with the invasion of only an hour earlier. During the coffee breaks, the friendly chatter reappears, and I observe and talk. Finally, I bump into Shmuel, who is limping like someone with bad shoes who had a very long walk in Amsterdam on a rainy sunday. It adds to his overall funniness, although I think he already is funny enough as is. We discuss the subject of yellow tourist bikes to go back and forth between the Novotel and the RAI – bikes that will play a role in our conference opening as well.
At the drink, which turns out to take place in a cosy Austrian log cabin, I meet Julian Harty who just flew in from Nairobi with a short detour to the UK – to switch from summer to winter clothes. On thursday evening, he has to move on to go speak at the (equally fantastic) Oredev conference, which unfortunately takes place in the same week this year. Speaking of hectic schedules, I am convinced that Julian’s travel arrangements would make Kofi Annan look lazy. It is good to have our committee finally complete, and on site. Actually, this is the first time that the four of us together meet face to face, as Shmuel was skyping in while the rest of us were meeting in Galway in march.
Test Side Story 