Tag: Software Testing & Quality Assurance

My Eurostar 2014 closing keynote

I had the privilige of delivering the closing keynote at the Eurostar 2014 conference in Dublin. I crafted a talk that was unique to this event, bringing the theme together, summarizing what the theme meant to me and exploring how it is all connected.

I know the slides can only tell you so much when the narrative isn’t there, but here is the online version of my Prezi:

Everything is connected – exploring diversity, innovation, leadership

Everything is connected

Although it was the first (and last) rendition of this talk, I think it went well. Several people found it to be “thought-provoking”, which is exactly what I was aiming (and hoping) for.

Now that this is over, I feel I am done with conference presentations for a while. I’m planning to take a long-awaited deep dive – with lots of reading, learning and working on new content. I’m taking it slowly. There are some important topics that need exploring, and now I am finally giving them (and me) the time to make that happen. I’m also looking forward to some exciting collaborations with others in the near future.

I’m following my energy. Let’s see where that leads us.

Advertisement

DEWT4 – a peer conference on teaching testing

dewt4-participants-v4

From left to right: Jeanne Hofmans, Rob van Steenbergen, Jurian van de Laar, Peter Simon Schrijver, Jean-Paul Varwijk, Bernd Beersma, Huib Schoots, Arjen Verweij, Zeger van Hese, Joris Meerts, Markus Gärtner, Bart Broekman, Angela van Son, Pascal Dufour, Ard Kramer, Jeroen Mengerink, Kristoffer Nordström‏, Philip Hoeben, Daniël Wiersma, Joep Schuurkes, Duncan Nisbet, Eddy Bruin, Wim Heemskerk, Ruud Cox, Richard Scholtes, Ray Oei

Teaching Software Testing

DEWT IntroIn the weekend of 7-9 February, the fourth edition of DEWT took place at Hotel Bergse Bossen in Driebergen, the Netherlands. DEWT stands for the Dutch Exploratory Workshop on Testing and is a LAWST-style peer workshop on testing like its older siblings LAWST, LEWT and SWET. This means a presentation is followed by a facilitated discussion that goes on as long as it brings value.

This edition was extra special to me since I volunteered to be the Content Owner during our preparatory meeting in september. Jean-Paul Varwijk agreed to fill the Conference Chair role and Peter Simon Schrijver would be the main facilitator. Why yes, you do need a good facilitator to make this kind of thing work.

The main theme of this edition was “Teaching Software Testing”

In this edition we also added the obligation – for all attendees – to send in a proposal for an experience report. I wanted attendees to look at teaching software testing in a broad sense, and asked for experience reports on:

  • How software testing is taught
  • Unconventional or alternative ways of teaching software testing
  • Lessons learned by teaching software testing
  • Learning how to teach software testing
  • The receiving end of teaching – learning (being taught)
  • The transfer of theoretical versus practical knowledge
  • Teaching novice testers versus teaching experienced ones
  • Acquiring teaching skills
  • …

Apart from the DEWT core members (10), an additional 16 people were invited, of whom three came from abroad – Markus Gaertner (D), Duncan Nisbet (UK) and Kristoffer Nordström (SE). Actually, that makes four since I am from abroad (B) as well – I keep forgetting that I am DEWT’s legal alien.

Friday, February 7

The first night of a DEWT conference is usually an informal meetup, with a welcoming dinner for the people that can make it in time. A great evening it was, with strangers getting to know each other and old friends catching up. Lots of games and testing talk – and in some way or another, My Little Pony (™) became a topic as well. There were not as many drinks as we would have liked, though, since our first evening happened to coincide with a wedding in our regular hangout, the Grand Cafe. This meant we were banned to a room with a part-time waiter, dividing his inevitably part-time attention (I’m guessing 85/15) between drunk party people and relatively sober software testers. His selection of Belgian beers and copious amounts of deep-fried snacks (it is common knowledge that Markus Gaertner will attend any meetup that involves bitterballen) made up for it.

We ended the night giving the bride and groom some heartfelt marital advice, and by sipping from that curious bottle Duncan brought from Gibraltar – Patxaran (Zoco). When Duncan started cleaning tables to compensate for our invisible waiter, we knew it was time to go to bed.

Saturday, February 8

In front of a notably bigger group than we ended the day with on friday (some people were only joining in on saturday morning), Jean-Paul, Peter and myself kicked off the conference. In the previous weeks, the three of us had come to an agreement on which talks should go highest on the list, being well aware that in the end, a schedule like this is always tentative since you never know when discussions are going to end or where the energy of the group will be going.

The roomKristoffer Nordström went first with “Learning and change in a dysfunctional organisation“, illustrating the difficulties of a consultant that represents both management and the outside. Are learnings and change even possible is this situation? He compared a team with a spring that is attached to context and culture. When a string is attached to something, it is very hard to change. You can bend the spring and make it work at first, but inevitably, the spring – the team – will veer back to its original position. He explained how he tried to cope with his plight: establish trust, show passion and enthusiasm, lead by example, show respect, take time to teach instead (not tell). Even simple things like smiling and saying hello to people helped him to achieve his goals. Kristoffer’s experience report was rich and well-prepared, and touched many things which I could relate with. The discussion afterwards went on longer than planned, but hey, we’re all flexible, right?

Next up was Arjen Verweij with “Preaching software testing: evangelizing testers among non-testers“. In his experience report, he described how he advocates for testing with different stakeholders:

  • Talk to project managers about value
  • Inform and explain customers about changes in the software
  • Convince engineers that you need their expertise.
  • Help support people by providing them with good tools that facilitate bug reporting
  • Work with sales to set reasonable expectations
  • Get buy-in from the developers by supporting their work

One of Arjen’s take-aways was to not mention “testing” if you want non-testers to test, which spawned a hefty discussion on-site in which several people on twitter got involved.

After lunch we decided to go for a walk in the woods to avoid that dreaded carb coma. The hotel staff provided us with instructions for a walk, and it turned out to be a strictly scripted procedure: no map, but a list of written instructions. Great, a bunch of (mostly) context-driven testers asked to follow a walking script. As could be expected, we got lost in a heartbeat. Our explorer’s instinct – supported by many a gps module – got us back with only 20 minutes delay.

Aside from harassing us with more space unicorn songs then we could handle, Markus Gaertner got us up and about with a workshop that used the principles from the book “Training From the Back of the Room!: 65 Ways to Step Aside and Let Them Learn” by Sharon Bowman, after which he elaborated on the 4 C’s, a framework to help design classes that leverage accelerated learning. The acronym stands for “Connection, Concept, Concrete Practice, Conclusion”. During the connections step, learners make connections with what they already know about the topic at hand. In the concepts step, learners take in new information in multi-sensory ways: hearing, seeing, discussing, writing, reflecting, imagining, participating and teaching it to others. The concrete practice step serves to actively practice a new skill using the new information, participate in an active review of what they have learned and again teach others what they know or can now do. During the conclusions step, learners summarize what they have learned, evaluate it and make a commitment to use it at work or in their lives.

Joep Schuurkes and Richard Scholtes were up next with “Teaching testing with a chain testing simulation“, in which they described their experiences in designing an apparently simple chain testing simulation exercise. In it, participants were provided with five laptops running the applications that make up the chain, and each was assigned to one of the laptops (or was assigned the role of testing coordinator), after which the group was given the assignment to “perform a chain test”. Joep and Richard contrasted the things they thought should happen with the things that actually happened, which lead to a couple of nice surprises. Chaos ensued, apparently, and people stayed on their own island for way too long. But it proved an engaging format for all involved – people continued during breaks, were discussing it the days after and it led to quite some aha-moments as well. Another take-away: putting an empty chair in between two people is an effective means to stop all communication.

Bart Broekman‘s experience report brought us “Back to the Middle Ages“. Or at least, a part of the theory did. He talked about the master-apprentice model, which is fundamentally different from the teacher/student model which is now so common. Later on he linked it to the Dreyfus model of skill acquisition. Bart saw the biggest gap to be bridged in going from “competent” to “proficient”. How can we make our students make that big leap? Bart went on to explain how he tried to do that through organising masterclasses, working with the student’s own content and real-life problems to solve.

By the time the discussion after Bart’s report died down, dinner was calling, and we gladly obliged. The evening was filled with drinks, puzzles, games, poetry recitations and Dutch people winning gold, silver and bronze medals in the Winter Olympics. Leave some for us, would you?

Sunday, February 9

Sunday morning saw the first (granted, UK-imported) Gibraltarian DEWT-invitee ever take the stage: Duncan Nisbet with a report on his experiences teaching testing to new/non testers, “You can’t learn to drive by reading a book“. A misleading title, since he proceeded to compellingly relate it to how he used to train his pupils in wildwater kayaking. In that line of sport, it is important that you first give your students a safe place to fail, like clear, calm and warm water. Whitewater is an unforgiving place for newbies. Slowly progress, and if you fail, do it more often. Books don’t prepare you for the whitewater experience. Duncan then explained how he tries to teach testing to newbies in the same way. Start simple and build confidence gradually. First, give them a Web/GUI to play wit, later make them aware that of the existence of logs that can help them in testing, and then on to other more specialised disciplines like performance and automation. The facilitated discussion afterwards spawned so many question threads that Simon Schrijver dedicated a whole blogpost on how he facilitated it.

Angela Van Son is a DEWT regular, although she is not a tester. Angela is a professional (procrastination) coach, and she made the program because I was convinced that she could contribute to the topic by offering a view on teaching in general. In “The skill of teaching: How do you make them WANT it?“, she told us about the 30 Day Video Challenge put out by Holly Sugrue that she participated in. She witnessed how Holly managed to energize and inspire the group to deliver in this challenge, a remarkable feat considering that the different people all had different goals to participate. Angela then analyzed what was so peculiar about this challenge. How did the challenge make the people want to master it? As it turned out, it was a combination of many things. It was well chosen, with clear limits. There was a lot of playful repetition that never got boring, and great group dynamics that pushed people forward. There were no obligations – it was a safe place to learn.

After lunch, to finish on a lighter and more active note, we scheduled a workshop on the design of exercises to teach testing, led by Huib Schoots and Ruud Cox. The crowd got divided in several small groups and given the task to design an exercise to teach some testing concepts. That exercise would then be run by one of the other groups followed by a debrief. The tricky part was of course that the time to accomplish this was very limited. I already knew from attending Jerry Weinberg’s Problem Solving Leadership that designing a good and fun experiential exercise can be very, VERY hard. But given the circumstances, the teams did a good job – resorting to ruthless peppermint crushing and exploratory walking. I felt that the debrief helped a lot in seeing where our own exercise could be improved.

This concluded DEWT4. Two + days filled with learning and fun. I hoped to achieve a good variety in the topics and I am glad how it turned out. The atmosphere was focused yet relaxed, and everyone seemed to enjoy themselves. A big thank you to all the participants for their time, their stories and their passion.

Sketch-notes

To finish of this lengthy report, here are some sketch-notes I made during the weekend. Click the thumbnail to see a bigger version.

For other reports and impressions of DEWT4, check out the DEWT after party blog page.

Back to the middle ages - Bart Broekman You cant learn to drive by reading a book - Duncan Nisbet Learning and change in a dysfunctional organisation - Kristoffer Nordstrom Preaching software testing - Testing with non-testers - Arjen Verweij Teaching testing with a chain testing simulation - Joep Schuurkes Richard Scholtes The skill of teaching - how to make them want it - Angela Van Son

Vacansopapurosophobia


tumbleweed

< crickets >

For those of you who have watched the tumbleweeds roll by on this blog in the past year, I apologize.

It’s been eleven months since my last blog post and I have no excuse.

Sure, I have been busy – 2013 was one hell of a ride. I founded my own company (Z-sharp), worked hard on getting things started, created presentations and papers to present at conferences, co-organized Belgium’s first public RST course with Michael Bolton, attended and organized peer workshops and delivered webinars.

I knew from past experience that being busy does not necessarily mean that your writing suffers. And yet I had no energy for blogging.

I kept telling myself “Just wait it out, ideas will pop up. There’s no need to do things half-heartedly. Pick your battles”.

Self-diagnosis

Ideas did pop up. Plenty of them, eventually. Strangely enough I felt no urge to act upon them, which in turn reinforced the feeling I was stuck. It started to freak me out. This was the first time I hit a dry writing spell of this length. What was happening?

“That’s it”, I thought. “Vacansopapurosophobia – the fear of a blank page” (the first word that came to mind was “writer’s block”, to be honest. But I like the sound of vacansopapurosophobia better, it has a nice supercalifragilisticexpialidocious ring to it).

Writer’s block – the fear of every aspiring writer! Or rather, blogger’s block. Wait – did I just self-diagnose myself?

“Self-diagnosis is the process of diagnosing, or identifying, medical conditions in oneself. It may be assisted by medical dictionaries, books, resources on the Internet, past personal experiences, or recognizing symptoms or medical signs of a condition that a family member previously had. Self-diagnosis is prone to error and may be potentially dangerous if inappropriate decisions are made on the basis of a misdiagnosis” (source: Wikipedia)

The danger of self-diagnosis is that you’re mainly forming conclusions based on internet folklore. Quite ironically, there *is* a lot of writing about writer’s block on the internet, with causes ranging from too much audience awareness and perfectionism over burnout to flat-out depression.

Weinberg on writing

I decided to seek guidance from a professional. I dug up my copy of “Weinberg on Writing“, in which my personal Yoda Jerry Weinberg describes his writing process. I still vividly recall my amazement several years ago when when I first read it. It fit my own amateurish and seemingly unstructured writing style like a glove.

In the book, Weinberg compares writing to the creation of stone-wall structures. Harnessing ideas and words into a written work is a lot like building a stone wall: gathering, arranging, rearranging, and discarding fieldstones as the wall evolves organically over time. To be successful in your writing, Weinberg suggests, you should have many fieldstones, chunks of work in progress. But be aware that “in progress” is a very vague concept: it may mean you’ve written two words, a hundred words, or even several chapter-like things.

The fieldstones allow you to make progress on any piece of work. The method helps to keep personal energy high, efforts focused and the daunting work of composition forward-moving.

Weinberg on not writing

I dove in the part on writer’s block (you can read an article based on that chapter here), and the following sentence struck a chord – or two:

“Writer’s block is not a disorder in you, the writer. It’s a deficiency in your writing methods – the mythology you’ve swallowed about how works get written.”

Of course! I knew this all along, but I let it get snowed in in my middle-aged excuse for a brain. The creation of a text is not a linear process, like reading is. Reading structures are presentation methods, not creation methods. Creation doesn’t work in such a linear way.

Later on I stumbled upon a rather amusing interview with Jerry Weinberg in which he dispels the myth of writer’s block. This taught me another valuable lesson: as long as you have things you can do, you aren’t blocked at all. When you feel stuck with one part, work on another – they don’t even have to be directly related. There is always something you can do to keep on moving.

All of a sudden, I came to the realization  that the solution to my problem was very simple.

“You have nothing to write about? How lovely is that! Isn’t that a GREAT subject?”

So here I am, writing my first blog post in ages, about why I wasn’t writing. I hope I’m here to stay.

Rapid Software Testing – skilled software testing unleashed

Up to 11

The software testing profession looks like a steadily maturing profession from the outside. After all, there are certifications schemes like ISTQB, CAT, IREB and QAMP (the one to rule them all), standards (ISO 29119) and companies reaching TMM (test maturity model) levels that – just like a Spinal Tap guitar amplifier – one day might even go up to 11. The number of employees that companies send off to get certified in a mere three days is soaring, and new certification programs are being created as we speak. Quick and easy. Multiple choice exams for the win!

The reality, however, is that the field of software testing is torn between different “schools” of testing. You could see these schools as determined and persistent patterns of belief, speech and behaviour. This means that different people – all calling themselves “test professionals” – have vastly different ideas of what testing is all about. Even something elementary as the definition of testing varies from “demonstration of fitness for purpose” to “questioning a product in order to evaluate it”, depending on who you talk with (for more info on the schools of software testing, I heartily recommend Brett Pettichord’s presentation on the subject).

And so it happens that different people think differently about “good” or “mature” software testing. I, for one, don’t believe in tester certification programs, at least not in the format they are in now and the way they are being used in the testing profession. The current business model is mainly designed to get as many people as possible certified within the shortest timeframe. Its prime focus is on certifiability, not on tester skill, and certainly not on the advancement of the craft. Advancement comes from sharing, rather than shielding.

Rapid Software Testing (RST)

So what are the options for a tester on a quest for knowledge and self-improvement? What is a budding tester to do?

I think there are valuable alternatives for people who are serious about becoming a world-class tester. One of these is Rapid Software Testing (RST), a 3-day hands-on course designed by James Bach and Michael Bolton.

Actually, calling this “a course” doesn’t do it justice. RST is at the same time a methodology, a mind-set and a skill set about how to do excellent software testing in a way that is very fast, inexpensive, credible and accountable. It is a highly experiential workshop with lessons that stick.

How is RST different?

During RST you spend much of the time actually testing, working on exercises, puzzles, thought experiments and scenarios—some computer-based, some not. The goal of the course is to teach you how to test anything expertly, under extreme time pressure and conditions of uncertainty, in a way that will stand up to scrutiny.

The philosophy presented in this class is not like traditional approaches to testing, which ignore the thinking part of testing and instead focus on narrow definitions for testing terms while advocating never-ending paperwork. Products have become too complex for that, time is too short, and testers are too expensive. Rapid testing uses a cyclic approach and heuristic methods to constantly re-optimize testing to fit the needs of your clients.

What’s in it for you?

  • The ability to test something rapidly and skilfully is critical. There is a growing need to test quickly, effectively, and with little information available. Testers are expected to provide quick feedback. These short feedback loops make for more efficient and higher quality development
  • Exploratory testing is at the heart of RST. It combines test design, test execution, test result interpretation, and learning into a seamless process that finds a lot of problems quickly. Experienced testers will find out how to articulate those intellectual processes of testing that they already practice intuitively, while new testers will find lots of hands-on testing exercises that help them gain critical experience
  • RST teaches you how to think critically and ask important questions. The art of questioning is key in testing, and a very important skill for any consultant
  • RST will provide you with tools to do excellent testing
  • RST will heighten your awareness

Bold claim bottom-line:

RST will make you a better tester.

RST comes to Belgium

Co-learning and Z-sharp are proud to announce that from 30 September – 2 October, Michael Bolton will visit Belgium to deliver the first ever RST course on Belgian soil, giving you the opportunity to experience this unique course in person. More info can be found here, or feel free to contact us for more info.

Brace yourself for an mind-opening experience that will energize and boost your mind.

All the way up to 11.

(for even more information and testimonials about RST, see Michael Bolton’s RST page)

 

It’s… thinking thursday

The Challenge

A couple of minutes ago, Michael Bolton tweeted

Thinking Thursday. Test this sentence: “In successful agile development teams, every team member takes responsibility for quality.”

My initial reaction was: “A Michael Bolton challenge – where’s the catch?” This is actually a sentence that shows up regularly in agile literature. Heck, I even said it myself a couple of times. What I really wanted to say at the time, was probably something along the lines of “In agile development, producing quality software should be a team effort – lots of collaboration and communication. No blaming or finger-pointing individuals.”

I tweeted some replies, but soon realised that I would hit the 140 character limit head on.

The Test

But then I thought – why not give these kinds of agile creeds Weinberg’s “Mary had a little lamb”-workout, usually reserved for demistifying ambiguous requirements. I used it earlier: stress every word in turn and see where the ambiguities are.

  • In?
    Does this mean that outside agile development teams, no team members take responsibility?
  • Successful?
    Does this imply that in unsuccessful agile development teams, no one takes responsibility for quality, or that some individuals take the blame? Successful to whom, and compared to what? What is meant with “success”, really? On time, within budget? Satisfied customers? All of these combined?
  • Agile?
    What “Agile” definition are we talking about? Capital A, small a? A mindset, a methodology?And what about successful waterfall teams? Do some individuals take responsibility there? I would like to think that in successful teams, all team members would like a part of the praise. What about those other kinds of development teams out there?
  • Development teams?
    Are we talking about developers only here? What about the tester and product owner role? Or all the other roles that played an important part in developing the product? “In agile teams, testers *are* part of the development team”, you say? I agree, as are the product owners. But in that case, we should think about another label for the team.
  • Every?
    Really? *Every* team member? Can all team members be equally responsible for quality? As Michael Bolton contends, testers do not assure quality. Do testers hire the programmers? Fix problems in the code? Design the product? Set the schedule? Set the product scope? Decide which bugs to fix, write code?
  • Team member?
    What about people that played a part in successfully delivering the product, but that are not considered as core team members? Who are the people that make up the team? Is that defined up front? Aren’t those team boundaries pretty dynamic?
  • Takes Responsibility?
    Doesn’t *taking* responsibility sound a bit too negative? Isn’t “responsibility a two-sided sword? Receiving praise when the quality is applauded, taking the blame when quality turns out to be sub par?
  • Quality?
    Quality, to whom? Qualitative, compared to what? What is quality, anyway?

Is there a problem here?

Well… The sentence under scrutiny sounds comfortably familiar, and in that sense it was a good thing to think it through in a little more detail. It sure leaves a lot to interpretation. Some of the terms used in it are highly subjective or their definitions simply not generally agreed upon.

Back to twitter

Later on, in a response to a tweet from Shrini Kulkarni, Michael said that his purpose was “exploring what bugs me (and others) about it”.

Actually, nothing bugged me about it *before* the exercise, but now it dawned upon me that the wording of that good agile practice does not do the practice justice. It is too vague; it does need rephrasing.

How about a Frustrating Friday challenge: make this sentence fresh and ambiguity-free.

You could postpone it to Semantic Saturday, if you wish. Your call.

Rebel rebel – the Danish Alliance @ Eurostar 2010

Something way cool happened at Eurostar this year. A group of like-minded people got together after the conference to do a mini-CONFERence in a more intimate setting. They called themselves the Danish Alliance (or Oprørsalliancen, when they felt like badly pronouncing Danish words). The concept was based on the Rebel Alliance, started by Matt Heusser at StarEast last year. I had been thinking about a localized version of the Alliance before, but it was the ever energetic Shmuel Gershon who put his efforts into organizing the first Alliance on European soil. Of course, this little guerilla conference couldn’t have happened without the generous help of the Eurostar folks, who set us up with a superb meeting room. Need I say that they ROCK?

The ingredients were simple: 

  • A handful of passionate testers
  • A safe setting
  • Drinks
  • Pizza
  • Music
  • Chocolates & cookies

Throw all these together and stir gently. Observe.

Whatever happens, happens. There was no agenda, really. In this case we mingled first, talked and drank a bit until pizzas arrived. Major  epiphany: Denmark has pizzas that come in the size of a small wallaby. After that, there were some lighting talks, timed by quality gatetimekeeper Michael Bolton (who definitely should get into the timekeeping business whenever he gets out of the QA business). You can see (transcripted!) videos of the talks in Shmuel’s write-up of the event. 

‘Talks’ don’t have to be ‘talks’, per se. James Lyndsay did a call to action to test one of his new black box testing machines. Andy Glover (the Cartoon Tester) got us drawing abstract concepts. Dorothy Graham even gave us a Sound of Music flashback by singing about her favorite techniques. Anything goes.

Discussions continued until the wee hours. I thought it was wonderful. This is the kind of stuff that doesn’t regularly happen during the day at conferences. Sure, the Eurostar programme was great, again (and I’ll be writing more about that later), but the real conferring often happens outside the track sessions and tutorials. It feels great to connect with other people that are all driven by the same thing: a passion for their craft.

So thank you Shmuel Gershon, Jesper L Ottosen, Joris Meerts, Dorothy Graham, James Lyndsay, Bart Knaack, Martin Jansson, Henrik Andersson, Michael Bolton, Andy Glover, John Stevenson, Rob Lambert, Carsten Feilberg, Ajay Balamurugadas, Markus Gaertner, Henrik Emilsson, Julian Harty, Rob Sabourin, Rikard Edgren, Lynn McKee and Rob Lugton. The force will be with you, always.

Exploring Rapid Reporter

Eploring Rapid Reporter

This is my attempt at an exploratory essay.

I will be documenting my first exploration of a note-taking tool (Shmuel Gershon‘s free Rapid Reporter) to assist exploratory testing sessions.  That’s right, I’ll be exploring an explorer’s tool – sounds like this has the makings of a pretty meta experience.

I first heard about the tool when Shmuel gave the Rebel Alliance at Star East a sneak peek of what he was working on at the time. The Rebel Alliance was a very cool “conference within a conference” organised by Matt Heusser, with plenty of lightning talks and great discussions. You can see a video of his short talk here.

Up till now, I have been using David Gilbert’s TestExplorer, Jonathan Kohl’s Session Tester and James Bach’s Scan Tool to document my exploratory testing sessions (with miscellaneous results), so when he finally offered it to the public during the Blogstar competition (in which Shmuel was a well-deserved runner-up by the way), I knew I just had to give it a try. 

I started out by reading the user guide on the site, skimmed through the known issues list on the page and read through the short faq section. During this little theory intake, I deliberately didn’t start the tool (although I had to suppress the urge). I wanted to start using it for a first time in a proper testing session. 

Start

I fired the RapidReporter.exe, was asked to enter a name and a charter, all pretty straightforward.

*Edit: Actually, I entered a session name when prompted for a name. Later, when looking in the .csv-file, I realised that it was supposed to be my own name, the name of the reporter. It confused me, so perhaps the naming was ambiguous? Would “Reporter” be a better option? (Made a NOTE about it).  

I was ready to test a piece of newly coded software. The cool thing was that RapidExplorer stayed on top, tucked away in a place that didn’t bother visibility: no alt+Tab switching, no interrupting my flow. Note-taking went smoothly. Switching the note type by the up/down arrows was intuitive and quick.

After a short while I noticed a blue progress bar ehm… progressing and it surprised me. That’s a nice feature, it gives you a visual clue of the session’s progress, but without the actual distraction and pressure of a detailed timer. The blue bar also implied a default duration, since I hadn’t indicated any preferred session length before starting. I wondered why the default duration wasn’t asked along with the name and charter, and figured that was probably because most people use the same timebox for their sessions anyway (60-90-120 minutes, whatever). So what is the duration of the current session? It isn’t indicated on the main bar. Is there a menu of some kind? The most intuitive thing would be a right-click, I’d say. Bingo! Three menu items there: Time until end, Open working folder and About. But where do I see the duration of this very session? The only time-related entry is the first one, but this only breaks down in a menu to select either ’60-90-120 min until end’ or to ‘stop the session’. Selected menu items in comparable products often have a selection checkmark or bullet next to them (NOTE).

I opened some menu items in the application under test and noticed an inconstistency, so taking a screenshot was in order. I didn’t remember reading about hotkeys, but I tried some anyway (CTRL+S, ALT+S, S – which just entered an ‘S’ in Rapid Reporter, of course). I was afraid I would lose the expanded menu if I clicked outside the application. I eventually ended up clicking the S button, but – as I expected – the screenshot didn’t show the expanded menu items. Apparently, there is no keyboard shorcut yet, which makes it impossible to capture these kinds of phenomena (NOTE). For all the other screenshots I ended up taking, clicking the S did the trick just fine. The captures were saved as time-stamped jpegs in the startup folder. A huge improvement, since I was used to take the screenshot, paste it in MSPaint, name it & save it. By the time I got to the application, I often forgot what I was actually doing in the first place. Darned short-time memory.

I got interrupted by a phone call and wanted to pause the timer. Mmm. No pause option to be seen. I tried hitting the spacebar to pause – don’t know where that came from. Media players or games, maybe? There *was* a the stop option in the menu, so I tried that. But this actually stopped the timer and reset the progress bar to zero. Is there an option to pause and “restart” where I paused it? I can think of some occasions in which that would be useful (NOTE).

I was still pondering the timer reset, when I noticed that the progress bar started progressing again. Is the “stopped” state really a stopped state? Tried it again, but now stopping really stopped it. Maybe I started a session without knowing that first time? Maybe something intermittent? (INVESTIGATE LATER)

When time was up, this was clearly indicated by the flashing stopwatch. It was still possible to add keep working with it even when time is up, which seems like a logical and practical thing to do. I can think of many situations in which you want to continue for a bit, clean up some loose ends.

I started a new session, worked with that for a while. I was really getting used to this new kind of notetaking. I wanted to extend the length of the session, and wondered what would happen if I changed it to 120 minutes. I expected the progress bar to redraw itself (keeping in mind the time spent and time still to go), since my current session was 60 minutes. But it didn’t. Did the duration really change? Possibly, but no visual clue to confirm that (NOTE).

I was curious about what was actually being recorded during the session and opened the csv-file in the startup folder. All my notes were there, time-stamped and delimited with commas. All nice and powerful.

Stop

I stopped the session while it was still ongoing (by using the cross in the upper right corner) and got the message that an error occured when writing  the note into a file. Yes, the csv-file was still open and although I didn’t enter a note since opening the file, the application attempts to write something into the file when closing. It would be nice if the excel closed automatically instead of throwing an error when nothing changed (NOTE).

I checked the screenshots taken during the session (by clicking “S”) and the Rapid Reporter bar is in the screenshots. At first I thought this could be annoying for a tester, but then I figured that it would always be in a non-disturbing place anyway. It can also convey useful information: how far you were in the session, what the note was at the time, etc. 

I decided to try taking a screenshot the regular way (using Printscreen), and it turns out that the Rapid Reporter bar is *not* in the picture. Strange. Seems like an inconsistency, but I might not have all information here. I’ll talk to Shmuel about it (NOTE).   

Conclusion

I decided to stop my first exploration of Rapid Reporter here. It was a pretty shallow tour of some basic functionalities – I didn’t even look into the reporting possibilities yet, I’ll certainly do that next time. This little write-up just documents my first two hours of exploring a new tool. I wanted to learn about it first, not necessarily find bugs.

To round it up: I must say I’m impressed. It’s easy to use, and non-intrusive indeed. When using other Session Based Test Management tools, I often ended up spending more time in the note-taking tools than in the application under test, which is a bit of a drag. Rapid Reporter puts all focus on the software being explored. It supports exploration, while allowing testers to stay in their flow. This is important for me anyway: everytime someone interrupts my flow, God kills a kitten. Ok, not really, but you catch my drift.

Of course, I stumbled upon some issues, but they didn’t annoy me, they didn’t make me stop using it. I didn’t expect perfection – after all, the readme file explicitly mentions that it’s still buggy. I would have gotten *really*suspicious if it would have claimed it was bug-free. So its current state is perfectly okay for me. After all, it is brand new. It’s work-in-progress. The best thing Shmuel could do, was to unleash it upon an international cohort of testers. And he did. It turned out to be a great gift to the testing community.

Will definitely use again. A+++

Why you shouldn’t miss EuroSTAR 2010

10 reasons why you shouldn’t miss Eurostar 2010

Two weeks from now you will find me in trendy Copenhagen, proud home of the world’s best restaurant (Noma) and Hans Christian Andersen’s Little Mermaid. But the real reason for my trek up north is not sightseeing or spending money on Tørret kammusling og biodynamiske gryn: Copenhagen is also the host of the 18th edition of the annual EuroSTAR testing conference.

If you’re not yet familiar with Europe’s biggest software testing conference, you should definitely check them out. If you’re still hesitating about attending, there’s no need to. If you’re thinking of going, do go. Here’s why:

  • First and foremost: the Content.
    • The programme committee assembled a promising line-up, centered around the main theme “sharing the passion”. Track session categories include test management, exploratory testing, virtualisation, techniques, Scrum, inspiration, education, Lean, MBT, people and automation, among many others. L’embarras du choix.
    • Keynotes, anyone? Antony Marcano, Rob Sabourin, Bob Galen, Dino Patti and Stuart Reid. Recent history teaches us that wherever Stuart lays his hat, controversy and discussion automagically appear. I’m confident that his keynote “When Passion Obscures The Facts: The Case for Evidence-Based Testing” will be no different. 
    • I especially look forward to the tutorials. Rob Sabourin will run a full-day tutorial on “Just In Time Testing – Effective Testing Strategies“. Michael Bolton will be doing a half-day tutorial on Test Framing (read his blogpost that coins test framing here). But that’s not all. There’s Lee Copeland too. And many, many more. L’embarras du choix, revisité.   
  • Test Lab.
    James Lyndsay and Bart Knaack will run their on-site Test Lab for the second consecutive year. They will be assisted by Henrik Emilsson and Martin Jansson, 2/3 of that restless online blogging collective called The Test Eye. The other 1/3 is Rikard Edgren, who is part of the programme committee this year – I guess you could say that Eurostar is TestEye-infected. From what I experienced last year, the test lab is a really unique experience. Live testing at a testing conference! Theory put into practice, and maybe some weekend testing sessions, testing dojos or katas. Anything goes, really.
  • Inspiration.
    Hearing all these different viewpoints, new ideas, talking with the experts, engaging in discussions… It’s a savory buffet full of food for thought. Attending conferences is intellectually stimulating, and you’ll probably learn more during these couple of days than you do during most ‘regular’ training courses. I see EuroSTAR as a multi-dimensional training course that as such deserves to be on every company’s training calendar.
  • Get Primed.
    Any problems you are facing at work – you just might see them differently when you get back. Things you hear at the conference and  people you talk to often trigger other ways of thinking. Conferences tend to broaden your perspective on things.
  • Reach out to the testing community.
    This year’s theme is ‘Sharing the passion’, which should make it easy to meet like-minded people who share the same interests. In his 2009 book “The Element”, Ken Robinson calls this “finding your tribe”: connecting with people who share the same passions and commitment (your “tribe”) helps in finding and developing your “element” (which is the place where passion and skill meet). Members of a passionate community tend to stimulate each other to explore the real extent of their talents. Whenever tribes gather in the same place, the opportunities for mutual inspiration can become intense.
  • Meet Testing Tweeps.
    Twitter has been doing brilliant things for testers already, community-wise. It’s a great way to interact with testers worldwide on a daily basis. It has also proven to be a very useful, fun and informative way to cover conferences, especially for the people missing out (watch that #esconfs hashtag for some conference goodness). If you’re on twitter, EuroSTAR will also be a good opportunity to meet numerous testing tweeps in person and to take your twitter-conversations with them to the next level.
  • Hallway/Bar discussions.
    In an earlier blog post, C is for Conferring, I mentioned that conferences are for conferring, and that the most interesting things often happen in the hallways, in between sessions. Or in the bar. Or somewhere totally unexpected. Make sure there are blank spots in your busy schedule to invite serendipity.
  • One word: Copenhagen.
    ‘Nuff said. But did I mention that the place of action is the Bella center? Last year, the Bella center hosted the first sustainable, international political summit – the United Nations Climate Change Conference (COP15) – attracting over 33,000 people. No worries, Eurostar Conferences assured me that Eurostar 2010 will be more succesful than its flunked climatic counterpart.
  • Interactive Panel Session.
    On wednesday morning, Lee Copeland will facilitate a Hot Topics Panel Session. The expert panel is there to address *your* burning issues, so if you want to ask the EuroSTAR Panel a question, you can do so via facebook. Yes, the Social Network goes testing.
  • It’s fun!
    By focusing on all the content, the learning and networking, I almost forgot to mention that above all, it’s fun. All of the above takes place in a fun and relaxed athmosphere.Fun sessions for the weary testers are foreseen as well (I’ve been told that the supertesters are something to look out for).

This concludes *my* list. Rob Lambert wrote about attending EuroSTAR too, in his post EuroSTAR will rock. Eurostar Conferences has also listed their Top 10 Reasons To Attend EuroSTAR 2010! And if you need to make a case for attending the conference, the 10 ways to convince your boss to send you to EuroSTAR 2010 article may be able to help you with that.

I hope to see you in Copenhagen. I’m @TestSideStory, by the way. I’ll be roaming the hallways – feel free to come and talk to me. I’ll be the one with that thorny rose clenched in the teeth.

Short service announcement: tomorrow, October 16, programme committee member Peter Morgan will present a webinar especially for first time attendees. It is called “Getting The Most Out Of EuroSTAR“. More info and a link to register for the webinar can be found here.

Agile Testing Days 2010 – Day 3 (Lederhosen and Certified Self-Certifiers)

Agile Testing Days 2010 – Day 3 (Lederhosen and Certified Self-Certifiers)

October 6

Wednesday. Michael Bolton warmed up the audience with the keynote performance How am I supposed to live without you? “Testers: Get Out of the Quality Assurance Business!“, and proved once again that he’s a hard act to follow. He immediately came out of the closet saying that he’s an Agile skeptic and stated what “being Agile” means to him:

  • Adhering to the Agile Manifesto
  • “Be able to move quickly and easily” (cf the definition in the Oxford English Dictionary)
  • De-emphasizing testing for repeatability
  • Re-emphasizing testing for adaptability
  • For testers, focusing on testing skills
  • Focusing on not being fooled

Michael then defined quality as “Value to some person(s) who matter” (© Weinberg, Bach, Bolton) and said that decisions about quality are always political and emotional, and taken by people who actually have the power to make these important decisions. A little bit later, the main message of the talk jumped right at us and bit us in the face:

If you are a tester, do *you* hire the programmers? Fix problems in the code? Design the product? Allocate staff? Set the company’s strategic direction? Allocate training budgets? Set the schedule? Decide on raises? Control the budget in any way? Negotiate customer contracts? Actually choose the development model? Set the product scope? Do you decide which bugs to fix, or write the code yourself?

Did you answer “No” to most of them? Then you will probably agree that it is simply impossible to “assure” quality. But no worries – it is not our job to assure quality. What we *can* do is test, and make sure we’re damn good at it. Testing in the sense of a sapient activity, providing information with the intent of *informing* a decision, not *taking* the decision. Not to be confused with checking, which mainly aims at confirming existing beliefs. Checking is automatable and non-sapient.

Michael Bolton shifted into a higher gear, and claimed that “acceptance tests” are examples, and that examples aren’t really tests. They are checks, not tests. Acceptance tests don’t tell us when we’re done, but they do tell us that we’re not finished when they fail. They should in fact be called “rejection checks”.

I looked around me. Usually, at this point in a presentation and at this time of day, people are dozing off. Even the biggest barflies were wide awake now. He ended with a set of statements that almost read like some kind of Tester’s Manifesto:

We’re not here to enforce The Law.
We are neither judge nor jury.
We’re here to add value, not collect taxes.
We’re here to be a service to the project, not an obstacle. 

I got out of the room early and skipped the Q&A part, since my presentation was up next. Apparently the Q&A got a bit out of hand (I suspect the A was probably more to blame than the Q), because the auditorium doors swung open 15 minutes late. In hindsight, I was lucky that I even had an audience; in a parallel track, Gojko Adzic was delivering one hell of a performance (a stand-up comedy routine, I was told) for an overly packed room. 

No stand-up comedy in my room, but an honest “inexperience report” called “A lucky shot at Agile?“. I had ditched Powerpoint one week earlier and decided to go for Prezi, the so much nicer alternative. Of course, this was a bit of a risk, but I think it turned out fine. The presentation went well, and I received some good and heartwarming feedback which really made the rest of my day. 

In case you are interested, here’s A lucky shot at agile – prezi.

<Shameless_plug>In case you’re interested in the full story, Eurostar conferences has released my paper on the subject in an ebook-format – available for free – here </Shameless_plug>

I stayed in the room to attend Anko Tijman‘s talk “Mitigating Agile Testing Pitfalls“. Anko’s talk revolved around five pitfalls that threaten agile teams, and what we can do to mitigate them:

  1. Not testing with the customer. We can mitigate this risk by building a relationship, building trust.
  2. Not testing as a team. Teams are collectively responsible for the quality of the product. Share knowledge not only with your testers, but with the whole team. Work on a collaborative definition of done, tackle risks.
  3. Unbalanced test strategy. Teams sometimes focus too much on unit tests or acceptance tests, postpone other test activities to the next phase. This in turn can lead to a lack of feedback. To overcome this, put more detail in Definition of Done, schedule knowledge sessions, share content on a wiki.
  4. Requirements are too vague/ambiguous. Collaboration is the key in overcoming this pitfall. Communicate!
  5. Tools. Focus only on tools that add value to the team and that support the practices of the team. Decide as a team which tools to use and which not.

By then it was time for lunch, which is always a good occasion to mingle with other testers, discuss and have some fun. And to ravage that German buffet, of course. I had the impression that everyone was eagerly anticipating the keynote that would follow, which was Stuart Reid with “Agile Testing Certification – How Could That Be Useful“. It became clear that he wasn’t exactly going to preach for his own parish.

And a controversial talk it was. Twitter servers were moaning as Stuart’s quotes and graphic interpretations thereof were launched into #AgileTD cyberspace. Strangely enough, the infamous twitter fail whale was nowhere to be seen, which surprised me since the whole auditorium was filled with bug magnets. Stuart Reid started off by stating that it is only a matter of time before a qualification for agile testing is proposed and launched, whether we like it or not. He continued to say that if we want our industry as a whole to improve, we should exert our influence to help create a certification scheme we can truly benefit from. Fair enough. But what followed next confused me.

Stuart Reid stated that “the certification genie is out of the bottle” – what started as a good intention has spiralled out of control, and there’s no way back. This sounded like nothing more than a public dismissal of ISTQB to me, coming from one of the founding fathers. He proceeded to give an overview of the typical money flows in such a certification scheme, which was pretty enlightening. At one point, Stuart even managed to upset Elisabeth Hendrickson by stating that “it’s not because you are teaching Agile, that the training itself has to be Agile”. The movie clip of that very moment will live long and prosper on the internet. The whole “if you can’t beat them, join them”-idea bothered me too, as if there are no alternatives. Instead of focusing on certifications, we could try to educate employers, starting right at the top level. Certification programs exist mainly because employers don’t really know what qualities define a good tester. For them, a certification is merely a tool to quickly filter incoming resumes. Anyway, I think it’s good that Stuart initiated the debate, which would continue the rest of the conference.

The room was buzzing afterwards. Nothing better than some good old controversy to get the afternoon started. David Evans calmed things down again with “Hitting a Moving Target – Fixing Quality on Unfixed Scope“. He had some great visuals to support a thoughtful story. Some heavily tweeted quotes here:

  • QA in Agile shouldn’t be Quality Assurance but rather Questions and Answers
  • The product of testing is confidence (to which Michael Bolton quickly added that the product of testing is actually the demolition of false confidence).
  • Acceptance Test Driven Development (ATDD) slows down development just as passengers slow down a bus. We should measure the right thing.

Then it was Markus Gärtner‘s moment to shine in the spotlights. He presented “Alternative Paths for Self-Education in Software Testing“. During the last year, I got to know Markus as a passionate professional, dedicated to learning and advancing the craft. An overly active and ever-blogging guy that may have found the secret of the 27-hour-day. He opened with the question “who is in charge of your career?” Is it your boss? Your employer? Your family? Your teachers from high school? Well, none of that. It’s YOU. If you find yourself unemployed a year from now, everything you do now is contributing to you being employed quickly again.

Markus listed several ways of learning and self-improvement:

  • Books:
  • Courses
  • Buccaneer Scholaring, a way of taking your education in your own hands, based on the book Secrets of a Buccaneer Scholar by James Bach
  • Testing challenges – challenges to and by the Testing Community
  • Testing Dojos – principles: collaboration in a safe environment, deliberate practice. Usually consists of a mission which allows the testers to practice their testing and learning. Can happen with observers or facilitators, can be a good occasion to practice pair testing too.
  • Weekend Testing – A few hours of testing + debriefing in the weekend  according to a charter or a mission. I participated in a couple of European weekend sessions, and I must say: great learnings indeed. 
  • The Miagi-Do School of Software Testing, a school founded by software craftsman Matt Heusser. It’s a zero profit school where people can improve their skills, learn from others and share knowledge, using a belt system like in martial arts. They are not widely advertised – as Markus said: the first challenge is finding them.  

Janet Gregory‘s closing keynote fitted nicely in Markus’ theme, since it was all “About Learning“. It was an inspiring talk, about congruence in learning, the importance of learning, the curiosity of children – how their unspoiled curiosity makes them natural testers. She also related the learning to the agile principles. She managed to tie in neatly with Rob Lamberts presentation about structures and creativity. A safe environment helps you to learn. She referred to trust as an important element in team safety. A blame culture will work counterproductive. No-one will learn anything.

After all this theory about learning, we were all yearning for some hands-on practice. The Diaz & Hilterscheid gang gave us the opportunity to practice that typically German custom called Oktoberfest. Just like last year, they dressed up in Lederhosen (I’m actually getting used to the look of José in Lederhosen, go figure) and started serving plenty of local food and one-liter glasses of beer. There was live music as well, which added to a fun Bayerisches athmosphere. The evening culminated in some vivid discussions of the burning issues of the day. Well, actually there was only one burning issue: certification. Elisabeth Hendrickson was determined to get everyone mobilised for a worthy cause and whipped out her iPad on which she had written some kind of self-certification manifesto. Someone threw a pile of index cards on the table. Elisabeth was on fire and started handing them out everywhere. “If you agree with it, copy it. If you don’t, don’t”. Index cards on tables. Pens. Beer. Lots of people copying index card after index card till their fingers went in a cramp. That night witnessed the birth of a community of certified self-certifyers, all of them proudly carrying the message:

We are a community of professionals.
We are dedicated to our own continuing education
and take responsibility for our careers.
We support advancing in learning and advancing our craft.
We certify ourselves.

Some people took the discussions to the hotel bar, while others decided to dance the night away. I think I even spotted some genuine limbo-ing on the dancefloor. Someone ought to tell these testers about risk…

To be continued… Day 4

Agile Testing Days 2010 – Day 1 (Agile transitions)

Agile Testing Days 2010 – Day 1

After a great experience at the Agile Testing Days last year, I decided to answer their call for papers early. By the time the full program was announced (somewhere in april), I had almost forgotten that I participated. So it was a pleasant surprise to see my name listed among all those great speakers. I decided to break out of my comfort zone for once and in the last minute I “prezi-fied” my existing presentation. Confidently stressed, I flew east to Berlin to be part of what proved to be a wonderfully memorable conference. 

October 3

It was sunday October 3, which meant I arrived on the 20th anniversary of the German unification. The last time I had been in the city centre, Berlin was still a divided city. I was 16, and overwhelmed by the contrast between the neon-lit Ku’damm and the clean but spookily deserted East. Going through checkpoint Charlie to the East – and happily back again, while others desperately wanted to but couldn’t – still ranks among the most awkward moments in my otherwise pretty uneventful youth. Sure, the Alexanderplatz, Ishtar gate and Pergamon museum impressed me, but why a country would deliberately lock up its people was totally beyond my 16-year-old self.

So, with a few hours of daylight left, I headed to some sites that I still remembered from the days of yore. The Brandenburger Tor was now the backdrop for big festivities: music, beer, bratwurst and parachute commandos executing a perfect landing at Helmut Kohl’s feet at the Reichstag. No concrete walls to be seen. Unter den Linden completely opened up again. It felt great. Sometimes nostalgia isn’t what it used to be.

October 4

© Stephan Kämper

The morning of tutorial day, the Seminaris Hotel conference lobby was buzzing with coffee machines and activity. I had enrolled for Elisabeth Hendrickson‘s “Agile transitions” tutorial, which turned out to be an excellent choice. Eight people were taking part in the WordCount experiment, of which Elisabeth recounts an earlier experience here. After a round of introductions, we divided roles within the WordCount company: tester – developer – product manager – interoffice mail courier (snail mail only) – computer (yes, computers have feelings too) or observer. Strangely enough, I felt this natural urge to be a tester. I didn’t resist it, why should I? Elisabeth then proceeded to explain the rules. We would play a first round in which we had to stick to a set of fixed work agreements, like working in silos, formal handoffs and communicating only through the interoffice mail courier. The goal of the game was basically to make our customer happy by delivering features and thus earning money in the process.

We didn’t make our customer happy, that first round. On the contrary – confusion, chaos and frustration ensued. Testers belting out test cases, feeding them to the computer, getting back ambiguous results. Developers stressed out, struggling to understand the legacy code. Our product manager became hysterical because the customer kept harassing him for a demo and no-one was responding to his messages. The mail courier was bored, our computer felt pretty abandoned too. It all felt wonderfully unagile.

In round 2 we were allowed to change our work agreements any way we wanted, which sounded like music to our agile ears! We co-located immediately and fired our mail courier. We organised a big kickoff-meeting in which the customer would explain requirements and walk us through the application. We already visualised the money flowing in. In theory, theory and practice are the same. In practice – not so much. We spent a whole round discussing how we would work. We lost track of time. There were no new features, and no money. We felt pretty silly.

Round 3 was slightly better. We were able to fix some serious bugs and our first new features were developed, tested and working. But just when we thought we were on a roll, our customer coughed up some examples that she really wanted to pass too. They didn’t. 

Pressure was on in round 4, which was going to be the last one of the day. Would we make history by not delivering at all? Well, no. We actually reinvented ATDD, by letting the customer’s examples drive our development. This resulted in accepted features, and some money to go with that. We managed to develop, test and demo some additional functionalities too. A not-so-epic win, but a win nontheless. Wordcount was still in business. If there would have been a round 5, I’m pretty sure WordCount Inc. would have made a glorious entrance at the Nasdaq stock exchange.

Elisabeth did a great job facilitating the discussions in between rounds and playing a pretty realistic customer. All the participants made for a very enjoyable day too. The day really flew by and ended with a great speaker’s dinner at the borders of the Schlachtensee. A Canadian, an American, a German and a Belgian decided to walk back to the hotel instead of taking the bus. It sounds like the beginning of a bad joke, but that refreshing 5km walk through the green suburbs was actually the perfect closure of a terrific day. And without a map, I might add. As the rapid Canadian pointed out later: documentation is overrated.