Children’s own pass/fail criteria (and nursery rhymes)

One month ago, my oldest daughter (6) started taking on rope skipping. The last time I had seen her practising, two weeks ago, she was still having trouble getting the rope neatly over (and under) herself, but yesterday she was able to complete several jumps in one go in a fluent movement. It was the first time I had seen her do that, so I was pretty impressed.

She was clearly in learning mode. I sat down to observe her more closely. 

– “Wow, where did you learn all that?”

– “I’ve been watching older girls do that in school, daddy. Watch”.

She started jumping and counting out loud.

– “One, two, three, four, five, six, …”

She tripped on the rope.

– “Woohoo! Six!”

– “You go, girl!”

– “Again! One, two, three, four, five, nooooo…”

– “Five is good”.

– “No, daddy, five is not good. Again!”

She repeated the process a couple of times. She jumped seven (“Yes!”), four (“Nooo!”), five (“Pfff!”), six (“Yippie!”). I started noticing a pattern. It struck me that she alternated frustration with joy, and she let it depend on the number of jumps. Time for some questioning.

– “Why are you happy with anything above or equal to six, but unhappy with anything lower?”

– “It has to be at least six, daddy”.

– “Why six?”

She seemed really annoyed that I didn’t see her point. She thought I was pulling her leg.

– “Because I’m _six_ years old, daddy. Didn’t you know? What else could it be?”.

I was totally flabbergasted. She managed to impose some totally arbitrary pass/fail criteria on herself. Where did that come from? I thought that using pass/fail tests actually sabotages kids’ natural learning processes? But this appeared to come out of herself. No-one told her that she had to make at least six.

I wondered – maybe she just chose her age as a starting point, just to set some initial learning goals for herself? Was she planning on raising the bar later on when reaching six would have become too easy? Unfortunately I didn’t have the chance to follow-up on that – lunchtime!

Flash forward to work. All this reminded me of commonly defined pass/fail criteria such as

“90% of all tests must pass”

Really? 

In “Are your lights on?”, Jerry Weinberg uses the well-known “Mary had a little lamb” nursery rhyme to show how a seemingly straightforward statement is prey to multiple interpretations, depending on which word you emphasize. An invaluable heuristic when looking at requirements. Why not try that on the familiar pass/fail criterium stated above?

“90%”? What if the tests that would have revealed some serious errors happen to be in that 10% you so confidently dismissed? Why not 89 or 91?

“All”? You know “all” possible tests that can be performed? Are they all documented? Some of them might still be residing in your head. What if in the meanwhile we performed some more important tests that revealed serious risks? Are these tests part of “All”?

“Tests”? Do you only count scripted tests, or do you also take exploratory ones into account? What about important usability issues some users might have found? Or acceptance test checklists? Or automated checks? 

– “Must”? What if not all 90% passes? Does this mean your solution is without value? The customer might value other things than you do. Is it up to you to decide how much value is in there?

“Pass” ? What about behavior that is totally acceptable for your client, but that we find annoying? Pass or fail? What about tests that pass all steps, but that reveal important problems as a side-effect? Sometimes a test’s pass/fail decision is not binary.

My daughter went to school this morning and – for the first time –  took her own jump rope with her. I wonder how many % of her rope jump cases will pass this time.

Advertisements

Delivering the message

We are testers. We communicate, collaborate and report. We are the headlights of a project. We light the way, break the news, deliver messages – and they’re not always pretty. How do we make sure our point is taken? It all boils down to adapting our style of delivery depending on the content and the audience. Context is indeed everything.

The delivery of a speech is mostly done through the nonverbal channel (whereas the content itself is verbal). This includes all speech elements other than the words themselves: eye contact, voice, articulation, gestures, facial expressions, body language – even appearance. Using all of these effectively requires timing and practice, lots of practice. An additional problem is that there is a wealth of delivery styles to choose from. How do we get the message across in the best possible way?

A really smart person once said: “Smart people learn from their mistakes, but REALLY smart people learn from other people’s mistakes.”

I think that’s true. In the video below, councillor Phil Davison – a candidate for the position of Republican treasurer for Stark County, Ohio –   delivers a passionate nomination speech in front of a receptive audience. His delivery style is – well… – pretty peculiar. 

Spoiler: he wasn’t  elected.

Feynman on naming

How Feynman’s take on naming things is applicable to testing

Feynman’ s father Melville played a big role in shaping little Richard’s way of thinking. He used to read bedtime stories from the Encyclopedia Britannica.

“See this Tyrannosaurus Rex over here? It says here that this thing was 25 feet high, and the head was six feet across. That means that if it stood in our front yard, it would be high enough to put his head through the window, but not quite because the head is a little bit too wide, it would break the window as it came by”.

He always tried to translate things into some kind of reality, so little Richard would be able to figure out what it really meant, what it was really saying.

Melville would also take his kid for long walks in the Catskill mountains, telling him about nature and explaining that in order to really *know* something, you should start observing and noticing instead of merely naming (a thing most of his classmates seemed to do):

“You can know the name of a bird in all the languages of the world, but when you’re finished, you’ll know absolutely nothing whatever about the bird… So let’s look at the bird and see what it’s doing — that’s what counts. I learned very early the difference between knowing the name of something and knowing something.”

From “The pleasure of finding things out” (1981)

I think the above quote illustrates a phenomenon that occurs all too often in software testing: the nominal fallacy. Basically, this means applying a label or name to something and thinking you have explained it.

What about boundary value testing (or domain testing), for instance?

“First, we identify the boundaries, then we identify tests at each boundary. For example, one test each for >, =, <, using the first value in the > range, the value that is equal to the boundary, and the first value in the < range”.

A pretty straightforward and effective technique, right? We think we master it, until we realise that most textbooks are only talking about known and visible boundaries. What about the boundaries that are not known, not even by the developers that wrote the code? Most of the time, the software’s complexity stretches far beyond our imagination.

In case you need convincing: let’s revisit the ParkCalc exercise that took place a couple of months ago (here‘s a good write-up of that event by Selena Delesie). Matt Heusser put out a challenge to test ParkCalc, a small app on the Grand Rapids Airport website to calculate parking fees. The site provided entry and leaving times, date pickers and five different kinds of parking to choose from. Quickly, an instant test-flashmob formed via twitter. Testers from all over the globe jumped on the bandwagon. James Bach joined in and challenged everyone to beat his highest total parking fee. Extreme testing ensued. What followed was a good exercise in factoring, investigation and on-the-fly test design. And it happened to illustrate the complexity of boundary value analysis as well. 

To get an even better idea of this complexity, there’s always Cem Kaner‘s online course on domain testing. Do we know boundary value analysis because we know its most basic definition?

I’m not trying to open Pandora’s box here, but these nominal fallacies also apply to a testing certification that mainly focuses on definitions. Naming things isn’t enough. As Feynman put it: knowing something requires practice and observation. The benefit? No need to memorize anything anymore when real understanding kicks in. 

Of course, all this isn’t new. Half of the starr-cross’d lovers were already aware of this, way back in the late sixteenth century:

“What’s in a name? That which we call a rose
By any other name would smell as sweet.”

(Juliet Capulet)

Exploring Feynman

On my intention to start exploring Richard Feynman (1918-1988)

The Plan

I’m planning to do a little blog series on the late Richard Feynman to record some of my impressions and learnings while I work myself through this intriguing oeuvre of his. No, that summer heat is not getting to me, yet. I’m not exactly planning on processing his massive back catalog – I’m not really into path integral formulation or the behavior of subatomic particles, let alone the superfluidity of supercooled liquid helium. I do value the sparse free time that I have – time is on my side, yes it is. Rather, I’d like to document my exploration of his more popular works, audio and video recordings.

Exploratory learning, as you wish. Dipping into it all and savoring the juicy bits, spitting out the others. And relate things to testing, of course.

Why Richard Feynman?

Feynman intrigues me, and I have nothing but deep respect and admiration for the man. He was witty, brilliant and had this perpetual curiosity to discover new things (Tuvan throat-singing, anyone?). He opposed rote learning or unthinking memorization as teaching methods – he wanted his students to start thinking, for a change. How great is that?

On occasion, he was a totally nutty professor – a flamboyant geek. But he also happened to build a truly astonishing career which eventually earned him the Nobel prize in physics in 1965. 

I’m planning to gradually learn about him and post my progress here. Stay tuned!

A lesson learned from James Bach

About “Secrets of a Buccaneer-Scholar” (James Bach)

I just finished James Bach’s “Secrets of a buccaneer-scholar” and it hit home in a weird way. I’m not an unschooler or a high school dropout, but I could still relate to a lot of things in his book. It was a tremendous read, giving me instant flashbacks to the days of yore.

As a young kid, I constantly skimmed through encyclopedia volumes that were lying around the house. I wasn’t “studying” from them, I was just fascinated by what I thought was all the knowledge of the universe compiled into 14 volumes. I let my mind wander while looking at the pictures, jumping randomly from subject to subject. When something looked fascinating enough to stay with it for a while, I dove in and read through the whole entry. I didn’t understand all of it, but I didn’t really mind. Most of the time it was just superficial browsing anyway – I blamed it on my short attention span. But as I was doing it more frequently, it became more systematic. Once in a while, I came across things that I previously ignored, but all of a sudden did seem interesting enough to investigate. Things I had previously read helped me to understand new things as well. I learned that I remembered lots of information without trying to. It just sticked because it was so damn interesting. I did the same thing with all the world maps and globes I could get my hands on. They really got my imagination running. The result is that I’m still bursting with trivia that spill out on the most inconvenient moments. It’s great in the occasional quiz, though.

I always thought that was a bit awkward. Not many kids I knew read encyclopedias and atlases in their spare time. It’s not that I didn’t enjoy school, but this kind of exploratory learning felt more natural to me. There was hardly any effort involved. It was pretty chaotic, but it was a learning style that fit me like a wet suit. 

As an adult, I am facing the same problems: I like to learn and educate myself, but in an almost impractical and inefficient way. I see interesting ideas and sources of knowledge everywhere, and this overwhelms me – so many things to learn, so little time! I purchase far more books than I can read (thanks for that, Paypal & Amazon). I start reading books but do not necessarily finish them. My reading isn’t very linear. I tend to get distracted often and feel the need to switch to something else. I procrastinate more than I would like. At this very moment, I’m trying to read nine books at the same time.

I used to feel bad about all this inefficiency. Until I finished James Bach’s book, a couple of hours ago.

It put things in perspective. It all makes a bit more sense to me now. Apparently it *is* okay and natural to let your mind wander. Allow yourself to be distracted. James calls it the “Follow your Energy”-heuristic: go with the flow of what engages your curiosity. Stick with what is fun and fits the natural rhythms of your mind. But in order to be more in control of your learning, combine it with the “Long Leash”-heuristic. Let your mind drift off, but in a controlled manner – keep it on a long leash. Remind yourself that you are on a mission and gently pull on the leash to regain focus again. 

These are just a couple of examples, but there’s more where that came from. In a way, a lot of the principles or heuristics described in the book reminded me of the young kid trying to work his chaotic way through that wealth of interesting information out there.

James Bach describes his pattern of learning with the “SACKED SCOWS” acronym:

  • Scouting Obsessively (…I discover the sources and tools I will need)
  • Authentic Problems (… engage my mind)
  • Cognitive Savvy (…means working with the rhythms of my mind) 
  • Knowledge attracts Knowledge (…the more I know, the easier I learn) 
  • Experimentation (…makes learning vivid and direct)
  • Disposable Time (…lets me try new things)
  • Stories (…are how I make sens of things)
  • Contrasting Ideas (…leads to better ideas)
  • Other Minds (…exercise my thinking and applaud my exploits)
  • Words and Pictures (…make a home for my thoughts)
  • Systems Thinking (…helps me tame complexity)

According to James Bach, a Buccaneer-Scholar is

“anyone whose love of learning is not muzzled or shackled by any institution or authority; whose mind is driven to wander and find its own place in the world”. 

So, am I a Buccaneer-Scholar? Maybe, I wouldn’t know. I wasn’t a rebel kid at war with the educational system – I actually enjoyed most of my time at school. I am not radically unschooling my kids, as James is doing. I wasn’t a whizz-kid either. I don’t think that’s the point. But I do love to learn new stuff, and preferably in ways that do not really make sense. At least, they didn’t until today.

Thank you, James.