Category: Heuristics

Finding Porcini

A couple of weeks ago I found myself in southwestern France, a region which – at the time – was being struck by a unseen spell of global wetting. Summer had arrived three months early, people said. April and may had been exceptionally dry and warm; but in July, autumn was knocking on the door of our vacation home. Early autumn, I was told by our gentle host Philippe, goes hand in hand with a peculiar phenomenon: fungi frenzy/mushroom madness. All the locals get this strange misty-eyed look and head for the woods to hunt for the precious boletus edulis, more commonly known as  fungo porcino, porcine mushroom, cep and lovingly referred to as “the brown plague”, as it tends to halt the local economy.

When Philippe invited me to join him on an early morning quest for porcini, I gladly accepted. It sounded like a treasure hunt, as fresh porcini are sold for outrageous prices to local restaurants. And who doesn’t enjoy a good treasure hunt after breakfast? We left for the woods, armed only with wooden baskets, a knife and a sturdy 4×4.

Mushroom hunting, I found out quickly, is an art in its own right. Slowly and carefully, like an old sensei, Philippe unveiled his mushroom hunting mysteries. And as the mysteries disappeared, a neat set of categorized heuristics came trickling through:

The Mission

  • “Ready Zeger? So, we’re looking for cèpes, têtes de negres and chanterelles. Leave the others be. Some are poisonous, others just don’t taste nice”
  • “When we stop? When we’re finding more mushrooms than we can carry. Or when we’re not finding anything anymore.”

Classification

  • “Wait, you don’t know what to look for, do you? Come over here for a sec. This is a vintage cèpe. A fungho porcino. Like all other boletes, it has tubes extending downward from the underside of the cap, rather than gills. The pore surface of fruit body is whitish when young, but ages to a greenish-yellow when older.”
  • “Be careful there. Some mushrooms look very much like porcini. You should look under the hood. If it’s yellow, don’t touch ’em.”
  • “When you’re not really sure, scratch the bottom of the hood. When the scratch turns purple, don’t touch.”

Timing

  • “Why we’re heading out all of a sudden? Porcini tend to appear after summer peaks, depending on the weather. Usually they pop up about a week after a wet spell.”
  • “Oh no, that’s not true, Zeger. The fact that we picked loads of mushrooms here doesn’t mean I’m not coming back here tomorrow. These things grow fast. They often push overnight, you know.”

Location

  • “Location is everything. You should look at open spots in the woods, where the sun can actually reach the ground. Look, that should be a good place over there. Do you see the sunbeams peaking through the leafs? Let’s head over there.”
  • “When you find one, mind your step. Where there is one, there are many. You might crush some perfectly good ‘shrooms hidden under some leaves or grass.”
  • “Spotting porcini takes a trained and experienced eye. Here’s a pro-tip: look for where the leaves bunch up – perhaps they are being pushed up by a growing mushroom.”
  • “Don’t spend your time looking near ferns, man. Ferns grow on intensely acid soil, porcini don’t”
  • “Hey Zeger! You see this black beauty here? This is a tête de nègre, a particularly tasty and expensive kind of cep. Look for them near oak trees.”
  • “This here’s a chanterelle. If you spot two of them, follow the line that connects them; they always grow on a straight line.”

Picking technique

  • “Use a knife. Never ever pull mushrooms out of the ground. Cut them. If you damage their mycelium, they won’t grow again next year.”
  • “Remember: be gentle, cut them near the bottom of the stem in a straight line. Don’t break the hood.”
  • “Wait! Don’t cut the really small ones – they’ll be worth much more later on.”

I was soaking with sweat after a couple of hours of intense scouting. In a short timespan, Philippe managed to transform me into a die-hard mushroom hunter. A novice still, but I felt I was learning quickly. Philippe’s heuristics (not best practices, mind you) helped me discern the good from the bad, finding porcini hidden under leaves and cantharelles in a neat straight line. I even developed my own heuristics as I went along: I started looking alongside paths through the woods – plenty of chances for the sun to peep through the deck of leaves, and easier to spot since the vegetation is less dense.

Testing porcini

As I was wandering through the woods with eagle eyes and at a snail’s pace, it all felt strangely familiar. When Philippe said “Where there is one, there are many”, it struck my tester chord. Here I am, a tester, looking for mushrooms, which doesn’t seem to be all that different than looking for bugs. No wonder I liked it so much. I also realized that when I’m looking for bugs, I use these kind of heuristics all the time, but all too often I’m not very aware of them. Which is a pity, because used consciously, these heuristics (“a fallible method for solving a problem”) can be a really powerful tool to boost your exploratory testing efforts.

  • Start with a mission – make sure you – and your team – know what to look for, since our conception limits our perception. Michael Bolton often quotes Marshall McLuhan on this: “I wouldn’t have seen it if I hadn’t believed it”
  • Make sure you’ve got your classification right. If you’re only interested in a specific kind of bug, maybe you shouldn’t waste time reporting others. You could consider parking them somewhere, or keeping the reporting rather lightweight by MIPping (mention in passing) them. But try to stick to your main focus for the session. And if you find a nice-looking bug, is it really? Scratch it, it might turn purple
  • Timing – as in mushroom picking – is also a factor to be considered in bug hunting. Are there typical times at which the application is less stable? When is an ideal testing time, really? Again, this is largely dependent on context
  • Location, location, location. Personally, I use many heuristics to guide me where to test. Which areas are more vulnerable? When you find one, there tend to be many others, indeed. As leaves bunching up *might* indicate a pushing mushroom, seemingly insignificant facts might be a tell-tale sign for bugs nearby: the code that developers write after a wild night of partying might not be all that good, for example. Or they can just have a bad day. I was once told by an old native American medicine man that developers are human too
  • As some mushrooms are picked and not cut, our bag-o-techniques should enable us to deal with any situation. As Lee Copeland points out in A Practitioner’s Guide to Software Test Design: a tester should carry his techniques with him at all times, just like a handyman’s toolbox follows him around everywhere he goes. Apply a specific technique, use an particular approach when the situation calls for it.

For the record: I’m not a mushroom master, yet. I lack practice, experience and domain knowledge to attain mastery. I’m not a testing master either, as I’m in constant learning mode. For every good practice I know, in context, I am aware that there’s always another context that I need to get myself familiar with. That prospect may seem humbling and daunting to many, but I wouldn’t want it any other way. That’s Context-Driven Testing for ya.

(For more info, see The Seven Basic Principles of the Context-Driven School as a starting place. There’s a lot more where that came from).

Advertisement

A lesson learned from James Bach

About “Secrets of a Buccaneer-Scholar” (James Bach)

I just finished James Bach’s “Secrets of a buccaneer-scholar” and it hit home in a weird way. I’m not an unschooler or a high school dropout, but I could still relate to a lot of things in his book. It was a tremendous read, giving me instant flashbacks to the days of yore.

As a young kid, I constantly skimmed through encyclopedia volumes that were lying around the house. I wasn’t “studying” from them, I was just fascinated by what I thought was all the knowledge of the universe compiled into 14 volumes. I let my mind wander while looking at the pictures, jumping randomly from subject to subject. When something looked fascinating enough to stay with it for a while, I dove in and read through the whole entry. I didn’t understand all of it, but I didn’t really mind. Most of the time it was just superficial browsing anyway – I blamed it on my short attention span. But as I was doing it more frequently, it became more systematic. Once in a while, I came across things that I previously ignored, but all of a sudden did seem interesting enough to investigate. Things I had previously read helped me to understand new things as well. I learned that I remembered lots of information without trying to. It just sticked because it was so damn interesting. I did the same thing with all the world maps and globes I could get my hands on. They really got my imagination running. The result is that I’m still bursting with trivia that spill out on the most inconvenient moments. It’s great in the occasional quiz, though.

I always thought that was a bit awkward. Not many kids I knew read encyclopedias and atlases in their spare time. It’s not that I didn’t enjoy school, but this kind of exploratory learning felt more natural to me. There was hardly any effort involved. It was pretty chaotic, but it was a learning style that fit me like a wet suit. 

As an adult, I am facing the same problems: I like to learn and educate myself, but in an almost impractical and inefficient way. I see interesting ideas and sources of knowledge everywhere, and this overwhelms me – so many things to learn, so little time! I purchase far more books than I can read (thanks for that, Paypal & Amazon). I start reading books but do not necessarily finish them. My reading isn’t very linear. I tend to get distracted often and feel the need to switch to something else. I procrastinate more than I would like. At this very moment, I’m trying to read nine books at the same time.

I used to feel bad about all this inefficiency. Until I finished James Bach’s book, a couple of hours ago.

It put things in perspective. It all makes a bit more sense to me now. Apparently it *is* okay and natural to let your mind wander. Allow yourself to be distracted. James calls it the “Follow your Energy”-heuristic: go with the flow of what engages your curiosity. Stick with what is fun and fits the natural rhythms of your mind. But in order to be more in control of your learning, combine it with the “Long Leash”-heuristic. Let your mind drift off, but in a controlled manner – keep it on a long leash. Remind yourself that you are on a mission and gently pull on the leash to regain focus again. 

These are just a couple of examples, but there’s more where that came from. In a way, a lot of the principles or heuristics described in the book reminded me of the young kid trying to work his chaotic way through that wealth of interesting information out there.

James Bach describes his pattern of learning with the “SACKED SCOWS” acronym:

  • Scouting Obsessively (…I discover the sources and tools I will need)
  • Authentic Problems (… engage my mind)
  • Cognitive Savvy (…means working with the rhythms of my mind) 
  • Knowledge attracts Knowledge (…the more I know, the easier I learn) 
  • Experimentation (…makes learning vivid and direct)
  • Disposable Time (…lets me try new things)
  • Stories (…are how I make sens of things)
  • Contrasting Ideas (…leads to better ideas)
  • Other Minds (…exercise my thinking and applaud my exploits)
  • Words and Pictures (…make a home for my thoughts)
  • Systems Thinking (…helps me tame complexity)

According to James Bach, a Buccaneer-Scholar is

“anyone whose love of learning is not muzzled or shackled by any institution or authority; whose mind is driven to wander and find its own place in the world”. 

So, am I a Buccaneer-Scholar? Maybe, I wouldn’t know. I wasn’t a rebel kid at war with the educational system – I actually enjoyed most of my time at school. I am not radically unschooling my kids, as James is doing. I wasn’t a whizz-kid either. I don’t think that’s the point. But I do love to learn new stuff, and preferably in ways that do not really make sense. At least, they didn’t until today.

Thank you, James.

The ongoing weekend testing adventure – EWT03

A write-up of European Weekend Testing session 3 – EWT03

I had to skip the previous weekend testing session involving Bing maps (which was a pity – I absolutely adore map applications), but today I was able to participate in EWT03. It was again an interesting experience. Not too many participants this time, which made for a more cozy atmosphere. Today’s testers were Marlena Compton, Anna Baik, Markus Gärtner and myself.

The Mission: You work in a small-medium company, and your manager has been asked to evaluate switching the company over to using Google calendar. He needs a quick assessment from you before his conference call this afternoon. Use the FCC CUTS VIDS touring heuristic to guide you.

We started firing off a bunch of questions at the boss, but he was in a hurry and kind of unresponsive – something about an important golf game he had to attend. He just wanted a quick assessment of  the tool, he apparently wasn’t into answering today. So much for questioning – we were on our own pretty soon. I wonder if there’s any way to keep an imaginary boss from leaving a virtual meeting. Still not sure about that.

We decided to divide the coverage since we started late and there was less than half an hour of testing time left. We ended up cutting the FCC CUTS VIDS into – surprise! – FCC, CUTS and VIDS (yes, sometimes you just go for the obvious). I settled for the VIDS:

  • Variability tour: look for things you can change in the application – and then you try to change them.
  • Interopeability tour: what does this application interact with?
  • Data tour: identify the major data elements of the application.
  • Structure tour : find everything you can about what comprises the physical product (code, interfaces, hardware, files, etc…)

In the meanwhile the boss had magically reappeared to give us a quarter of an hour extra – he probably forgot his bag of golf clubs. But I quickly realised that time would still be way too short to look at all aspects and concentrated on variability and interoperability – and on a list of existing bugs as well, something I remembered from the first session. In my quest for things that can be changed in the application, I settled for the system date, not really a part of the application but still something that can change while you’re using it. Some interesting bugs ensued, including a very severe one in Skype group chat – not really part of the mission. I thought I had lost connection but it turned out that the order of the chat messages was totally messed up. It took a while before I realised what had happened and lost a good deal of time because of that. Later on, I switched to interoperability but I also lost my internet connection for for a while, which was annoying but in retrospect also pointed me to a synchronisation bug.

The debriefing, moderated by Anna, revealed some good points from the other participants. Marlena covered the FCC (feature, complexity, claims) part and did a feature tour first, which proved very helpful. She explored all functionalities to get more familiar with them before she went off to investigate other areas. She wondered how the rest of us could start testing without doing such a feature tour/exploration first. But in fact, I think we all started with a bit of general exploration. I know I did. It feels kind of natural, probing around the surface for a while before before diving into the abyss of detail. Concerning the splitting up of the heuristic among several testers, this turned out to be a bit distracting, confusing and counter-productive. There is quite some overlap between the different areas and sticking to only some of them might mean that you’re missing out on otherwise obvious problems. This led us to the use of sapience in testing, being able to picking the heuristics that are useful for the given features.

We didn’t get to brief the boss in the end, so you could say that our mission didn’t really succeed. But hey, some say that failure breeds success. Soon, things will be looking so bright we’ll have to wear shades.

Be careful about the claims you make

About heuristics and an unbreakable phone.

An oracle is a principle or a mechanism that helps us in recognizing if the software under test is working according to some criteria. A heuristic is a fallible way of solving a problem; it can also help us discover and explore. James Bach and Michael Bolton have provided us with a lot of helpful heuristics already. One of them is a consistency heuristic mnemonically called HICCUPPS. Michael wrote an article about his use of this particular heuristic here (pdf). The idea is that a product should be consistent with:
  • History (consistent with it’s past behavior)
  • Image (consistent with the image the organization wants to project)
  • Comparable Products (consistent with a similar product)
  • Claims (product behaves the way some document or person says it should – e.g. in advertisements)
  • User’s expectations (consistent with what user wants or expects from the product)
  • Product (behavior should be consistent throughout the whole product) 
  • Purpose (consistent with its apparent purpose)
  • Statutes (behaving in compliance with legal or regulatory requirements) 

Although this is all about hardware, I was reminded of the ‘Claims’-part of this heuristic when I recently saw a video of a BBC reporter at the Consumer Electronics Show (CES) who broke the alledgedly unbreakable SONIM phone on his first attempt. Examples don’t come any clearer than this. Don’t go about telling that your product is unbreakable unless you’re 100% sure it is. Then again, how can you ever be 100% sure? When you put up a fish tank for everyone to use, expect the unexpected. That’s an open invitation for creative people to think ‘out of the tank’. I guess they’ll have to revise their marketing tagline. How about ‘Unbreakable by anything but edges of fish tanks’?.

* edited typo: ‘Statutes’ instead of ‘Statuses’