Volcanic systems thinking

General systems thinking and the effects of a volcano outburst

Advertisements

I believe testing is about looking at things from as many perspectives as you can. Testing is also about relating things to one another, seeing things in a greater context.  In that sense you could say that testing is applied systems thinking. 

Years ago, Michael Bolton pointed me to the amazing book “Introduction to general systems thinking” by Jerry Weinberg. Actually, he pointed me to practically every publication by Jerry Weinberg – I’m still trying to prioritise my reading list. The book taught me that taking a holistic view of a system within its environment, may enable us to see patterns of behavior/actions and recognize interactions and interdependencies among its components. That way, we can better understand the system, maybe even predict how it will evolve over time.

The recent volcanic eruptions of the Eyjafjallajökull in Iceland provide a great example of (and also an exercise in) real-life general systems thinking and how several systems are possibly interconnected. I started wondering… What can the possible consequences of the recent volcanic eruption be, worldwide? There is a fair chance that other volcanoes nearby will erupt too. Scientists say history has proven that when the Eyjafjallajökull volcano erupts, the Katla volcano follows — the only question is how soon. And Katla, located under the massive Myrdalsjokull icecap, threatens disastrous flooding and explosive blasts when it blows.

If we can rely on history repeating itself, we’re onto something big. Let’s look at a true story. In June 1783, the eruption of the Icelandic Laki volcano was the start of a chain of unlikely events that affected everyone’s lives:

The immediate impact was catastrophic: around 25% of the Icelandic population died in the famine and fluorine poisoning after the eruptions ceased. Around 80% of sheep, 50% of cattle and 50% of horses died because of dental and skeletal fluorosis.

The rest of Europe soon followed:

  • A thick, poisonous smog cloud floated across the jet stream, resulting in many thousands of deaths throughout 1783 and the winter of 1784. Inhaling sulfur dioxide gas caused victims to choke. 
  • The thick fog caused boats to stay in port, unable to navigate.
  • Weather patterns started changing across western Europe:
    • The summer of 1783 was the hottest on record at that time. 
    • Severe thunderstorms with hailstones even killed cattle.
    • In 1784, a most severe winter caused 8,000 deaths in the UK.
    • During the melting that followed in spring, all of Europe reported severe flood damage.

And these were only the short term effects. The meteorological impact of the Laki eruption resonated on, contributing significantly to several years of extreme weather in Europe and the rest of the world:

  • In New Orleans, the Mississipi river froze, and ice started appearing in the gulf of Mexico.
  • African and Indian monsoon circulations weakened, leading to precipitation anomalies over the Sahel that resulted in low flow in the river Nile.
  • In France, a surplus harvest in 1785 caused poverty for rural workers.
  • For years afterwards, severe droughts followed. There were a series of bad winters and summers, including a violent hailstorm in 1788 that destroyed crops.
  • All this contributed significantly to the build up of poverty and famine that eventually triggered the French Revolution in 1789.

Wow. Time-out. Really? Could it be that the eruption of an Icelandic volcano lies at the basis of the French revolution, six years later? Since all these seemingly independent systems (meteorological, economic, agricultural and sociological) *are* connected, that’s perfectly plausible. Apparently, the indirect and long-term consequences of the eruption have greater impact than the initial effects of the event. Even art was affected. The most beautiful sunsets started appearing in late 18th-century paintings.

And what were the effects of the French revolution again? The abolition of Feudalism, for starters. The creation of a new order based on the famous ‘Declaration of the Rights of Man’. The main theme of the French Revolution, ‘Liberty, Equality and Fraternity’, later became one of the most famous political dogmas across the world. You could say that the Revolution paved the way for democracy. It brought about a lot of economic and social reforms, not only in France, but across Europe. Culture was also affected, at least in the short term, with the revolution permeating every creative endeavour. It changed the face of Europe: national identities joined forces, everywhere. In short: the revolution helped shape the future course of the world.

If we now think about the current eruptions of that rather friendly tongue-twister called Eyjafjallajökull, we notice that there are many more systems in play than there were in 1783. There’s commercial aviation and worldwide travel now; both of them fuel our economies and came to an abrupt stop. Airline stocks dropped rapidly. All this resulted in massive economic losses and temporary unemployment.  Several events are being cancelled because people cannot get there in time. People are trapped abroad or forced to stay at home. And these are only the short-term effects. I wonder what this year’s summer will be like. I already look forward to walking or skating across the channel to visit London. Or maybe throw a snowball or two at the gendarmes of Saint-Tropez.

And that’s just Eyjafjallajökull. Not Katla, not Laki. Just sayin’.

A lesson learned from James Bach

About “Secrets of a Buccaneer-Scholar” (James Bach)

I just finished James Bach’s “Secrets of a buccaneer-scholar” and it hit home in a weird way. I’m not an unschooler or a high school dropout, but I could still relate to a lot of things in his book. It was a tremendous read, giving me instant flashbacks to the days of yore.

As a young kid, I constantly skimmed through encyclopedia volumes that were lying around the house. I wasn’t “studying” from them, I was just fascinated by what I thought was all the knowledge of the universe compiled into 14 volumes. I let my mind wander while looking at the pictures, jumping randomly from subject to subject. When something looked fascinating enough to stay with it for a while, I dove in and read through the whole entry. I didn’t understand all of it, but I didn’t really mind. Most of the time it was just superficial browsing anyway – I blamed it on my short attention span. But as I was doing it more frequently, it became more systematic. Once in a while, I came across things that I previously ignored, but all of a sudden did seem interesting enough to investigate. Things I had previously read helped me to understand new things as well. I learned that I remembered lots of information without trying to. It just sticked because it was so damn interesting. I did the same thing with all the world maps and globes I could get my hands on. They really got my imagination running. The result is that I’m still bursting with trivia that spill out on the most inconvenient moments. It’s great in the occasional quiz, though.

I always thought that was a bit awkward. Not many kids I knew read encyclopedias and atlases in their spare time. It’s not that I didn’t enjoy school, but this kind of exploratory learning felt more natural to me. There was hardly any effort involved. It was pretty chaotic, but it was a learning style that fit me like a wet suit. 

As an adult, I am facing the same problems: I like to learn and educate myself, but in an almost impractical and inefficient way. I see interesting ideas and sources of knowledge everywhere, and this overwhelms me – so many things to learn, so little time! I purchase far more books than I can read (thanks for that, Paypal & Amazon). I start reading books but do not necessarily finish them. My reading isn’t very linear. I tend to get distracted often and feel the need to switch to something else. I procrastinate more than I would like. At this very moment, I’m trying to read nine books at the same time.

I used to feel bad about all this inefficiency. Until I finished James Bach’s book, a couple of hours ago.

It put things in perspective. It all makes a bit more sense to me now. Apparently it *is* okay and natural to let your mind wander. Allow yourself to be distracted. James calls it the “Follow your Energy”-heuristic: go with the flow of what engages your curiosity. Stick with what is fun and fits the natural rhythms of your mind. But in order to be more in control of your learning, combine it with the “Long Leash”-heuristic. Let your mind drift off, but in a controlled manner – keep it on a long leash. Remind yourself that you are on a mission and gently pull on the leash to regain focus again. 

These are just a couple of examples, but there’s more where that came from. In a way, a lot of the principles or heuristics described in the book reminded me of the young kid trying to work his chaotic way through that wealth of interesting information out there.

James Bach describes his pattern of learning with the “SACKED SCOWS” acronym:

  • Scouting Obsessively (…I discover the sources and tools I will need)
  • Authentic Problems (… engage my mind)
  • Cognitive Savvy (…means working with the rhythms of my mind) 
  • Knowledge attracts Knowledge (…the more I know, the easier I learn) 
  • Experimentation (…makes learning vivid and direct)
  • Disposable Time (…lets me try new things)
  • Stories (…are how I make sens of things)
  • Contrasting Ideas (…leads to better ideas)
  • Other Minds (…exercise my thinking and applaud my exploits)
  • Words and Pictures (…make a home for my thoughts)
  • Systems Thinking (…helps me tame complexity)

According to James Bach, a Buccaneer-Scholar is

“anyone whose love of learning is not muzzled or shackled by any institution or authority; whose mind is driven to wander and find its own place in the world”. 

So, am I a Buccaneer-Scholar? Maybe, I wouldn’t know. I wasn’t a rebel kid at war with the educational system – I actually enjoyed most of my time at school. I am not radically unschooling my kids, as James is doing. I wasn’t a whizz-kid either. I don’t think that’s the point. But I do love to learn new stuff, and preferably in ways that do not really make sense. At least, they didn’t until today.

Thank you, James.

Testing analogitis (a phantom menace)

A story about the phantom of Heilbronn and testing

Blame this post on my wandering mind. I’m suffering from a severe case of analogitis. I’m starting to see testing analogies everywhere.

On  April 25, 2007, a 22-year-old female police officer was fatally shot in Heilbronn, Germany. The analysis of the DNA that was found at the crime scene revealed some astonishing information. The DNA belonged to a woman and started popping up in several seemingly unrelated cases, some of them dating as far as fifteen years back. Traces were found:

  • on a cup after the killing of a 62-year-old woman in Idar-Oberstein, Germany
  • on a kitchen drawer after the killing of a 61-year-old man in Freiburg, Germany
  • on a syringe containing heroin near Gerolstein, Germany
  • on the leftovers of a cookie in a trailer that was forcefully opened in Budenheim, Germany
  • on a toy pistol after the robbery of Vietnamese gemstone traders in Arbois, France
  • on a projectile after a fight between two brothers in Worms, Germany
  • on a stone after a burglary in Saarbrücken, Germany
  • after a burglary at an optometrist’s store in Gallneukirchen, Austria
  • after 20 burglaries and thefts of cars and motorbikes Germany and Austria
  • on a car used to transport the bodies of three Georgians killed in Heppenheim, Germany
  • after a burglary in a disused public swimming pool in Niederstetten, Germany
  • after four cases of home invasion in Quierschied, Tholey and Riol, Germany
  • after an apartment break-in in Oberstenfeld-Gronau, Germany
  • after the robbery of a woman in a club house in Saarhölzbach, Germany
  • in the car of an auxiliary nurse who was found dead near Weinsberg, Germany

The so-called “phantom of Heilbronn” was born. An unknown woman was scattering her DNA all over the place, committing murders, breaking into houses, eating cookies, drinking beer, toting toy guns, she was even shooting up heroin. Profilers could tell that she was quite a busy lady, but she had no identity – the police had absolutely no clue. No Mentalist or CSI Heilbronn coming to the rescue – things were looking bleak.

But in March 2009, the case took a new turn. Investigators discovered the very same DNA sequence on the burned body of a male asylum-seeker in France – an anomaly, since the sequence was of a female. They eventually found out that the phantom serial killer did not actually exist and that the laboratory results were due to contamination of the cotton buds used for DNA probing. They discovered that the cotton swabs were already contaminated before shipping, and that they all came from the same factory. That factory, in turn, employed several Eastern European women who fit the type the DNA was assumed to match. The cotton swabs were not intended for analytical, but only medical use. They were sterile, but not certified for human DNA collection.

At this point you are probably wondering what all this has to do with testing. Bear with me a little.

Reading the Phantom of Heilbronn story reminded me of a phenomenon I briefly described in my 2007 paper “Software testing, profession of paradoxes?“. When we investigate something, the outcome of our investigations is sometimes influenced by the observation itself. The very act of testing influences its outcome. This axiom is also often referred to as the ‘observer effect’.

The idea is that since any form of observation is also an interaction, the act of testing itself can also affect that which is being tested. For example:

  • When log files are used in testing to record progress or events, the application under test may slow down drastically
  • The act of viewing log files while a piece of software is running can cause an I/O error, which may cause it to stop
  • Observing the performance of a CPU by running both the observed and observing programs on the same machine will lead to inaccurate results because the observer program itself affects the CPU performance
  • Observing (debugging) a running program by modifying its source code (e.g., adding extra output or generating log files) or by running it in a debugger may cause certain bugs to diminish or change their behavior, creating extra difficulty for the person trying to isolate the bug (also known as a ‘Heisenbug’)

Paradoxically, software testing is not always considered as the best way toward better quality. Just like in the Phantom of Heilbronn case, the advanced state of the technology might work against us, rather than with us. In 1990, Boris Beizer described the “complexity barrier principle” (1990): software complexity (and therefore that of bugs) grows to the limit of our ability to manage that complexity. 

Sometimes, testing is not the best option. Sometimes, testing and fixing problems does not necessarily improve the quality and reliability of the software. Oh sweet paradox, how can I embrace thee? Let me count the ways…

Pinpointing & iphone mindfulness

About pinpointing a bug on the iphone and mindfulness.

Two months ago my trusted smartphone started failing on me on an intermittent basis. In mid-conversation, people stopped hearing me, resulting in lots of confused hellos on the other end and eventually a whole lot of lost connectons too.

At first I was thinking along the lines of a hardware defect. My phone did end up on the ground a couple of times, and I thought that caused my microphone or antenna to be a bit dodgy. I decided to roam the internet in search of an explanation. That resource never failed me so far and I thought that with a little patience and well-chosen search terms I would be able to find some Slovakian or Belarusian forum-guy experiencing exactly the same problem. Well, none of that. I did encounter a lot of nationalities, and some of them I can’t even pronounce, let alone spell correctly. I also met a couple of Mac-savvy gurus with too much time on their hands, but no one recognized my problem or was able to help me.

I was puzzled. I had recently upgraded from version 2.2 to 3.0 – did the upgrade go awry? The problems started appearing around that time. Some more googling ensued. I got lost on strange webpages with flashing banners indicating I was visitor nr. 100,000. Another banner told me I had just won the green card lottery for a US visa – without even entering.  Miracles do exist. I got heavily distracted by ugly first-generation websites and decided to stop using the internet as my oracle.

I started doubting my googling and pinpointing skills, I even started questioning my phoning skills. How hard could it be to find the cause of a problem that is so easy to reproduce? I just needed to call someone or be called and within 5 minutes, people would stop hearing me. One hundred percent reproducible.

I was stuck here, so I decided not to think about it for a while to get a fresh angle on the problem. A couple of days later I noticed by chance that the problem didn’t occur when calling hands-free in the car over bluetooth. Wait, did the problem just solve itself, “automagically”? I called from my car again. No problem. I switched off the bluetooth setting and called manually. The problem re-occurred. Bingo!

Apparently it had something to with my phone-handling. As far as I knew I didn’t change the way I used my phone, but something was definitely different. I decided to be very conscious while calling, aware of every action I normally perform instinctively. Mindfulness for the iphone, but unfortunately without reaching the path to liberation and enlightenment. It must have looked rather strange to bystanders, but it eventually paid off. By using the phone in a kind of slow-motion way, I noticed that something was wrong with the light detection mechanism. The iphone is designed so that when you put the phone to your ear, the screen goes black to conserve power and to make it insensitive to touching. In this case it was the other way around: when I put the phone to my ears during a call, it went black but not much later turned back to an active, touch-sensitive state – something you normally don’t notice while being on the phone. Then I wondered… which options are active while making a call? And which one of these options would cause the problem of people not hearing me anymore, while I could still hear them?

I felt a little bit silly after realizing that I had been hitting the mute button with my cheek for two months without noticing.

This iphone incident taught me a valuable lesson about testing too. All too often, we do things instinctively, without really thinking. Or we think we are thinking, but our minds are actually on autopilot, and we’re just going through the motions. Becoming very conscious about what you are doing will improve our noticing skills. Noticing is a core testing skill that helps us reveal information about our environment, our actions, ourselves and the product we are testing. When we become more mindful, aware of what we are doing – focus on the current moment – we might become better testers, too.

All this learning thanks to a faulty iphone, you just gotta love Apple!

Through the looking-glass, and what testers found there – EWT05

Today it was weekend testing time again, and things got pretty crowded with a pleiad of international testers: facilitators Anna Baik and Markus Gärtner, !ndra from Hyderabad, Shiva Shankari, Krishnaveni K, Nagashree Manjunath, Jeroen RosinkAjay BalamurugadasJassi and myself.

The target today was Virtual Magnifying Glass, an open source screen magnifier for Windows, Linux, FreeBSD and Mac OS X.

The mission was basically: “Test this!”. Although a mission like that is way too vague to start from, it *is* a mission that we all encounter once in a while. It is our job to ask questions to make sure we understand what is needed.  So I wondered… do they just need bugs? Or an advice? Or information about how the application works? Ajay also went in questioning mode: “Test : create test ideas, hunt for bugs, compare with a different product, learn the product, so many things? What exactly is required?”. So with a little help of Ajay, the mission was rephrased to “Find quality related valuable information about the product”.

So far so good. I wanted to try pair testing over skype and Ajay was willing to team up with me. We set up a call and were soon discussing in person which approach to take. Although we encountered connection problems later on, it was a good experience that bears repeating.

Lessons learned:

  • It’s easy to feel frustrated because time is too short. After a round of introductions, some explanation and a discussion about the mission, there’s less than 30 minutes left to explore the software. Even for seasoned Rapid Testers this would be an uncomfortably short timebox. Avoid frustration by setting realistic goals for yourself.
  • When pairing, you’re not starting testing right away. There’s some discussion first, some setting up needs to be done too. Pairing is probably more beneficial when there is more time to test, so there’s plenty of time to talk things through. Now I felt a bit rushed because the clock was ticking.
  • Read-me files are a good place to start when exploring an unknown piece of software. In this case, they provided me with a good model of the software, a good basis to start exploring from.
  • Logging bugs takes time. Time not being spent testing. An issue also addressed in the “Why is testing taking so long (part1)(part2)” blogposts by Michael Bolton. 

Some nice, thought-worthy quotes were dropped during the debriefing too:

  • “Clearing traps is a skill. Recognizing traps is a bigger skill”
    Ajay Balamuragadas (c) 2010
  • “Minds are shaped when guided under pressure in a certain direction trying to maintain vision and control” 
    Jeroen Rosink (c) 2010
  • “Watch out where the huskies go, and don’t you eat that yellow snow” 
    Frank Zappa (c) 1974

Well… I guess all that snow is finally getting to me. Frank never made it to this European Weekend Testing session. He wasn’t much of a tester either. But he sure knew how to play that guitar. And I think he would have been a great explorer, having fun all the way.

Behold, the “better Tester”

Reaction to a discussion in the “Software Testing & Quality Assurance” LinkedIn-group

There are some strange and intriguing questions being posed in the Software Testing & Quality Assurance LinkedIn group lately. “Who are Testing Criminals?“, “Who should take the blame for defects in production software ? ” or “Test Plan ! Do we really follow it ! ” (an awful lot of exclamation marks for a question, if you ask me) to name but a few. I read them through, bit my tongue, controlled my breathing and managed to ignore them in the end. 

But two days ago, another gem was posted:

Who is better Tester?

@Who finds highest defects..?
@Who can say in 100% confidence that this product\application is BUG\DEFECTS free..?
@Who creates highest testing scenarios,Testcases,etc..?
@Who has as many as software testing certifications like ISTQB or something like that…?

Mmm… I thought about Pradeep Soundararajan’s blog post about the infamous Black Viper Testing Technique as a provoking response to another discussion in that same group, and decided to react.

Who is the better tester? Well… None of them, actually.

  • “Highest number of defects”
    What does that tell you, out of context? Maybe the person that found the highest number of defects has tested more hours than the rest. Maybe he tested a buggy part of the product, while others were concentrating on a more “mature” part. Maybe he only found “low priority” bugs, while others – with less bugs on their list – found more severe showstoppers. Which tester would you prefer?
  • “100% Bug free”
    I wouldn’t hire a tester that dares to make claims such as “this software is 100% bug free”. It isn’t. It can’t be. If you continue testing, more bugs will be found. This is the catch-22 of testing. We stop testing when we’re finished, but we’re never finished. So we stop testing because of other reasons.
  • “Highest number of test cases/scenarios”
    Again, without context, this is a worthless statement. Maybe he works faster, but more sloppy. Maybe the other testers were investigating bugs they found while scripting. Time spent hunting/pinpointing bugs is valuable, and if testers are asked to engage in that, they’re also not “writing tests”. Maybe other people are designing tests in a slower pace because they tend to talk to stakeholders about the requirements first, to be sure that they understand what is really going on. The person with the highest test case count may be designing tests badly, or testing things the wrong way. Or maybe he’s just testing the wrong things. The productivity of a tester can/may never be measured in number of test cases, but this seems to happen all too often.
  • “Number of certifications”.
    What do certifications tell you? That a tester was able to sit through a short course and was able to score +28 out of 40 on a multiple choice exam? Does certification actually tell you anything about the real skills the tester possesses? Is he able to think critically, to question the product in order to evaluate it?

I would say the better tester is the one that adds the most value with his testing, one that is able to explore the software and recognizes problems. A good tester doesn’t just ask ‘Pass or Fail?’. A good tester asks ‘Is there a problem here?’ (props to Michael Bolton for that one).

There. I said it. I actually felt relieved after writing that down. Who is better Self-therapist?

C is for Conferring

On conferences and how to keep it cheap

The blogosphere is pregnant with conference posts these days. And good ones too I might add.
  • In his blogpost conferences on the cheap, Matt Heusser gives some helpful tips on keeping the cost of attending testing conferences down in these difficult times.
  • Matt also created a conference wiki where everyone can add any kind of testing event or conference that is taking place. A great source of information, that is still expanding as we speak. If you know of any noteworthy events that are not yet listed, feel free to add them. 
  • Very recently, the Software Testing Club spawned an interesting testing conference discussion whether or not people are paying for conferences out of their own pocket. 

I think being at conferences is great fun. You meet peers. You talk and share ideas. And get ideas, as well. Occasionally, you attend a track that makes you go “what was that all about?!”, but in general I get tremendously inspired. Not only from the track sessions – most of the time the hallway discussions provide some decent food for thought as well. Of course these benefits aren’t always very measurable, but conferences are doing their very best to provide attendees with ways to convince their managers and prove the added value. Eurostar even provides a justification kit and a Conference Evaluation & ROI Worksheet for reporting on the ROI back home.

But there’s always other ways: 

  • You can get in for free as a speaker, but in that case there’s some effort involved. Writing abstracts, papers, making a presentation. Can be quite a hassle – tit for tat. And you have to get selected of course.
  • Some conferences give away conference tickets in lotteries. Slim chance, but you never know.
  • Step out of the dark side and show your face to the community
    – be a video star. Are you ready for your close-up, <insert name here>?