Lets Test Keynote: Michael Bolton - If It’s Not Context-Driven, You Can’t Do It Here

Posted By Duncs ~ 26th May 2012

Michael Bolton had the priviledge of the opening keynote of the conference & unsurprisingly he chose to talk about context-driven testing. Turns out his tongue-in-cheek title was taken as such by everyone, so he had to take some time explaining the irony in it! Slides for the “If It’s Not Context-Driven, You Can’t Do It Here” session are available here

The keynote was based around a comparision between how science is changing & so is testing. The keynote was framed by CBC’s “Thinking About Science” series & namely the first episode (with Simon Schaffer) which focussed on a path breaking book “Leviathan and The Air Pump (Hobbes, Boyle and the Experimental Life)” by Steven Shapin and Simon Schaffer.

Michael used the CBC series & the book to back up his theory about the evolution of testing.

The book is about the debates between Robert Boyle & Thomas Hobbs around Boyle’s air pump. Boyle liked to experiment to generate knowledge which was separate from universal theory, was based on probabiliy & as such was more “matter of fact”. Hobbs was on the other side of the coin - In order for something to be a fact, he required “absolute certainty” based on “logic & geometry”

Because Boyle was not looking for absolute certainty, common agreement wasn’t required in order to generate knowledge. Hobbs wasn’t into this idea.

Boyle used 3 methods to generate knowledge: material technology (construction & use of air pump), literary technology (write up of the experiments to those not present to witness them) & social technology (how the scientists related to each other & considered each other claims)

Hobbs not only questioned the integrity of Boyle’s experiments, but also the integrity of the pump itself - it didn’t form a perfect vacuum, for example. Michael drew a comparison to environmental problems we experience in the workplace.

The keynote moved on to draw comparisons between the uncertainty in Boyle’s experiments to the uncertainty in the 7 principles of context-driven school of testing:

I hadn’t previously observed or considered the amount of uncertainty in the principles - I saw it more of a safety language because otherwise they would have been less principles & more best practice!

Michael then posed the question whether a Testers role is to reduce uncertainty or is it actually to question certainty to ensure it is warranted - are we in the business of “shoring up illusions” or are we in the business of “unwarranted confidence demolition”. This question tied back in nicely to the Boyle / Hobbs debates - Context-driven school mirroring Boyle’s ideas & other schools (which incorporate the creation of test artefacts) as mirroring Hobbs.

(Michael did touch upon the different schools & how because the context-driven community created the idea of testing “schools” there may be some negative conotations with schools other than the context-driven one. Have a watch of this video for more information)

The keynote moved onto how testing is evloving away from just the computer science into the social sciences - as there are so many variables to the software we are testing, our testing can only provide what the social sciences deliver: “partial answers that might be useful”.

Michael puts forward his theory that testing is more like anthropology - its about people as well as the computer science stuff. He also referenced Cem Kaners likening of a computer program to building instructions of a house for people to live in rather than a list of instructions for a computer to understand.

This makes sense to me, possibly because I’m not as technical as other Testers - My aim is largely to ensure the users of the software are going to be happy with it.

As Testers, we ask questions in order to tell a story about the software & how it relates to the people who aer going to use it (reminds me of a methaphor about people not using a drill because they want to, more the fact they need a hole. Or drivers only using a car to get from a to b, not for the pleasure of driving).

Michael concluded the keynote with 3 stories that are told when we are testing software:

He described this as a rope with three intertwined threads. Each level supports the 1 above it. I saw more it as triangle (like the fire triangle) in that if you take one of the 3 stories/elements away, you don’t have convinving testing.

 

Overall, a great keynote with several interesting links for further research. The whole framing of testing as a social science & using Boyle & Hobbs to back up the argument made the whole keynote really engrossing. A great way to kick off the conference!

Key takeaways for me include the bias for the context-driven school of testing because the schools were created by the context-driven community, we cant just rely on testing, we have to demonstrate it to others by telling a convincing story & I need to read “Leviathan and The Air Pump (Hobbes, Boyle and the Experimental Life)” by Steven Shapin and Simon Schaffer!

 

Video of part 1 of the “It’s Not Context-Driven, You Can’t Do It Here” keynote

Video of part 2 of the “It’s Not Context-Driven, You Can’t Do It Here” keynote