Dissecting The Testing Quadrants – Wrap Up

Posted By Duncs ~ 25th April 2014

And finally we come to the end of this exploration into the Testing Matrix, so what have I learned?

From my initial chat with James, I knew I had a fair challenge ahead of me.

We had completely deconstructed my understanding of the model & I was going to have to build that understanding back up again.

From looking at other people’s thoughts on the model, I could see they had fallen into similar, if not the same, traps as me.

One example included a comment from Lisa on her post “using the Agile Quadrants” from 2011 suggested that some people believed the flow through the model was linear – they would start in Q1 & work their way through to Q4.

This is primarily the reason why I chose to label the quadrants Top Left (TL), Bottom Left (BL), Top Right (TR) & Bottom Right (BR).

This is also the language I use when using the quadrants to discuss the testing – it saves people having to remember which quad is which.

Another change I’ve tried to represent is the movement through the matrix. For example, movement of artifacts, people & activities. This is harder than I imagined so I now have even more respect for those who have come before me in trying to shape the matrix.

The one quadrant which caused me the biggest headache was Bottom Right. The quadrant has always felt like a bucket for the leftover artifacts & activities which didn’t seem to fit anywhere else.

My approach was de-label what the artifacts were actually doing, such as performance or load testing, & think of them as another way of questioning a different aspect of the product.

This made it easier for me to determine if they could be run before or after writing the production code. From here, it was a case if determining if the Business would be interested in the results.

From the different incarnations of the matrix I observed (& related material) I really liked how Gojko pushed the scope of the Top Right & Bottom Right quadrants to include feedback loops of the code in the Production environment.

The subsequent comments made for interesting reading.

What the comments highlighted for me (& what I later found out for myself) is how hard it is to create a one-size-fits-all model. It’s almost as if the different variations of the model sit on a timeline, Brian’s matrix at one end, Gojko’s at the other & development teams need to pick one most suited to them (I say this tongue in cheek of course).

I was asked some great questions on the different representations post related to this idea of the timeline. They were focused on putting the different versions of the matrices into the context of what was happening in the software development world at the time of the matrix being updated.

Unfortunately I haven’t had time to go into that much detail in this tome, but I feel Gojko’s post clearly showed how software has development has grown since the matrix was conceived by Brian Marick back in 2003; 2 ends of a spectrum, as it were.

Most of the matrices agreed on the top & bottom rectangles, but there was a wide range in what the left & right columns were for. I found these differences most interesting:

Left Column	Right Column
Support Programming (preparing & reassuring)	Critique Product (uncovering prior mistakes and omissions)
Supporting the Team	Critique Product
Defect Prevention	Defect Detection
Checking	Testing
Confirm	Investigate
Check for expected	Analyse unexpected / undefined / unknown
Test to Specification	Test to Failure
Written before code	Written after code

With Gojko’s matrix I mind, I’ve tried to use labels & language which allows the use of the matrix to stretch beyond the development environment.

One gaping hole I really want to plug is requirements gathering & refinement – such as the 3 Amigos process.

If we talk of testing as questioning a product, & it is possible to question a product in different states, then I feel requirements refinement is questioning the product in the Business stakeholders head.

Although it doesn’t have the label “Requirements Test” or “Stakeholder Test”, we are still questioning a version of the product & how it should behave.

You know what – I’m going to add “Stakeholder Test” to TL & refine that idea.

One final note, I believe Lisa & Janet have a chapter in their new book looking back at the testing quadrants & where they are now - it’ll be interesting to see their take on the evolution of the quadrants.

Previous posts:

Post 1: Intro

Post 2: Rectangles

Post 3: Columns

Post 4: Different Representations of the Model

Post 5: Example usage of the matrix