One of the advantages of being part of the ISST is the fantastic opportunity to dig into topics & ideas with a community of critical thinkers. There is nothing like a good Skype IM chat to sharpen your own thoughts & beliefs!
The other day, the topic of “single source of truth” arose. Who had heard of it? Is it actually a thing?
For me, it is. Or at least it was. My beliefs have been challenged & now I’m rethinking what I have previously thought to be true.
This post was written immediately after that chat and is my way of trying to get straight in my head the conversation I had around “single source of truth” and what that phrase actually means.
These are my views from before the conversation
I have come across the phrase “single source of truth” in relation to a suite of automated checks serving as “living documentation” or “executable specs” from the Spec By Example & BDD domains.
The idea, as I understand it, is that the automated checks assert that the code is behaving as the development team & business stakeholders agreed it should.
Let me take a step back to how this behaviour was agreed in the first place.
A typical process by which a feature would be developed would include all stakeholders necessary for the development of that feature getting together before any code is written to discuss the problem, need or opportunity so that they understand why they are developing the feature.
This conversation includes questioning & challenging of the explicit requirements as well calling out & understanding the implicit requirements.
The output from this conversation is a proposed solution and ideas for testing that feature. There may also by artefacts called “acceptance tests” created which are effectively the requirements written in such a way that a human or a tool can run them to assert the behaviour of a system.
The Programmers practicing TDD test the code as they write it & create artefacts labelled “unit tests” which they walkthrough with the Testers. This walkthrough helps both the Programmers & Testers check that the tests are correct. When they re-run the unit tests then they are checking that they haven’t inadvertently changed the logic of the code.
These names of these unit tests are written in a way that they are easily understood by a human.
The Testers explore the feature to discover ways in which it deviates from the desired behaviour previously agreed by the stakeholders. A result of this testing is that undesired behaviour (potential bugs) is highlighted and discussed with both the Programmers & Business stakeholders. From this conversation, more unit tests & acceptance tests can be created.
The end result is that all stakeholders are aware of the unit test & acceptance test artefacts which should assert if the behaviour of the system deviates from the desired behaviour in the future.
These artefacts serve as a single source of truth in the form of “living documentation” or “executable specs” for a feature because the stakeholders who were involved in the development of that feature have bought into those artefacts.
The automated execution of the unit & acceptance tests is not part of testing. That is checking.
And my views immediately after the conversation
It appears I have been blindsided by the “single source of truth” phrase. As I mentioned before, I’m still trying to get my head around this shift away from “single source of truth” & what it means to my ideas & beliefs.
“Single source of truth” (& “living documentation” or “executable specs”) as I have described above is just an application of automation.
It appears that because the artefacts were created as part of a team effort, I thought that they were infallible. What a trap to fall into.
Is this an example of confirmation bias through groupthink with a dash of halo effect for good measure?
I certainly wasn’t skeptical enough, that much is true. I saw how others I respected were using the phrase & didn’t question it.
For me now, the phrase “single source of truth” & automation are interchangeable;
The automated check was correct at that time, for that situation but it does & can not cover every situation. This is true for the “single source of truth” phrase
Let’s break down the phrase.
Sometimes I have referred to the living documentation as “THE single source of truth”. Wow. That was stupid. I thought that was a valid conclusion based on the way the artefacts were created, but thinking as living documentation as applied automation & all the pitfalls that go with automation makes the use of the word “the” stand out as pretty dumb. It’s an open door to confirmation bias.
“A single source of truth” would be better, but then the use of “single” suggests only one, which is pretty much the same as saying “The source of truth”.
So maybe “A source of truth” might be more suitable? But then what is the actual source?
Are the artefacts the source of truth? Yes, but only for the instance that they represent the way the software is used which likely to pretty slim. So what value do they offer if they be considered the truth for the narrowest of situations?
Is the product the source of truth? You’d have to hope so. But how do we discover the truth from the product? For me now, the truth emerges through the development & testing of that product & is shared among the stakeholders both explicitly & tacitly. It would be very difficult to express all the truths of the product explicitly. Think about it; how has “handing over” a piece of work worked for you in the past - as both the giver & receiver? What about when someone leaves the team or organisation taking their domain knowledge with them?
What about truth?
A definition of “truth” from dictionary.com
- conformity with fact or reality; verity:
The product itself should be reality, so are the artefacts actually checking the source of truth rather than being the source of truth themselves?
Implications for Testers
Aside from the general lesson of challenging ideas & models before taking them to heart, think about the phrase “single source of truth” actually means.
The likelihood is that that source of truth is unlikely to be:
- the only truth
- infallible
So question that truth. Where has it come from? How has it been created?
In order to think critically, we need to challenge our ideas & beliefs. Yes, it can be painful, but hopefully it in the long run the challenge will give you a deeper respect & understanding for your ideas & beliefs.