What is Testing?

This is a question, that on the face of it you think would be very easy for me to answer but it is something I have often struggled to acutely put into words. When people asked me what I do, it was often met with a face deep in thought, struggling to find the words to describe my job and the value I bring to a team.

In order for me to express what I understand testing to be, I need to explore what it is I do. Now, it is quite apt that I have mentioned the word explore because exploration is at the very core of what software testing is all about. I will dig into this statement a little later on.

To consider an activity as testing there needs to be a purpose. Let’s look at the following scenario: –

I am testing a product and time has come for me to talk the client through the testing that is being carried out. Given my knowledge of the product I select tests to run that will not highlight any flaws and known issues be that performance, functional or any other issue that could paint the product in a bad light. [1]

Is this testing? I am at no point planning to use the information that comes out of this demonstration as I am already aware of the results that these selected “tests” will exhibit. This activity has no purpose other than show the software appearing to work under a specific set of criteria. (I should emphasise that I consider this to be bad practice and transparency is the key to building a successful product)

Testing centres on learning. This can be about the product, processes or different ideas. As we learn about the product we are also gathering information which we can use to form further test ideas about how we want to use the software. If I go back the scenario mentioned earlier there is no learning going on here (at least in relation to the software). Always focus on what information you are trying to obtain to stay on track when it comes to testing.

It is also important to consider the questions we ask of software to ensure we get the appropriate information in return, whether it be good or bad. When scrolling through Twitter recently, I saw a tweet that analogises this quite well: –

We can ask any question of the software we wish but will it give us the right answers for understanding the level of quality built into the product? That should always be our end goal.

People often assume that testing is something that occurs after a product has been built. For a long time I was of this opinion, after all how can you test something that does not exist yet? As my career has moved forward in the last few years I have come to realise that what we are referring to as “testing” at the end of this process is actually “checking”. Consider the following questions: –

  1. Does the button turn blue when pressed?
  2. Does the page reload when the link is pressed?

These are checking activities. Due to the explicit nature of their assertions we can script these activities. We have a clear input and an output. Checking is something that is also prime for being automated but that is a whole discussion for another day.

Testing can (and should) happen throughout the entire development process. All of the following tasks embody what testing is concerned with: –

  • Three Amigos sessions encompass testing when the team discuss how the software should behave
  • Pair programming with a developer allows for testing in parallel
  • Code review is a form of static analysis which in turn is testing
  • Monitoring product metrics is a form of testing

That is not to say that interacting with the product post development is no longer a testing activity. It 100 percent is. The above allow us to be better informed in our post development work. These activities are also best when they are exploratory.

For a long while I saw exploratory testing as an activity that was only carried when there was a lack of/ vague requirements. In the world of Agile, we should be embracing change and pivot as and when required. We should use our pre-development testing activities to drive what we do once a product has been formed.

If we stick to the notion that test scripts are all compiled and finalised at the start of the project, it creates an incredible amount of overhead when it comes to evolving requirements.

Test scripts are also innately poor in their description. For the most part (at least in my own experience of running and writing test scripts), there are always gaps which assumes knowledge of a product or process. As testers we fill in those gaps with our knowledge and plough forwards which means we are already exploring but within the bounds of a script. Wouldn’t it be easier to strip away all the unnecessary bloat and just explore?

That is not to say, that exploration is unstructured, far from it. With the information that has been collected from earlier in the process we can define our testing approach while also using the wider scope to give us greater flexibility and freedom in our testing.

In closing, my description here is quite broad and this was intentional because despite different products requiring different approaches, the overall goal should remain the same. How can we probe the software sufficiently to discover the level of quality built into the product?

I also think it’s important to regularly assess and review what testing means to us. This will ensure it remains an integral part of the Software Development Life-cycle.

References

[1] Perfect Software and other illusions about testing – Gerald Weinberg

Advertisements

One thought on “What is Testing?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

w

Connecting to %s