Why do you need to test software?

The role of the software tester and the cost of bugs

What is testing and why do we need it?
If you were making soup, to ensure that it was as good as possible you would taste it as you went along. In this way you could modify it accordingly before it was ready to be served. The same basic principle is true for software. Essentially testing is the process of ensuring software behaves as well as expected. It is vital because once we have determined what it is that we are building, we also need to have an idea about whether we are building it correctly. Simply put, software testing asks:

  • does the thing do what it's supposed to do?
  • does it do it the way it's supposed to?

Testing not only allows us to check we are building the product that was expected, but also to think about different ways the product could be used and the different ways it could possibly fail. This sort of information is important because it allows the business stakeholders – those who define the requirements of the software – to make informed decisions about the project.

Testing can also flag up a whole variety of other potential problems, such as where users might find the layout of the product confusing, or where there are risks that could affect the product as it's being built.

What are bugs?
A bug is the name given to an undesired behaviour in the software that threatens the value of the product. This could be deviation from the requirements as defined by the business stakeholders, or it could be a frustrating workflow in a user interface (UI) which deviates from what a user expects. The testing process aims to uncover these bugs.

Amy Phillips, head of test at Songkick, explains how bugs can creep in

When does testing normally occur?
Preferably, testing occurs throughout the entire development lifecycle of a piece of software.

When discussing the requirements of the product, the developers and testers within the team will ask questions of the business stakeholders to determine if the product they have in mind will actually solve the problem they are trying to overcome.

Likewise, during development itself all members of the team will perform some form of testing to ensure their part of the puzzle is behaving as expected.

The benefit of carrying out the majority of testing tasks during development is that issues discovered during this phase are easier and quicker to fix for the development team. One of the main reasons for this is that it is far less taxing to alter code when you are still actively working on it, rather than having to go back to it days or even weeks after you thought you were done. It also allows the system to change direction if an issue informs the product stakeholders that what is being built is no longer correct.

Once in the production environment, further validation can be made to see if the product did actually solve the problem or not. This involves monitoring the product’s and customers’ behaviours to see if the theories we had were correct or if we need to make further amendments.

The testing discipline is currently undergoing change. Where previously the majority of tests would be executed by testers in a dedicated testing phase, now there is a move for all development team members to perform some form of testing throughout the lifecycle.

Any time you ask a question of the product – from talking to the business stakeholder before any code is written, to interrogating logs once the product is live – you are testing.

The tester as the conscience of a team – Amy Phillips describes the role of the tester

What are the costs associated with testing?
Testing, like any process in the development cycle, costs both time and money. The trade-off comes when you are working out the value of the information gathered through testing against how much it cost to get that information.

The adage of “you get what you pay for” holds true for testing as well; you can buy cheap testing, but it’s likely that the information you gather this way will provide little value.

Likewise, if you don’t do any testing, how much will it cost the organisation to manage and fix all of the bugs that the customers find? These bugs may ultimately result in loss of sales, a negative impact on the reputation of the organisation, and even possibly damage to the customers.

“The challenge is always trying to find the right balance between risk and speed” – Amy Phillips, Songkick

Repeated testing can be made more efficient (and possibly cheaper) by using tools which automatically check whether the information we previously gathered continues to be valid and true. These checks save time and money when running, but have increased costs related to their creation and maintenance. There is also the high potential risk that these checks will fail to gather new information.

Why aren’t all developers also testers?
Testing is a craft, as is software development. However, developers and testers have different functions, and so the way they work is different.

The developer aims to build something. The tester aims to find where that thing may be unsuitable. It can very hard to switch between the two mind-sets.

Developers can test but tend not to have studied testing in the same way as a tester has. Additionally, a developer will know how a system they’re working on functions at a code level but may be almost too familiar with it. Testers on the other hand are likely to not have a full understanding of how the code interprets the actions and therefore are more likely to try things that the developer may never have even considered.

How does the tester fit into a development team and is this role changing?
Testers act as a voice of reason on the development team. They can be thought of as the detectives of the team: questioning the idea in the stakeholder’s head, exploring the system as its being built and seeing how customers are using the software once it is in the production environment.

“It's not just the job of anyone with the word tester in their name” – Jitesh Gosai, BBC Future Media

The tester’s role is changing in that it’s no longer just the testers who do the testing. Increasingly testers are sharing their ideas and skills so that others in the team can help gather information as well.

David Buckhurst and Jitesh Gosai, BBC Future Media, discuss how the role of the tester is changing

How is testing changing and what does the future hold?
The rise of automation has had, and continues to have, a big impact on testing. Will this mean there are less testers in the future, as some have argued? This is covered in our next article.

This article was put together by the BBC Academy with contributions from the BBC's Duncan Nisbet, senior test engineer, David Buckhurst, technical architect, and Jitesh Gosai, senior developer-in-test, as well as Songkick's head of testing, Amy Phillips.