What are the seven
fundamental principles of testing?
These are the principles that have been
collated and established by the ISTQB as testing and software development has
evolved over the years, and are recognised as the absolute core of testing.
This is part of the reason why we are proud to
say all our testers at Box UK are ISTQB qualified!
If you’re involved
with any aspect of software testing it’s worthwhile to
fully review and comprehend these standards, checking that you are following
them within your organisation and teams, as they will help you achieve high
standards of quality and give your clients confidence that their software is
production-ready.
The seven principles of
testing
1. Testing shows the
presence of defects, not their absence
We test software to discover issues, so that
they can be fixed before they are deployed to live environments – this enables
us to have confidence that our systems are working. However, this testing
process does not confirm that any software is completely correct and completely
devoid of issues. Testing helps greatly reduce the number of undiscovered
defects hiding in software, but finding and resolving these issues is not
itself proof that the software or system is 100% issue-free. This concept
should always be accepted by teams, and effort should be made to manage client
expectations.
It is important to remember however that while
testing shows the presence of bugs and not their absence, thorough testing will
give everyone confidence that the software will not fail. Having a
comprehensive test strategy that includes thorough test plans, reports and
statistics along with testing release plans can all help with this; reassuring
clients as to testing progress and providing confidence that the right areas
are being tested.
Additionally, ongoing monitoring and testing
after systems have gone into production is vital. Thinking forward to potential
issues that could arise is another good way to help mitigate against any future
problems, for example considering load testing if a site is launching a new
marketing campaign, so you can be confident the software will withstand any
anticipated larger volumes of traffic.
2. Exhaustive testing
is impossible
As much as we would like to believe or wish it
true(!), it is absolutely impossible to test EVERYTHING – all combinations of
inputs and preconditions – and you could also argue that attempting to do so is
not an efficient use of time and budget. However, one of the skills of testing
is assessing risks and planning your tests around these – you can then cover
vast areas, while making sure you are testing the most important functions.
With careful planning and assessment, your test coverage can remain excellent
and enable that necessary confidence in your software, without requiring that
you test every single line of code.
3. Early testing saves
time and money
Testing early is fundamentally important in
the software lifecycle. This could even mean testing requirements before coding
has started, for example – amending issues at this stage is a lot easier and
cheaper than doing so right at the end of the product’s lifecycle, by which
time whole areas of functionality might need to be re-written, leading to overruns
and missed deadlines.
Involving testing early is also a fundamental
Agile principle, which sees testing as an activity throughout, rather than a
phase (which in a traditional waterfall approach would be at the end) because
it enables quick and timely continuous feedback loops. When a team encounters
hurdles or impediments, early feedback is one of the best ways to overcome
these, and testers are essential for this. Consider the tester as the
‘information provider’ – a valuable role to play.
Essentially, testing early can even help you
prevent defects in the first place!
4. Defects cluster
together
This is the idea that certain components or
modules of software usually contain the most number of issues, or are
responsible for most operational failures. Testing therefore, should be focused
on these areas (proportionally to the expected – and later observed – defect
density of these areas). The Pareto principle of 80:20 can be applied – 80
percent of defects are due to 20 percent of code!
This is particularly the case with large and
complex systems, but defect density can vary for a range of reasons. Issues are
not evenly distributed throughout the whole system, and the more complicated a
component, or the more third-party dependencies there are, the more likely it
is that there will be defects. Inheriting legacy code, and developing new
features in certain components that are undergoing frequent changes and are
therefore more volatile, can also cause defect clustering.
Knowing this could prove to be very valuable
for your testing; if we find one defect in a particular module/area there is a
strong chance of discovering many more there. Identifying the more complex
components, or areas that have more dependencies or are changing the most, for
example, can help you concentrate your testing on these crucial risk areas.
5. Beware of the
pesticide paradox
This is based on the theory that when you use
pesticide repeatedly on crops, insects will eventually build up an immunity,
rendering it ineffective. Similarly with testing, if the same tests are run
continuously then – while they might confirm the software is working –
eventually they will fail to find new issues. It is important to keep reviewing
your tests and modifying or adding to your scenarios to help prevent the
pesticide paradox from occurring – maybe using varying methods of testing
techniques, methods and approaches in parallel.
6. Testing is context
dependent
Testing is ALL about the context. The methods
and types of testing carried out can completely depend on the context of the
software or systems – for example, an e-commerce website can require different
types of testing and approaches to an API application, or a database reporting
application. What you are testing will always affect your approach.
7. Absence-of-errors
is a fallacy
If your software or system is unusable (or
does not fulfill users’ wishes) then it does not matter how many defects are
found and fixed – it is still unusable. So in this sense, it is irrelevant how
issue- or error-free your system is; if the usability is so poor users are
unable to navigate, or/and it does not match business requirements then it has
failed, despite having few bugs.
It is important,
therefore, to run tests that are relevant to the system’s requirements. You should
also be testing your software with users – this can be done against early
prototypes (at the usability testing phase), to gather
feedback that can be used to ensure and improve usability. Remember, just
because there might be a low number of issues, it does not mean your software
is shippable – meeting client expectations and requirements are just as
important as ensuring quality.
Thanks for sharing an important article.Keep Sharing !!!You can read more about it at What are seven Software Testing principles ?
ReplyDelete