Risk can be defined as the possibility of a negative or undesirable outcome or event. Risk is a factor that could result in future negative consequences, expressed as the multiplication of impact and likelihood:
Level of risk = impact if it did happen * probability of the risk occurring
Testing effort increases exponentially if we strive to execute more and more tests by applying stronger testing criteria. The cause is the combinatorial explosion.
The impact of failure is also exponential. This means that if testing is weak and most of the bugs are found in production, then the cost fixing them increases exponentially. Based on this, an optimum can be reached:
To avoid combinatorial explosions, we can use different testing methods in parallel, rather than simply blindly trying to increase the percentage of potential tests we run. For example, we can apply exploratory testing, equivalence partitioning with boundary value testing, defect prevention (e.g. through early test design such as ATDD and BDD), and static analysis together. In this case, the testing cost does not increase exponentially with respect to the detected failures. This is because different methods will detect different sets of bugs, though the sets do partially overlap. Now the cost curves look like this:
The figures show us that risk-based testing guides not only the determination of test priority and allocation of test effort, but also the selection of test design techniques. For riskier user stories, we must not only use stronger coverage criteria, we must also apply a wider range of test design techniques. For less risky user stories, we can use weaker coverage criteria and fewer test design techniques.
Our aim is to get as close as possible to this optimum model through a careful selection of test techniques and their associated coverage criteria. Applying risk analysis, our experience, and metrics on previously observed failures, we can select the best test techniques and coverage criteria for a user story.
You can use the 4Test model-based testing tool, which has been invented to make test design more efficient. After registration, you will receive the link to the 4Test automated test design tool and user guide.
4Test-Plus is a disruptive test design automation and scriptless test automation tool based on an extended Gherkin language and test first method. Requirement engineering becomes transparent for all stake holders. 4Test-Plus enables real-time test automation and execution.