Model Based Testing for Agile – 4Test

Model Based Testing (MBT) is a test design method which starts from a model of the planned code on the basis of which test cases are then generated from the model. The executable tests can then be generated from the original test cases. Experts strongly believe that MBT will be widely used within 5-10 years.

MBT can be graphical, text based or mixed. Most existing tools that implement MBT are graphical and based on UML or BPMN. Text based methods are rare and are based on formal modeling languages such as OCR, a programming language tailored to this task, or special test languages such as Test Description Language. All of these tools require programmer knowledge. None of these methods supports agile testing, where the tests are created by a team.

MBT can use development or test models. If a development model is applied, then this and the one for code development should be different, otherwise the same defects occur in both the code and the test, i.e. the design bugs remain undetected. Since development models are designed for coding they should be extended to involve test design methods, such as equivalence partitioning or boundary value testing. This is not straightforward, however, and I therefore prefer test-based models. Unfortunately, graphical models fall into the category of development, and as far as I know, the wide-spread tools are graphical or mixed.

4Test is a new method where a test design method - Constraint Driven Testing (CDT), is united with a test automation method - Keyword Driven API Testing (KDAT).

We have extended our manual CDT so that the method enables test automation. Let's start with a short summary of the method. The starting point is a user story/feature specification. The agile team analyses the feature and tries to collect the categories finding the preconditions, inputs, and sometimes the outputs. For example, based on a login feature we can select login name and password as categories.

The categories are further divided into choices; for example, a login name can be existent or non-existent, a password can be valid or invalid. The numeric-content categories can be divided into choices based on the boundary value analysis. When it's not reasonable to select a choice more than once, then we mark it as (S) - single.

The key elements of the method are the constraints which are relations among the choices. For example, a constraint expresses that if I have a book loyalty card, then I have a price reduction of 10%. The agile team is able to create these conditions even if developers are not available.

We can use the well-known Gherkin syntax for the description of the constraints. The original syntax is extended and an entire constraint can be written in one line. Here is an example:

  • WHEN card owner IS yes AND new books' price IS 49.99 THEN price reduction for new books IS 10%
Keywords are bold, categories are underlined and choices are italics. We can use further keywords such as: ARE, OR, GIVEN and NOT. Therefore, constraints are understandable for both humans and machines.

4Test is a model based testing method, where the model contains three elements: {category, choice, constraint}. When all these elements are available, the text-based model can easily be created by any text editor. A simple example demonstrates this.

Step 2: The categories are further divided into choices.

  • card owner (I): yes; no
  • price reduction for new books (O): no reduction; 10%; 15%
  • new books price in EUR (I): 49;99; 50
  • second hand book price in EUR (I): 0; 100 (S)
  • total amount: 142.5

Step 1: we gain categories which are the preconditions, input and sometimes the output values:

  • card owner (I)
  • new books price (I)
  • second-hand books price (I)
  • price reduction for new books (O)
  • total amount (O)

I: input, O: output

Step 3: There are constraints among the choices.

  1. WHEN card owner IS yes AND new books price in EUR IS 49.99 THEN price reduction for new books IS 10% AND total amount IS 44.99
  2. WHEN new book price in EUR IS 50 AND card owner IS no THEN price reduction for new books IS 10% AND total amount IS 45
  3. WHEN new book price in EUR IS 50 AND card owner IS yes THEN price reduction for new books IS 15% AND total amount IS 42.4
  4. WHEN new book price in EUR IS 50 AND card owner IS yes AND second hand book price in EUR IS 100 THEN total amount IS 142.5

Constraints determine the tests, but a constraint doesn't contain all the choices, therefore it is not a test case. In most of the cases, constraints describe the dependences among two or more choices, leaving some other choices free. The test cases are generated based on all these dependences.

Categories with choices and Gherkin-based constraints can be developed by an agile team. The main difference between 4Test and ATDD or Specification by Example (SBE) is that instead of developing ad hoc test cases, by applying 4Test a test specification is generated. This can also be considered as live documentation, but as the maintenance of it is much simpler, the generated test cases are optimized since no missing or superfluous test cases are generated (regarding specification).

After reviewing the constraints, we realize that constraints 3 and 4 can be merged:

  1. WHEN card owner IS yes AND new books price in EUR IS 49.99 THEN price reduction for new books IS 10% total amount IS 44.99
  2. WHEN new book price in EUR IS 50 AND card owner IS no THEN price reduction for new books IS 10% total amount IS 45
  3. WHEN new book price in EUR IS 50 AND card owner IS yes AND Second hand book price in EUR IS 100 THEN total amount IS 142.5 AND price reduction for new books IS 15%
Based on the constraints theses tests are generated:

​ We can see from the table that only the choice of price reduction's "no reduction" hasn't been covered by the test cases. Therefore, the real test table to generate is the following: 

Finally, we have to complete the missing output values and can give names for the tests:

ComputePrice

The attentive reader will have realized that here we test two different things together: a) 15% reduction b) no reduction for second hand book. We can split the test by leaving constraint 3 and 4 separated. However, only a very unrealistic bug would cause a tester to make a wrong conclusion about the cause of the bug. This would occur when the implemented reduction is 10% instead of 15% and an incorrect 2.5% price reduction for second hand book would have been implemented.

Based on the final test table, the test cases can be arranged to be very understandable for everybody. Example:

T1: card reduction

CDT + KDAT = 4Test

The next step is to connect KDAT with CDT. KDAT is a test-first agile method where the Graphical User Interface is replaced by a Test Tool Interface (TTI). A TTI is a piece of code that can send input to and receive output from the business logic. A GUI is a connection between the human and the business logic, while the TTI is a connection between the test tools and the business logic. During GUI test automation the automation tool sends and receives data to and from the GUI. During KDAT all data are transferred along the Test Tool Interface.

KDAT can easily be integrated with CDT. This is because in the case of KDAT the tests do not contain any implementation-related elements, such as click here or there. CDT has also only….As a result, the generated test table of CDT can easily be converted to be readable for any test execution tool. Fortunately, modern test automation tools involve keyword driven testing.

A test case in our generated test table contains input and output values. The first row of the test table contains categories, which are the parameters. Finally, the feature is a keyword on the test automation tool side and method on the application side. Therefore, the generated test for T1 is something like this:

The keyword itself is also generated:

ComputePrice

We have generated the tests prior to implementing the application. The developer implements the ComputePrice method with these four parameters and a return value of the price.

A more complex example showing when keywords are structured will be published in a subsequent article.

Summary

CDT and KDAT are ideal methods to be united. The connection between them is straightforward. The features are mapped to keywords which represent method/function calls.The input categories are mapped into parameters of the keywords. The output parameters are the return values of the methods/functions. The keyword hierarchy shows the program structure.

API testing is much more stable and faster than GUI testing. Executable tests can easily be generated by 4Test. We therefore hope that The Forrester Wave prediction about the spread of API testing will make 4Test is a must-have.

Share this article:

">
0

Comments

 
No comments yet
Already Registered? Login Here
Guest
Friday, 23 June 2017