A recent Forrester Wave study validates the test automation pyramid, i.e. API testing should be increased as a GUI test expense:
We define API testing as a method where business logic is directly tested without execution of the GUI. However, with the exception of some special cases, the traditional solution requires a considerable amount of coding. It’s an extension of unit testing, where the business units are integrated. In this case, reliable testing knowledge is also required. In addition, this is not an agile method, so many teams continue to test through the GUI.
I think that the cause of the current small ratio of API testing is the need of reliable testing knowledge besides the considerable coding work. In addition, the maintenance cost is also significant. To spread API testing a different method is necessary, which eliminates the current shortcoming of API testing.
Keyword-driven testing is a software testing methodology that separates test design from test development. Among other benefits, this allow additional professional groups, such as business analysts, to work on the test automation process. By applying KDT, test cases can be created without implementation issues and independently from the writing of test code. In this way, functional tests can be developed prior to any implementation work.
Our new Keyword-Driven API Testing or KDAT is a test-first agile method where the Graphical User Interface is replaced by a Test Tool Interface (TTI). A TTI is a piece of code that can send input to and receive output from the business logic. A GUI is a connection between the human and the business logic, while the TTI is a connection between the test tools and the business logic. During GUI test automation the automation tool sends and receives data to and from the GUI. During KDAT all data are transferred along the Test Tool Interface:
The method signatures of the Test Tool Interface can be marked out as keywords. The test tool is able to recognize and call these methods (keywords). In this way, the necessary input for the business logic can be set by the test tool as keyword parameters, and no GUI is executed during the testing process.
TTI can be created at any phase of the development or the maintenance, however, the best solution is to plan this interface as part of the software in the design phase. In this way quite a few additional coding is necessary.
The process of KDAT is the following. The agile team such as the three amigos makes the test design. The result of the test design is a test table with keywords and parameters, read our other blog. Based on the keywords, the TTI code part is planned together with the “traditional” code. The test table can be automatically converted to a test file which is understandable for the testing tool. When the code is ready, tests are executed and the test results are evaluated.
In some cases it is well worth applying KDAT on existing system. Crucial or often changing parts of the code can be the subject of the late automation. In this case making the TTI is more difficult, but manageable. Our recent case study where the application under test was Monopoly revealed that the total test automation cost of KDAT and GUI testing was approximately the same. In this case study keywords and test cases created manually, though our 4Test method enables generating executable test for KDAT. Since the system alredy has been working, after the creation of the test table, the tests can be executed.
It’s reasonable to consider advantages with respect to GUI testing and existing API testing separately. Considering traditional API testing, the advantage of KDAT is that testing and coding work is separated and everybody is able to focus on what they do best. With the keyword-driven method, KDAT can be used in agile projects and the tests will be understandable for everybody. Another significant advantage is that much less code to be written. In addition this code is more stable since the tests are changing more frequently than the keywords based on the test are developed.
On the other hand, API tests are much faster than GUI tests, therefore the former can be executed several times a day. Obviously, the test are more stable, since they are immune for GUI changes. In addition KDAT is cheaper than the GUI testing in the maintenance phase because of the stability, and for green field projects it’s cheaper for both the development and the maintenance phases.