What is Incremental Testing? Method, Types and Example

With the help of this article, I am going to cover one of the important Integration Approaches – Incremental Testing.

By the end of this section, the audience will have a fair knowledge of the following:

• What is Incremental Testing?
• Its Objective
• Methodologies
• Advantages
• Drawbacks

What is Incremental Testing

Incremental Testing, also known as Incremental Integration Testing, is one of the approaches of Integration Testing and incorporates its fundamental concepts.

It is like a test which combines Module and Integration testing strategy.

In this testing, we test each module individually in the unit testing phase, and then modules are integrated incrementally and tested to ensure smooth interface and interaction between modules.

In this approach, every module is combined incrementally, i.e., one by one until all modules or components are added logically to make the required application, instead of integrating the whole system at once and then performing testing on the end product. Integrated modules are tested as a group to ensure successful integration and data flow between modules.

As in integration testing, the primary focus of doing this testing is to check interface, integrated links, and flow of information between modules. This process is repeated until the modules are combined and tested successfully.

Example

Let’s understand this concept with an example:

The system or software application consists of the following Modules:

Incremental Integration testing approach

• Each Module i.e. M1, M2, M3, etc. are tested individually as part of unit testing
• Modules are combined incrementally i.e. one by one and tested for successful interaction
• In Fig2, Module M1 & Module M2 are combined and tested
• In Fig3, Module M3 is added and tested
• In Fig4, Module M4 is added and testing is done to make sure everything works together successfully
• The rest of the Modules are also added incrementally to each step and tested for successful integration

Fig2

Fig3

Fig4

Objective of Incremental Test

• Ensure that different modules work together successfully after integration
• Identify the defects earlier and in each phase. This gives developers an edge in identifying where the problem is. For example, if testing after M1 and M2 is integrated is successful but when M3 is added, the test fails; this will help the developer in segregating the issue
• Issues can be fixed early in the phase without much rework and at less cost

Incremental Integration Testing Methodologies

Before we start with this topic, I would like to give a brief intro about Stubs and drivers since we will be using these terms often.

Stubs and drivers are pseudo code or dummy code used in Integration or component testing when one or more modules are not developed but are required to test some other module.

Stubs are used in a top-down testing approach and are known as “called programs”. Stubs help simulate the interface between lower lever modules which are not available or developed.

Drivers are used in Bottom-up testing approach and are known as “calling programs”. Drivers help simulate the interface between top level modules which are not developed or available.

A question that might occur to some of us is why not wait till all the application modules are developed instead of using stub/driver before starting testing?

The simple answer is that it increases the project execution time since testers will be sitting idle till all the modules are developed. Also, this makes the root analysis of defects difficult. This type of testing approach is known as Big-Bang Integration testing.

Now that we have covered Stubs and drivers, let’s move on to different methodologies of Incremental Testing:

#1) Top Down

As the name suggests, testing takes place from top to bottom, i.e., from the central module to sub module. Modules framing the top layer of the application are tested first.

This approach follows the structural flow of the application under testing. Unavailable or undeveloped modules or components are substituted by stubs.

Let’s understand this with an example:

• Module: Website Login aka L
• Module: Order aka O
  • Module Order Summary aka OS (Not yet developed)
• Module: Payment aka P
  • Module Cash Payment aka CP
  • Module Debit/Credit Payment aka DP (Not yet developed)
  • Module Wallet Payment aka WP (Not yet developed)
• Module: Reporting aka R (Not yet developed)

Top down Incremental Integration Testing Approach

The following test cases will be derived:

Test Case1: Module L and Module O will be integrated and tested
Test Case2: Module L, O and P will be integrated and tested
Test Case3: Module L, O, P and R will be integrated and tested.

And so on other test cases are derived.

This type of testing, where all modules at a layer are first integrated and tested, is known as “breadth-first”. Another category is “depth-first”.

The following test cases will be derived for “depth-first”:

Test Case1: Module L and Module O will be integrated and tested
Test Case2: Module L, O and OS will be integrated and tested
Test Case3: Module L, O, OS,P will be integrated and tested
Test Case4: Module L, O, OS, P, CP will be integrated and tested

And so on other test cases are derived.

Merits of Top-down Methodology

• Early exposure to architecture defects
• It outlines the working of an application as a whole in early stages and helps in early disclosure of design defects
• The main control points were tested early

De-Merits of Top-down Methodology

• Significant modules were tested late in the cycle
• It can be quite challenging to write test conditions
• The stub is not a perfect implementation of the related Module. It just simulates the data flow between the two modules

#2) Bottom-up

In this approach, testing takes place from bottom to top, i.e., modules at bottom layer are integrated and tested first and then sequentially other modules are integrated as we move up. Unavailable or undeveloped modules are replaced by drivers.

Take a look at the below-mentioned example for better understanding:

Module Ranks, Marks, Percentage and Sports Grades are not yet developed so they will be substituted with related Drivers:

Bottom up Incremental Integration Testing Approach

The following test cases will be derived:

Test Case1: Unit testing of module Practical and Theory
Test Case2: Integration and testing of Modules Marks-Practical-theory
Test Case3: Integration and testing of Modules Percentage-Marks-Practical-Theory
Test Case4: Unit testing of Module Sports Grade
Test Case5: Integration and testing of Modules Rank-Sports Grade-Percentage-Marks-Practical-Theory

The Merits of Bottom-up Methodology

• This methodology is very useful for applications where bottom up design models are used
• It’s easier to create test conditions in the bottom up approach
• To start testing at the bottom level of hierarchy means testing of critical modules or functionality at an early stage. This helps in early discovery of errors
• Interface defects are detected at an early stage

De-Merits of Bottom-Up Methodology

• Drivers are more difficult to write than stubs
• Design defects are caught in the later stage
• In this approach, we do not have a working application until the last module is build
• The driver does not have a complete implementation of the related Module. It just simulates the data flow between two modules.

#3) Sandwich Testing

This approach is a hybrid of top-down and bottom-up methodology. Stubs and drivers are used for incomplete or undeveloped modules.

Testing Approach

• The middle layer is identified from which bottom-up and top-down testing are done. This middle layer is also known as the target layer
• Target layer is identified as per Heuristic approach, i.e., select a layer which allows minimal use of Stubs and drivers
• Top-down testing starts with the middle layer and moves downwards towards the lower level modules. The layer below the middle layer is known as Bottom layer
• Bottom-up testing also starts with the middle layer and moves up towards the top layer modules. The layer above the middle layer is known as Top layer
• With the use of stubs and drivers, the user interface and functions of lower level modules are tested respectively
• In the end, only the middle layer is left for the execution of the final test

Example:

The following test cases can be derived from the Sandwich Testing Strategy:

Test Case1: Test A, X, Y, and Z individually – where Test A comes under Top layer test and Test X, Y and Z comes under Bottom layer tests
Test Case2: Test A, G, H and I
Test Case3: Test G, X, and Y
Test Case4: Test Hand Z
Test Case5: Test A, G, H, I, X, Y, and Z

Merits of Sandwich Testing Methodology

• It is very beneficial for a big project which has various sub-projects
• Top-down and bottom-up testing methodology can be run side by side

De-merits of Sandwich Testing Methodology

• Before module unification, subsystems and their interfaces were not tested thoroughly
• Higher cost due to involvement of both top-down and bottom-up testing methodology
• This type of testing is not advised for a system where modules are highly inter-dependent

Conclusion

Incremental Testing comes under the blanket of Integration testing. In this approach of testing, integration testing is done on the individual module as a part of unit testing and in next phase, modules or components of the application are integrated incrementally and testing is performed on combined modules as a group.

Out of three methodologies of Incremental Integration Testing, the choice of which methodology to choose depends on the structure of the application and also on the position of high-risk modules.

All three methodologies for incremental testing come under the Horizontal category because of the following behavioral aspects:

• All three methodologies focus on layer testing
• All of them consider a structural or hierarchical design
• They all integrate layers incrementally

Merits:

With this testing approach, it is easier to identify defects early, and it also helps the developer determine the cause of the issue. Since it uses the basics of structured testing, this testing approach is very effective and accurate.

Demerits:

This type of testing approach is time-consuming because of the use of stubs and drivers. It is also repetitive.

About the author: This helpful tutorial is written by Neha B. She is an ISTQB certified Lead Quality Analyst with 8+ years of experience.

Let us know if you have any questions/suggestions.