What is Early Testing? Benefits, Test Design, and Example

This article is a detailed guide to Early Testing. Let’s get started.

Software testing should start early in the Software Development Life Cycle. This helps to capture and eliminate defects in the early stages of SDLC i.e. requirement gathering and design phases.

An early start to testing helps to reduce the number of defects and ultimately the rework cost in the end.

The various aspects of Early Testing which would help the QA Managers and Leads while developing or devising the Testing Strategy document in SDLC are explained here.

Adoption of Early Test will immensely result in the successful delivery of a Quality Product.

Early Testing

By the end of this tutorial, the Readers, QA Managers, Leads and Testers will have a fair knowledge of the below concepts:

• Why Early Testing in SDLC (Project or a Software Release)?
• Scoping of Early Testing efforts
  • How to Test Early?
  • Start and Exit
• Pros and Cons

Let us now explore the nuances in detail!!

Principles of Testing

Figure 1 – Simplified view of Principles of Testing

For a given Software, System or Product release in SDLC, there are various well-defined methodologies and strategies for most of the following Principles of Testing.

• What is Testing?
• Why Testing?
• What to Test?
• How to Test?

However, some of the most lingering questions that many Readers, Testers, Leads, and QA Managers would ask or would like to get more clarity on include (grey area in Figure 1)

• When to start testing in a software release or when should testing start in a project?
• When to start testing and when to stop testing?
• Why should testing start early in SDLC?
• What is an early test in software development?

For easy understanding of the audience, I have clubbed all the questions under one umbrella called Early Testing.

Why Testing Early in SDLC

Let us discuss some of the events and activities that are part of the testing.

Usually, the Program Management Team assigns a Program Manager (PM) to a given Software Release or a Project.

The PM in collaboration with all the stakeholders including Marketing, Development, QA and Support teams comes up with a Release Schedule

In this tutorial, I have chosen Quarterly Release Schedule using the Waterfall model to explain the Early Testing Concepts in detail.

Software Release Testing Schedule

Most organizations still follow traditional Time Based Release (TBR) models where Software and Product releases are planned for quarterly, half-yearly or yearly delivery.

Predominantly, the Waterfall model is used to execute such Software releases. In some cases, for a shorter release cycle, the Agile/Scrum model is adopted.

Figure 2 – Typical Quarterly Release Testing Schedule (Not overall Project or Release Schedule)

Impact of Critical or High Severity Defects

Figure 3 – Typical Impact of Critical Defects

Firstly, during the course of Testing, it is expected that

• Critical or high severity defects to be identified and logged by Testers.
• Developers will need to fix those defects.
• Subsequently, testers will need to verify the fixes.

Secondly, it is widely acknowledged by many Product and Software Engineering organizations that fixing and verifying high severity or critical bugs at a very large number is

• Time-consuming
• Resource hogging (human + machine)
• Prone to collateral, fixing critical bugs mostly touches a large part of the code including the intersection areas.

Lastly, if a large number of critical bugs are found during the end of a given release, then one or more of the following negative developments take place.

• High probability of the Testing cycle being extended.
• High probability of the release deadline being missed.
• A particular feature that has a large number of defects may altogether need to be pulled from that particular release.
• Customer commitments are being missed.

What about the other Defects

There are medium and low-priority defects that will be identified and logged by the Testers. These also need to be handled appropriately by the Development and QA Team.

Thus, overall it is a voluminous exercise.

There is no Silver Bullet

It is a well-known fact that no amount of testing can unearth every defect that a Software Product or System has. Meaning, there is an end to testing nor is the product defect-free.

However, from the ‘Serviceability’ point of view in a Competitive and Time To Market (TTM) model, there is a need to break the typical mindset to unearth maximum defects early in a Release cycle, especially identification of critical and high severity defects.

Any or all of the above will have a negative impact on the Organization’s business. In this context, adopting “Early Testing” as a separate Test activity will be beneficial for the overall management of SDLC for a given project or release.

Scope of Early Testing Efforts

Having understood the objective of Testing Early in the previous section titled . Why Early Testing? Let us now discuss the “Scope of Early Test Effort’ in detail.

As we are introducing Testing Early as a new activity to be tracked exclusively during the course of Testing execution, it is recommended to practice the scope of the testing effort as explained below

Assumption

• The entire Project and Software Release schedule has been approved and made available to all stakeholders.
• The overall Test Strategy document has been developed, reviewed, and approved by all the stakeholders.
• High, Medium and Low priority features to be tested are well documented.
• Test Plans and Test cases for all the Features are developed, reviewed, and approved by all the stakeholders.
• All Test Plans and Test Cases are uploaded to a central repository for tracking testing execution.
• All human resources, infrastructure equipment, and tools are available for setting up the testbed(s) and executing Test plans.

How to Test Early

Figure 4 – Overall approach to the scope of Testing Early

Approach

• Let us take an Example of Release XYZ having 3 High Priority features A, B, and C, 10 Medium priority features, and 15 Minor (or Low priority) features.
• High Priority features are those that are generating high revenue and/or standard compliance and/or competitor catch-up and/or competitor one-upmanship and all of this.
• High priority features usually involve some complex coding, with a large number of new lines of code added.
• A large number of new lines of code may also mean a high probability of intersection areas.
• Usually, High Priority features and/or features that have a large number of new lines of code are the best candidates for Testing Early.
• There does not need to be a separate Test Plan developed for Early Test activity.
• QA Leads and Testers along with the Development Leads and SMEs (Subject Matter Experts) need to discuss and agree upon the Code/Testing coverage for this testing activity.
• Identify appropriate high priority test cases and even some medium priority test cases if you think they are necessary from each of the feature test plans A, B, and C.
• Once the appropriate features and subsets of Test cases are identified, make sure they are tracked using the Test tracking tool adopted by the organization.

Hint: Collaboration is Key! During Early Test activity both the Development and QA teams need to collaborate closely to make sure that the set objectives are achieved with quality results.

Start and Exit on Early Test

It is important that both the Development and the QA Team brainstorm and agree to all the approaches of the entire Early Test activity including the Start and Exit dates so that all are on the same page.

Entry Criteria for Start

• Percentage of Integration testing completion
• Number of open bugs
• No blockers to start Early Test

Activity Phase

• Tracking progress
• No. of code drops during this testing
• Bug fixing approach
• Bug Verification approach
• Record these testing results

Exit Criteria

• Hand-off activities to the Next Phase of Testing (usually Feature Testing).
• Resolution of unresolved bugs found during Early Tests.
• Resolution for blockers, if any, for the next phase of Testing.
• Publish Early testing results.

Pros and Cons

Every new initiative or activity has its own merits and demerits.

Let us explore the pros and cons of this testing approach.

Pros

• Ideally suited for the Waterfall model.
• Helps uncover critical bugs early in the testing cycle.
• Identify critical bugs early in the release cycle.
• Helps the Development Team to stabilize the Code early.
• Helps to minimize the collateral due to bug fixes.
• Helps the Development Team identify vulnerabilities across intersection areas in detail early in the release cycle.
• The Management Team can make appropriate business decisions with due diligence on unresolved critical bugs in that particular Release or a Project.
• Helps to extend test coverage and cycle effectively.
• Helps distribute Development and Testing resources efficiently and effectively.

Cons

• Not ideally suited for Agile/Scrum models. However, such models can be adopted as Early Tests in Sprints with appropriate tweaking.
• There is a chance of reduced Integration Testing by the Development Team.

Conclusion

Customers or end-users buy or adopt serviceability products or a system or solutions. Validating software that is running on such a system or product for its serviceability is the primary requirement.

Key components of Principles of Testing like Why to Test? What is Testing? What to Test? How to Test? are mostly well defined and understood.

However, there are some lingering questions that keep propping up the minds of Readers, Testers, Leads, and Managers on concepts like Early Testing.

Adoption of Early Testing as an integral activity of the overall Testing Schedule for any given Software Project or a Release immensely benefits the Organization to deliver a robust qualified Product or a System.

Have you ever realized the importance of Early Testing in your career? Feel free to share your thoughts and experiences in the comments section below! We would love to hear from you.