Test Coverage Analysis

These metrics are typically referred to as "coverage" and are a measure of completeness with respect to test selection criteria.

The amount of structural coverage should be commensurate with the level of risk posed by the software. Use of the term "coverage" usually means 100 percent coverage. For example, if a testing program has achieved "statement coverage," it means that 100 percent of the statements in the software have been executed at least once.

STATEMENT COVERAGE

This criterion requires sufficient test cases for each program statement to be executed at least once; however, its achievement is insufficient to provide confidence in a software product's behavior.

DECISION (BRANCH) COVERAGE

This criterion requires sufficient test cases for each program decision or branch to be executed so that each possible outcome occurs at least once. It is a minimum level of coverage for most software products, but decision coverage alone is insufficient for high-integrity applications.

CONDITION COVERAGE

This criterion requires sufficient test cases for each condition in a program decision to take on all possible outcomes at least once. It differs from multicondition branch coverage only when multiple conditions must be evaluated to reach a decision.

LOOP COVERAGE

This criterion requires sufficient test cases for all program loops to be executed for zero, one, two, and many iterations covering initialization, typical running, and termination (boundary) conditions.

DATA FLOW COVERAGE

This criterion requires sufficient test cases for each feasible data flow to be executed at least once. Many data flow testing strategies are available.

MULTI-CONDITION COVERAGE

This criterion requires sufficient test cases to exercise all possible combinations of conditions in a program decision.

CORRESPONDENCE BETWEEN DATA AND FIELD TYPES

Verify proper controls that limit the data or specific types of data that can be entered and accepted in field type. For example, verify that a date field (MM/DD/YYYY) cannot accept 19/34/2018.

DATA FLOW COVERAGE

This criterion requires sufficient test cases for each feasible data flow to be executed at least once. Many data flow testing strategies are available.

PATH COVERAGE

This criterion requires sufficient test cases for each feasible path, basis path, etc., from start to exit of a defined program segment, to be executed at least once. Because of the very large number of possible paths through a software program, path coverage is generally not achievable. The amount of path coverage is normally established based on the risk or criticality of the software under test.

POPULATING REQUIRED FIELDS

Some applications and web pages contain fields that are marked as required. Test the condition of leaving the required populated fields as unpopulated. Verify proper application response to missing data input.

ALLOWED NUMBER OF CHARACTERS

Verify that a limited number of characters can be accepted in a field. For example, a field that allows only 25 characters should not accept 26.

Allowed data bounds and limits

Applications can use input fields that accept data in a certain range. Verify that data out of range (below lower or above upper bounds) is not accepted.