Software Testing

Testing Terminology

• software testing is the activity of evaluating a software item to detect the differences between the item‘s required response and its actual response for a given set of input conditions
• input, software item, response
• scope of a test is the collection of underlying software components that will be tested
• object-oriented software is tested, the scope could be a class, one or more methods in a class, a group of classes, or the entire system
• dynamic testing: executing the software
• static testing: often performed both on software components and certain work products
• Fault-directed tests are designed to try and expose faults in the item under test.
• Conformance-directed tests are designed to demonstrate that the item under test satisfies its requirements.
• A test-case is a specific set of input data, events and interactions, and associated expected results, that is applied to the item under test.
• test set is a collection of one or more test cases applied to the item under test
• Regression testing involves repeating a subset, and sometimes a complete set, of previously-executed tests after a change has been made to the item under test.

Testing Perspectives

• Black box testing is an approach or strategy that focuses on designing test cases based on the requirements for a product.
• There are three key drivers to how effective a black box testing effort will be:
• the test design techniques used
• the quality of the requirements
• the level of subject matter expertise there is on the test design team
• lack of a systematic testing process
• testing too late in life cycle
• poor requirements
• unclear responsibilities
• data first test cases

Testing Principles

• One major misconception is that software testing can prove that a software product is free from defects.
• In order for dynamic testing to prove the correctness of a software product we would have to do exhaustive testing.
• Testing activities should begin as early as the requirements phase
• Test plans should be driven by, and linked to, the product requirements
• States the components of a test case
• Test case should be as repeatable as possible.
• Keep a test repository
• Regression testing
• In practice, more defects are generally found during testing in projects where an independent test team participates.
• Testing effectiveness can be improved by including people with business expertise on the test team.
• Tester should never assume that software is correct.

Testing Activities and Work Products

• In most software project life cycles, there is a phase devoted to testing
• If a waterfall life cycle model is used, defect detection and repair costs can be significant. If an incremental life cycle model is used, defect detection and repair costs are not as extensive provided that the timeline for delivering increments is short.
• incremental delivery is one way to help mitigate these costs.
• life cycle approach to testing
• Some type of testing is performed at each major phase of the software life cycle
• testing activities are performed essentially in parallel to development activities
• There is more frequent error detection and repair activity and some error detection and correction feedback at every major life cycle phase.
• Major testing issues that can be addressed at the requirements phase include requirements completeness, requirements testability, and requirements ambiguity.
• The type of testing that would be used at this point in a project would be static testing...namely conducting requirements walkthroughs or inspections.
• A tester‘s interest in those reviews would be to assure that the three issues are addressed
• During the design phase, the main design issues are typically correctness and completeness.
• Two deliverables that would be produced under the life cycle approach are a test plan and a test description document.
• In terms of timing, a test plan would be scheduled for delivery about half way through the design phase, or at the end of preliminary design if a preliminary design phase is used.
• The test descriptions would be scheduled for delivery ideally by the end of the design phase.
• A similar set of correctness and completeness issues applies during the coding through testing phases.
• Of course, we would be performing dynamic testing during this period, but static testing could also be performed on additional test deliverables, and developers could also perform static testing on the code.
• Traceability might also be applied to trace test cases back to requirements
• Testing tools might also be used during this period to help generate the actual test case data and to verify the extent to which tests actually cover code components.
• Additional testing work products might be test procedures, possibly test reports, and what are typically called problem reports.

Testing Levels and Responsibilities

• Black Box Testing focuses on designing tests based upon the requirements for the item under test.
• The code for the item under test is not needed or used in the design of tests.
• White Box Testing focuses on designing tests based upon examination of the structural logic for the item under test.
• It is also called structural testing, logic-based testing, clear box testing and glass box testing
• Object-oriented designs tend to be driven by class responsibilities and collaborations rather than by functional decomposition.
• Unit testing in object-oriented development projects works a little differently.
• Since the individual methods belong to a class, some or all of the methods are developed and then individual methods of the class are tested one at a time.
• A test driver simply creates a class instance and invokes one or more of its methods.
• If a method calls another method in the same class, either a stub is used or the actual method is used. If a method calls a method in another class, a stub may be used if the other class is not yet developed.
• In object-oriented integration testing, there really isn’t a functional-decomposition hierarchy, so we typically look at class collaborations, dependencies, and architectural layers.
• Object-oriented design architecture is layered, so maybe these four classes constitute the application layer of the architecture.
• We may then use a test driver to simulate the user-interface layer during the integration increments and while the actual GUI components are still under development.
• Depending on the level of test that is being done, the testing strategies will be different.
• At the unit level, both black box and white box tests are used.
• As we go into integration testing, tests generally become mostly black box in nature, and at the system and acceptance levels where the software product is tested in its entirety...the tests should be all black box.
• When it comes to object-oriented testing, the responsibilities are the same, and the objectives at the system and acceptance levels are the same...but there are additional types of testing that need to be done at the lowest levels.
• There are different types of relationships that exist between classes in object-oriented software development, and those relationships require that more types of testing be done.
• Also, object-oriented projects tend to be use case driven, so it is common for the use cases to serve as the basis for system and acceptance testing...at least for a product‘s functional requirements.

Requirements Based Test Design

• The first technique that will help us to design tests is called input space partitioning.
• Input space partitioning is used to partition the input side of a function into a set of equivalence classes that can help us identify test conditions.
• An equivalence class is a subset of the input domain such that any values in that class will result in an equivalent system response.
• Boundary testing focuses on identifying test conditions at the extreme points of the equivalence classes.
• model-driven test design: started with a requirement, modeled its business rules, and then used the model to help generate test cases

Code Based Test Design

• WHITE BOX TESTING focuses on designing tests based upon examination of the structural logic for the item under test.
• It is also called structural testing, logic-based testing, clear box testing and glass box testing.
• When we deal with code-based testing, one of the things we want to include as part of our testing goals is something called a coverage criteria.
• Including a coverage criteria ensures that the code under test is adequately exercised.
• three different levels of coverage criteria: statement coverage, branch coverage, and decision/condition coverage.
• Statement coverage involves writing enough tests so that each programming statement in the component under test is exercised at least once.
• For branch coverage we need to write enough tests so that each true/false branch is exercised at least once.
• The next type of coverage criteria is decision/condition coverage. This requires that we write enough tests so that each true/false decision branch is executed at least once and all combinations that can cause a decision to be true and false be tested.
• Basis Path Testing: Utilizes the flowgraph for a method in conjunction with the cyclomatic complexity metric to generate a set of test paths that yield a very high degree of code coverage.