Testing Techniques

1. Static Testing Techniques

“Analysis of a program carried out without executing the program.”

… BS 7925-1

1.1.Review - Definition

Review is a process or meeting during which a work product or set of work products, is presented

to project personnel, managers, users, customers, or other interested parties for comment or

approval.

[IEEE]

1.2.Types of Reviews

There are three general classes of reviews:

· Informal / peer reviews

· Semiformal / walk-through

· Formal / inspections.

1.2.1. Walkthrough

“A review of requirements, designs or code characterized by the author of the material under review guiding the progression of the review. “

[BS 7925-1]

A 'walkthrough' is an informal meeting for evaluation or informational purposes. Little or no preparation is usually required.

These are led by the author of the document, and are educational in nature. Communication is therefore predominately one-way in nature. Typically they entail dry runs of designs, code and scenarios/ test cases.

1.2.2. Inspection

A group review quality improvement process for written material. It consists of two aspects; product (document itself) improvement and process improvement (of both document production and inspection).

[BS 7925-1]

An inspection is more formalized than a 'walkthrough', typically with 3-8 people including a moderator, reader, and a recorder to take notes. The subject of the inspection is typically a document such as a requirements specification or a test plan, and the purpose is to find problems and see what's missing, not to fix anything. Attendees should prepare for this type of meeting by reading thru the document; most problems will be found during this preparation. The result of the inspection meeting should be a written report. Thorough preparation for inspections is difficult, painstaking work, but is one of the most cost effective methods of ensuring quality. Led by trained moderator (not author), has defined roles, and includes metrics and formal process based on rules and checklists with entry and exit criteria.

1.2.3. Informal Review

· Unplanned and Undocumented

· Useful, Cheap and widely used

· Contrast with walkthroughs is that communication is very much two-way in nature

1.2.4. Technical Review

Technical reviews are also known as peer review as it is vital that participants are made up from the 'peer group', rather than including managers.

· Documented

· Defined fault detection process

· Includes peers and technical experts

· No management participant

1.3. Activities performed during review

Activities in Review: Planning, overview meeting, Review meeting and follow-up.

Deliverables in Review: Product changes, source document changes and improvements.

Factors for pitfall of review: Lack of training, documentation and management support. Review of the Requirements / Planning and Preparing Acceptance Test At the beginning of the project the test activities must start. These first activities are:

· Fixing of test strategy and test concept

· risk analysis

· determine criticality

· expense of testing

· test intensity

· Draw up the test plan

· Organize the test team

· Training of the test team - If necessary

· Establish monitoring and reporting

· Provide required hardware resources (PC, data base, …)

· Provide required software resources (software version, test tools, …)

The activities include the foundations for a manageable and high-quality test process. A test strategy is determined after a risk evaluation, a cost estimate and test plan are developed and progress monitoring and reporting are established. During the development process all plans must be updated and completed and all decisions must be checked for validity. In a mature development process reviews and inspections are carried out through the whole process. The review of the requirement document answers questions like: Are all customers’ requirements fulfilled? Are the requirements complete and consistent? And so on. It is a look back to fix problems before going on in development. But just as important is a look forward. Ask questions like: Are the requirements testable? Are they testable with defensible expenditure? If the answer is no, then there will be problems to implement these requirements. If you have no idea how to test some requirements then it is likely that you have no idea how to implement these requirements. At this stage of the development process all the knowledge for the acceptance tests is available and to hand. So this is the best place for doing all the planning and preparing for acceptance testing.

For example one can

· Establish priorities of the tests depending on criticality

· Specify (functional and non-functional) test cases

· Specify and - if possible - provide the required infra-structure

· At this stage all of the acceptance test preparation is finished and can be achieved.

1.4. Review of the Specification / Planning and Preparing System Test

In the review meeting of the specification documents ask questions like: Is the specification testable? Are they testable with defensible expenditure? Only these kinds of specifications can be ealistically implemented and be used for the next steps in the development process. There must be a re-work of the specifications if the answers to the questions are no. Here all the knowledge for the system tests is available and to hand. Tasks in planning and preparing for system testing include:

· Establishing priorities of the tests depending on criticality

· Specifying (functional / non-functional) system test cases

· Defining and establishing the required infra-structure

As with the acceptance test preparation, all of the system test preparation is finished at this early development stage.

· Review of the Architectural Design

· Detailed Design Planning and

· Preparing Integration/Unit Test

During the review of the architectural design one can look forward and ask questions like: What is about the testability of the design? Are the components and interfaces testable? Are they testable with defensible expenditure? If the components are too expensive to test a re-work of the architectural design has to be done before going further in the development process. Also at this stage all the knowledge for integration testing is available. All preparation, like specifying control flow and data flow integration test cases, can be achieved. All accordingly activities of the review of the architectural design and the integration tests can be done here at the level of unit tests.

1. Roles and Responsibilities

In order to conduct an effective review, everyone has a role to play. More specifically, there are certain roles that must be played, and reviewers cannot switch roles easily. The basic roles in a review are:

· The moderator

· The recorder

· The presenter

· Reviewers

Moderator:

The moderator makes sure that the review follows its agenda and stays focused on the topic at hand. The moderator ensures that side-discussions do not derail the review, and that all reviewers participate equally.

Recorder:

The recorder is an often overlooked, but essential part of the review team. Keeping track of what was discussed and documenting actions to be taken is a full-time task. Assigning this task to one of the reviewers essentially keeps them out of the discussion. Worse yet, failing to document what was decided will likely lead to the issue coming up again in the future. Make sure to have a recorder and make sure that this is the only role the person plays.

Presenter:

The presenter is often the author of the artifact under review. The presenter explains the artifact and any background information needed to understand it (although if the artifact was not selfexplanatory, it probably needs some work). It’s important that reviews not become “trials” – the focus should be on the artifact, not on the presenter. It is the moderator’s role to make sure that participants (including the presenter) keep this in mind. The presenter is there to kick-off the discussion, to answer questions and to offer clarification.

Reviewer:

Reviewers raise issues. It’s important to keep focused on this, and not get drawn into side discussions of how to address the issue. Focus on results, not the means.