Defect Reporting Guidelines

The key to making a good report is providing the development staff with as much information as necessary to reproduce the bug. This can be broken down into 5 points:

1. Give a brief description of the problem
2. List the steps that are needed to reproduce the bug or problem
3. Supply all relevant information such as version, project and data used.
   
4. Supply a copy of all relevant reports and data including copies of the expected results.
5. Summarize what you think the problem is.

When you are reporting a defect the more information you supply, the easier it will be for the developers to determine the problem and fix it.

Simple problems can have a simple report, but the more complex the problem– the more information the developer is going to need.

For example: cosmetic errors may only require a brief description of the screen, how to get it and what needs to be changed.

However, an error in processing will require a more detailed description, such as:

1) The name of the process and how to get to it.

2) Documentation on what was expected. (Expected results)

3) The source of the expected results, if available. This includes spread sheets, an earlier version of the software and any formulas used)

4) Documentation on what actually happened. (Perceived results)

5) An explanation of how the results differed.

6) Identify the individual items that are wrong.

7) If specific data is involved, a copy of the data both before and after the process should be included.

8) Copies of any output should be included.

As a rule the detail of your report will increase based on a) the severity of the bug, b) the level of the processing, c) the complexity of reproducing the bug.

Anatomy of a bug report

Bug reports need to do more than just describe the bug. They have to give developers something to work with so that they can successfully reproduce the problem.

In most cases the more information– correct information– given the better. The report should explain exactly how to reproduce the problem and an explanation of exactly what the problem is.

The basic items in a report are as follows:

Version: This is very important. In most cases the product is not static, developers will have been working on it and if they’ve found a bug– it may already have been reported or even fixed. In either case, they need to know which version to use when testing out the bug.

Product: If you are developing more than one product– Identify the product in question.

Data: Unless you are reporting something very simple, such as a cosmetic error on a screen, you should include a dataset that exhibits the error.

If you’re reporting a processing error, you should include two versions of the dataset, one before the process and one after. If the dataset from before the process is not included, developers will be forced to try and find the bug based on forensic evidence. With the data, developers can trace what is happening.

Steps: List the steps taken to recreate the bug. Include all proper menu names, don’t abbreviate and don’t assume anything.

After you’ve finished writing down the steps, follow them - make sure you’ve included everything you type and do to get to the problem. If there are parameters, list them. If you have to enter any data, supply the exact data entered. Go through the process again and see if there are any steps that can be removed.

When you report the steps they should be the clearest steps to recreating the bug.

Description: Explain what is wrong - Try to weed out any extraneous information, but detail what is wrong. Include a list of what was expected. Remember report one problem at a time; don’t combine bugs in one report.

Supporting documentation:

If available, supply documentation. If the process is a report, include a copy of the report with the problem areas highlighted. Include what you expected. If you have a report to compare against, include it and its source information (if it’s a printout from a previous version, include the version number and the dataset used)

This information should be stored in a centralized location so that Developers and Testers have access to the information. The developers need it to reproduce the bug, identify it and fix it. Testers will need this information for later regression testing and verification.