Thursday, April 30, 2009

hOW TO wRITE gOOD tEST CASES

What is a good test case?
  • Accurate - tests what it’s designed to test
  • Economica - no unnecessary steps
  • Repeatabe, reusabe - keeps on going
  • Traceabe - to a requirement
  • Appropriate - for test environment, testers
  • Sef standing - independent of the writer
  • Sef ceaning - picks up after itsef
Types of test cases
  • A rose by any other name smells as sweet
  • Step-by-step or word/action instructions
  • Table, matrix
  • Script for record/playback or performance test
  • All need the same structure (anatomy)
Anatomy of a test case
  • Statement of purpose, what is being tested
  • Method, how it will be tested
  • Setup, environment, data
  • Steps - actions and expected results, implied in atable or matrix
  • Proofs, files, printouts, reports, screen grabs(optional)
Improving testability - language
  • Testability = easy to test
  • Use active case, do this, do that
  • System displays this, does that
  • Simple, conversational language
  • Exact, consistent names of fields, not generic
  • Don’t explain Windows basics
Improving testability - length
  • 10-15 steps or less, unless user cannot savework
  • Uniform time to test
  • Wide range of testers
  • Pros and cons of cumulative cases
  • Order of cases follows business scenarios
The seven most common mistakes
  • Making cases too long
  • Incomplete, incorrect, or incoherent setup
  • Leaving out a step
  • Naming fields that changed or no longer exist
  • Unclear whether tester or system does action
  • Unclear what is a pass or fail result
  • Failure to clean up
Test Case Checklist
Quality Attributes
  • Accurate - tests what the description says it will test.
  • Economical - has only the steps needed for its purpose
  • Repeatable, self standing - same results no matter who tests it.
  • Appropriate - for both immediate and future testers
  • Traceable - to a requirement
  • Self cleaning - returns the test environment to clean state
Structure and testability
  • Has a name and number
  • Has a stated purpose that includes what requirement is being tested
  • Has a description of the method of testing
  • Specifies setup information - environment, data, prerequisite tests, security access
  • Has actions and expected results
  • States if any proofs, such as reports or screen grabs, need to be saved
  • Leaves the testing environment clean
  • Uses active case language
  • Does not exceed 15 steps
  • Matrix does not take longer than 20 minutes to test
  • Automated script is commented with purpose, inputs, expected results
  • Setup offers alternative to prerequisite tests, if possible
  • Is in correct business scenario order with other tests
Configuration management
  • Employs naming and numbering conventions
  • Saved in specified formats, file types
  • Is versioned to match software under test
  • Includes test objects needed by the case, such as databases
  • Stored as read
  • Stored with controlled access
  • Stored where network backup operates
  • Archived off-site

Wednesday, April 29, 2009

Classic Testing Mistakes

The role of testing
  • Thinking the testing team is responsible for assuring quality.
  • Thinking that the purpose of testing is to find bugs.
  • Not finding the important bugs.
  • Not reporting usability problems.
  • No focus on an estimate of quality (and on the quality of that estimate).
  • Reporting bug data without putting it into context.
  • Starting testing too late (bug detection, not bug reduction)


Planning the complete testing effort
  • A testing effort biased toward functional testing.
  • Underemphasizing configuration testing.
  • Putting stress and load testing off to the last minute.
  • Not testing the documentation
  • Not testing installation procedures.
  • An overreliance on beta testing.
  • Finishing one testing task before moving on to the next.
  • Failing to correctly identify risky areas.
  • Sticking stubbornly to the test plan.


Personnel issues
  • Using testing as a transitional job for new programmers.
  • Recruiting testers from the ranks of failed programmers.
  • Testers are not domain experts.
  • Not seeking candidates from the customer service staff or technical writing staff.
  • Insisting that testers be able to program.
  • A testing team that lacks diversity.
  • A physical separation between developers and testers.
  • Believing that programmers can’t test their own code.
  • Programmers are neither trained nor motivated to test.


The tester at work
  • Paying more attention to running tests than to designing them.
  • Unreviewed test designs.
  • Being too specific about test inputs and procedures.
  • Not noticing and exploring “irrelevant” oddities.
  • Checking that the product does what it’s supposed to do, but not that it doesn’t do what it isn’t supposed to do.
  • Test suites that are understandable only by their owners.
  • Testing only through the user-visible interface.
  • Poor bug reporting.
  • Adding only regression tests when bugs are found.
  • Failing to take notes for the next testing effort.


Test automation
  • Attempting to automate all tests.
  • Expecting to rerun manual tests.
  • Using GUI capture/replay tools to reduce test creation cost.
  • Expecting regression tests to find a high proportion of new bugs.


Code coverage
  • Embracing code coverage with the devotion that only simple numbers can inspire.
  • Removing tests from a regression test suite just because they don’t add coverage.
  • Using coverage as a performance goal for testers.
  • Abandoning coverage entirely.

Sunday, April 12, 2009

User Acceptance Testing

After completion of system testing & their modifications, the project management concentrates on User acceptance testing to collect feedback.

These are two ways to conduct user acceptance testing. They are µ - testing and b - Testing. 
 
µ - Testing                                


1. By real customers          
2. In development site       
3. Suitable for Applications 
  b - Testing
1. By Model Customers
2. In Model Customer site
3. Suitable for Products



6. Release & Maintenance: -
After completion of user acceptance testing and their modifications, the project management concentrates on the release team.
This release team consists of a few programmers few test engineers & some hardware engineers.
a)     Port Testing
b)     Test Software Changes
 a) Port Testing: -
The corresponding release team conducts port testing on the customer site. During this test, the release team observes the below factors.
  •          Compact Installation
  •         Overall Functionality
  •         Input device handling
  •         Output device handling (Like Monitor, Printer, etc)
  •         Secondary storage device handling (Like Floppy disk, CD-Rom, etc)
  •         Operating System with other handling
  •         Co – Execution with other software.
After comparison of Port testing, the responsible release team Provides required training sessions to the customer. Side people & get back to the organization.
b) Test Software Changes: -
During the utilization of the released software, the customer side people send change requests (CR) to the organization. To receive these CRs, the organization establishes a special team along with a few programmers, some test engineers & a project manages category person. This team is called “Change Control Board” (CCB).








Saturday, April 4, 2009

Descriptive programming in QTP



Introduction:

This document demonstrates the usage of Descriptive programming in QTP 8.20. It also discusses situations where Descriptive programming can be used. Using Descriptive Programming automation scripts can be created even if the application has not been developed.

Descriptive Programming:

Whenever QTP records any action on any object of an application, it adds some description on how to recognize that object to a repository of objects called object repository. QTP cannot take action on an object until unless its object description is in the Object Repository. But descriptive programming provides a way to perform action on objects which are not in Object repository
Object Identification:
To identify an object during the play back of the scripts QTP stores some properties which helps QTP to uniquely identify the object on a page. Below screen shots shows an example Object repository:



Now to recognize a radio button on a page QTP had added 2 properties the name of the radio button and the html tag for it. The name the left tree view is the logical name given by QTP for the object. This can be changed as per the convenience of the person writing the test case. QTP only allows UNIQUE logical name under same level of hierarchy. As we see in the snapshot the two objects in Browser->Page node are “WebTable” and “testPath”, they cannot have the same logical name. But an object under some other node can have the same name. Now with the current repository that we have, we can only write operation on objects which are in the repository. Some of the example operations are given below


Browser("Browser").Page("Page").WebRadioGroup ("testPath").Select "2"
cellData = Browser("Browser").Page("Page").WebTable ("WebTable").GetCellData (1,1)
Browser("Example2").Page("Page").WebEdit("testPath").Set "Test text"


When and Why to use Descriptive programming?
Below are some of the situations when Descriptive Programming can be considered useful:
1.      The objects in the application are dynamic in nature and need special handling to identify the object. The best example would be of clicking a link which changes according to the user of the application, Ex. “Logout <>”.
2.      When object repository is getting huge due to the no. of objects being added. If the size of Object repository increases too much then it decreases the performance of QTP while recognizing a object.
3.      When you don’t want to use object repository at all. Well the first question would be why not Object repository? Consider the following scenario which would help understand why not Object repository
Scenario 1: Suppose we have a web application that has not been developed yet. Now QTP for recording the script and adding the objects to repository needs the application to be up, that would mean waiting for the application to be deployed before we can start of with making QTP scripts. But if we know the descriptions of the objects that will be created then we can still start off with the script writing for testing
Scenario 2: Suppose an application has 3 navigation buttons on each and every page. Let the buttons be “Cancel”, “Back” and “Next”. Now recording action on these buttons would add 3 objects per page in the repository. For a 10 page flow this would mean 30 objects which could have been represented just by using 3 objects. So instead of adding these 30 objects to the repository we can just write 3 descriptions for the object and use it on any page.
4.      Modification to a test case is needed but the Object repository for the same is Read only or in shared mode i.e. changes may affect other scripts as well.
5.      When you want to take action on similar type of object i.e. suppose we have 20 textboxes on the page and there names are in the form txt_1, txt_2, txt_3 and so on. Now adding all 20 the Object repository would not be a good programming approach.


How to use Descriptive programming?
There are two ways in which descriptive programming can be used
1.      By creating properties collection object for the description.
2.      By giving the description in form of the string arguments.


1.      By creating properties collection object for the description.
To use this method you need first to create an empty description


Dim obj_Desc ‘Not necessary to declare
Set obj_Desc = Description.Create
Now we have a blank description in “obj_Desc”. Each description has 3 properties “Name”, “Value” and “Regular Expression”.


obj_Desc(“html tag”).value= “INPUT”
When you use a property name for the first time the property is added to the collection and when you use it again the property is modified. By default each property that is defined is a regular expression. Suppose if we have the following description


obj_Desc(“html tag”).value= “INPUT”
obj_Desc(“name”).value= “txt.*”
This would mean an object with html tag as INPUT and name starting with txt. Now actually that “.*” was considered as regular expression. So, if you want the property “name” not to be recognized as a regular expression then you need to set the “regularexpression” property as FALSE


obj_Desc(“html tag”).value= “INPUT”
obj_Desc(“name”).value= “txt.*”
obj_Desc(“name”).regularexpression= “txt.*”
This is how of we create a description. Now below is the way we can use it


Browser(“Browser”).Page(“Page”).WebEdit(obj_Desc).set “Test”
When we say .WebEdit(obj_Desc) we define one more property for our description that was not earlier defined that is it’s a text box (because QTPs WebEdit boxes map to text boxes in a web page).
If we know that we have more than 1 element with same description on the page then we must define “index” property for the that description
Consider the HTML code given below
INPUT type=”textbox” name=”txt_Name”>
INPUT type=”radio” name=”txt_Name”>
Now the html code has two objects with same description. So distinguish between these 2 objects we will use the “index” property. Here is the description for both the object
 For 1st textbox:
      obj_Desc(“html tag”).value= “INPUT”
obj_Desc(“name”).value= “txt_Name”
obj_Desc(“index”).value= “0”
For 2nd textbox:
      obj_Desc(“html tag”).value= “INPUT”
obj_Desc(“name”).value= “txt_Name”
obj_Desc(“index”).value= “1”
Consider the HTML Code given below:


INPUT type=”textbox” name=”txt_Name”>
INPUT type=”radio” name=”txt_Name”>


We can use the same description for both the objects and still distinguish between both of them
obj_Desc(“html tag”).value= “INPUT”
obj_Desc(“name”).value= “txt_Name”
 When I want to refer to the textbox then I will use the inside a WebEdit object and to refer to the radio button I will use the description object with the WebRadioGroup object.
Browser(“Browser”).Page(“Page”).WebEdit(obj_Desc).set “Test” ‘Refers to the text box
Browser(“Browser”).Page(“Page”).WebRadioGroup(obj_Desc).set “Test” ‘Refers to the radio button
But if we use WebElement object for the description then we must define the “index” property because for a webelement the current description would return two objects.


Hierarchy of test description:
When using programmatic descriptions from a specific point within a test object hierarchy, you must continue to use programmatic descriptions
from that point onward within the same statement. If you specify a test object by its object repository name after other objects in the hierarchy have
been described using programmatic descriptions, QuickTest cannot identify the object.
For example, you can use Browser(Desc1).Page(Desc1).Link(desc3), since it uses programmatic descriptions throughout the entire test object hierarchy.
You can also use Browser("Index").Page(Desc1).Link(desc3), since it uses programmatic descriptions from a certain point in the description (starting
from the Page object description).
However, you cannot use Browser(Desc1).Page(Desc1).Link("Example1"), since it uses programmatic descriptions for the Browser and Page objects but
then attempts to use an object repository name for the Link test object (QuickTest tries to locate the Link object based on its name, but cannot
locate it in the repository because the parent objects were specified using programmatic descriptions).


Getting Child Object:
We can use description object to get all the objects on the page that matches that specific description. Suppose we have to check all the checkboxes present on a web page. So we will first create an object description for a checkboxe and then get all the checkboxes from the page


Dim obj_ChkDesc
Set obj_ChkDesc=Description.Create
obj_ChkDesc(“html tag”).value = “INPUT”
obj_ChkDesc(“type”).value = “checkbox”


Dim allCheckboxes, singleCheckBox
Set  allCheckboxes = Browse(“Browser”).Page(“Page”).ChildObjects(obj_ChkDesc)


For each singleCheckBox in allCheckboxes
      singleCheckBox.Set “ON”
Next


The above code will check all the check boxes present on the page. To get all the child objects we need to specify an object description i.e. we can’t use the string arguments that will be discussed later in the 2nd way of using the programming description.
Possible Operation on Description Object
Consider the below code for all the solutions
Dim obj_ChkDesc
Set obj_ChkDesc=Description.Create
obj_ChkDesc(“html tag”).value = “INPUT”
obj_ChkDesc(“type”).value = “checkbox”


Q: How to get the no. of description defined in a collection
A: obj_ChkDesc.Count ‘Will return 2 in our case
Q: How to remove a description from the collection
A: obj_ChkDesc.remove “html tag” ‘would delete the html tag property from the collection
Q: How do I check if property exists or not in the collection?
A: The answer is that it’s not possible. Because whenever we try to access a property which is not defined its automatically added to the collection. The only    way to determine is to check its value that is use a if statement “if obj_ChkDesc(“html tag”).value = empty then”.
Q: How to browse through all the properties of a properties collection?
A: Two ways
      1st:       For each desc in obj_ChkDesc
                              Name=desc.Name
                              Value=desc.Value
                              RE = desc.regularexpression
                  Next
      2nd:      For i=0 to obj_ChkDesc.count - 1
                              Name= obj_ChkDesc(i).Name
                              Value= obj_ChkDesc(i).Value
                              RE = obj_ChkDesc(i).regularexpression
                  Next
2.      By giving the description in form of the string arguments.
You can describe an object directly in a statement by specifying property:=value pairs describing the object instead of specifying an object’s
name. The general syntax is:
TestObject("PropertyName1:=PropertyValue1", "..." , "PropertyNameX:=PropertyValueX")
TestObject—the test object class could be WebEdit, WebRadioGroup etc….


PropertyName:=PropertyValue—the test object property and its value. Each property:=value pair should be separated by commas and quotation
marks. Note that you can enter a variable name as the property value if you want to find an object based on property values you retrieve during a run session.
Consider the HTML Code given below:


INPUT type=”textbox” name=”txt_Name”>
INPUT type=”radio” name=”txt_Name”>
Now to refer to the textbox the statement would be as given below
Browser(“Browser”).Page(“Page”).WebEdit(“Name:=txt_Name”,”html tag:=INPUT”).set “Test”
And to refer to the radio button the statement would be as given below
Browser(“Browser”).Page(“Page”).WebRadioGroup(“Name:=txt_Name”,”html tag:=INPUT”).set “Test”


If we refer to them as a web element then we will have to distinguish between the 2 using the index property
Browser(“Browser”).Page(“Page”).WebElement(“Name:=txt_Name”,”html tag:=INPUT”,”Index:=0”).set “Test” ‘ Refers to the textbox
Browser(“Browser”).Page(“Page”).WebElement(“Name:=txt_Name”,”html tag:=INPUT”,”Index:=1”).set “Test” ‘ Refers to the radio button