Capability Maturity Model - Developed by the software community in 1986 with leadership from the SEI. The CMM describes the principles and practices underlying software process maturity. It is intended to help software organizations improve the maturity of their software processes in terms of an evolutionary path from ad hoc, chaotic processes to mature, disciplined software processes. The focus is on identifying key process areas and the exemplary practices that may comprise a disciplined software process.
What makes up the CMM? The CMM is organized into five maturity levels:
·Initial
·Repeatable
·Defined
·Manageable
·Optimizing
Except for Level 1, each maturity level decomposes into several key process areas that indicate the areas an organization should focus on to improve its software process.
Level 3 – Defined: Key practice areas - Organization process focus, Organization process definition, Training program, Integrated software management, Software product engineering, Intergroup coordination, Peer reviews
Level 4 – Manageable: Key practice areas - Quantitative Process Management, Software Quality Management
Level 5 – Optimizing: Key practice areas - Defect prevention, Technology change management, Process change management
Six Sigma
Six Sigma is a quality management program to achieve "six sigma" levels of quality. It was pioneered by Motorola in the mid-1980s and has spread to many other manufacturing companies, notably General Electric Corporation (GE).
Six Sigma is a rigorous and disciplined methodology that uses data and statistical analysis to measure and improve a company's operational performance by identifying and eliminating "defects" from manufacturing to transactional and from product to service. Commonly defined as 3.4 defects per million opportunities, Six Sigma can be defined and understood at three distinct levels: metric, methodology and philosophy...
Training Sigma processes are executed by Six Sigma Green Belts and Six Sigma Black Belts, and are overseen by Six Sigma Master Black Belts.
ISO
ISO - International Organization for Standardization is a network of the national standards institutes of 150 countries, on the basis of one member per country, with a Central Secretariat in Geneva, Switzerland, that coordinates the system. ISO is a non-governmental organization. ISO has developed over 13, 000 International Standards on a variety of subjects.
Testing of software is defined very differently by different people and different corporations. You have process standards bodies, like ISO, SPICE, IEEE, etc. that attempt to impose a process to whatever types of development projects you do (be it hardware, software, embedded systems, etc.) and some of that will, by proxy, speak to testing. However, these are more there to guide the process and not the testing, such as it is. So, for example, IEEE will give you ideas for templates for such things as test case specifications, test plans, etc. That may help you out. On the other hand, those IEEE templates tell you nothing about actually testing the product itself. They basically just show you how to document that you are testing the product. The same thing pretty much applies with ISO. ISO is the standard for international projects and yet it, like IEEE, does not really force or even advocate a certain "testing standard." You also have other process and project oriented concepts out there like the Capability Maturity Model (CMM)
Some of the organization that define testing standards are
· BS – British Standards
· ISO- International Organization of Standards
· CMM- Capability Maturity Model
· SPICE- Software Process Improvement and Capability Determination
· NIST-National institute of Standards and Technology
· DoD-Department of Defense
1.SW – CMM:
SEI - Software Engineering Institute, Carnegie Mellon University.
CMM - Capability Maturity Model
Software Process
A software process can be defined as a set of activities, methods, practices, and transformations that people use to develop and maintain software and the associated products Software Process Capability
Software Process Capability describes the range of expected results that can be achieved by following a software process.The software process capability of an organization provides one means of predicting the most likely outcomes to be expected from the next software project the organization undertakes.
Software Process Maturity
Software Process Maturity is the extent to which a specific process is explicitly defined, managed, measured, controlled, and effective. Maturity implies a potential growth in capability and indicates both the richness of an organization’s software process and the consistency with which it is applied in projects throughout the organization
The five levels of SW- CMM
Leel 1: Initial
Level 2: Repeatable
Level 3: Managed
Level 4: Defined
Level 5: Optimum
2. SW – TMM
SW-TMM is a testing process improvement tool that can be used either in conjunction with the SW-CMM or as a stand-alone tool.
2.1. Levels of SW –TMM
2.1.1. Level 1: Initial
· A chaotic process
· Not distinguished from debugging and ill defined
· The tests are developed ad hoc after coding is complete
· Usually lack a trained professional testing staff and testing tools
· The objective of testing is to show that the system and software work
2.1.2. Level 2: Phase Definition
· Identify testing as a separate function from debugging
· Testing becomes a defined phase following coding
· Standardize their process to the point where basic testing techniques and methods are in place
· The objective of testing is to show that the system and software meets specifications
2.1.3. Level 3: Integration
· Integrate testing into the entire life cycle
· Establish a formal testing organization
· establishes formal testing technical trainings
· controls and monitors the testing process
· begins to consider using automated test tools
· Te objective of testing is based on system requirements
· Major milestone reached at this level: management
· recognizes testing as a professional activity
2.1.4. Level 4: Management and Measurement
· Testing is a measured and quantified process
· Development products are now tested for quality attributes such as Reliability, Usability and Maintainability.
· Test cases are collected and recorded in a test database for reuse and regression testing
· Defects found during testing are now logged, given a severity level, and assigned a priority for correction
2.1.5. Level 5: Optimization / Defect Prevention and Quality Control
· Testing is institutionalized within the organization
· Testing process is well defined and managed
· Testing costs and effectiveness are monitored
· Automated tools are a primary part of the testing process
· There is an established procedure for selecting and evaluating testing tools
2.2. Need to use SW-TMM
· easy to understand and use
· provide a methodology to baseline the current test
· process maturity
· designed to guide organization
· selecting process improvement strategies
· identifying critical issues to test process maturity
· provide a road map for continuous test process improvement
· provide a method for measuring progress
· allow organizations to perform their own assessment
Organizations that are using SW-CMM
· SW-TMM fulfills the design objective of being an excellent companion to SW-CMM
· SW-TMM is just another assessment tool and easily incorporated into the software process assessment
Organizations that are not using SW-CMM
· provide an unbiased assessment of the current testing process
· provide a road map for incremental improvements
· save testing cost as the testing process moves up the maturity levels
2.3. SW-TMM Assessment Process
· Prepare for the assessment
· choose team leader and members
· choose evaluation tools (e.g. questionnaire)
· training and briefing
· Conduct the assessment
· Document the findings
· Analyze the findings
· Develop the action plan
· Write the final report
· Implement the improvements best to implement the improvements either in a pilot project or in phases track progress and achievements prior to expanding organization wide also good in a limited application easier to fine-tune the new process prior to expanded implementation
2.4.SW-TMM Summary
· baseline the current testing process level of maturity
· identify areas that can be improved
· identify testing processes that can be adopted organization-wide
· provide a road map for implementing the improvements
· provide a method for measuring the improvement results
· provide a companion tool to be used in conjunction with the SW-TMM
3. ISO : International Organisation for Standardisation
· Q9001 – 2000 – Quality Management System : Requirements
· Q9000 – 2000 – Quality Management System : Fundamentals and Vocabulary
· Q9004 – 2000 – Quality Management System : Guidelines for performance improvements
4.4.ANSI / IEEE Standards
ANSI - ‘American National Standards Institute’ IEEE Standards: Institute of Electrical and electronics Engineers (Founded in 1884) Have an entire set of standards devoted to software. Testers should be familiar with all the standards mentioned in IEEE.
1. 610.12-1990 IEEE Standard Glossary of Software Engineering Terminology
2. 730-1998 IEEE Standard for Software Quality Assurance plans
3. 828-1998 IEEE Standard for Software Configuration Management
4. 829-1998 IEEE Standard for Software Test Documentation
5. 830-1998 IEEE Recommended Practice for Software Requirement Specifications.
6. 1008-1987(R1993) IEEE Standard for Software Unit Testing
7. 1012-1998 IEEE Standard for Software Verification and validation
8. 1012a-1998 IEEE Standard for Software Verification and validation –Supplement to 1012-1998 Content
9. 1016-1998- IEEE Recommended Practice for Software Design description
10. 1028-1997 IEEE Standard for Software Reviews
11. 1044-1993 IEEE Standard Classification for Software Anomalies
12. 1045-1992 IEEE Standard for Software Productivity metrics
13. 1058-1998 IEEE Standard for Software Project Management Plans
14. 1058.1-1987 IEEE Standard for Software Management
15. 1061-1998.1 IEEE Standard for Software Quality Metrics Methodology.
4.5.BCS - SIGIST
A meeting of the Specialist Interest Group on Software Testing was held in January 1989 (this group was later to affiliate with the British Computer Society). This meeting agreed that existing testing standards are generally good standards within the scope which they cover, but they describe the importance of good test case selection, without being specific about how to choose and develop test cases.
The SIG formed a subgroup to develop a standard which addresses the quality of testing performed. Draft 1.2 was completed by November 1990 and this was made a semi-public release for comment. A few members of the subgroup trialed this draft of the standard within their own organizations. Draft 1.3 was circulated in July 1992 (it contained only the main clauses) to about 20 reviewers outside of the subgroup. Much of the feedback from this review suggested that the approach to the standard needed re-consideration.
“Agile testing involves testing from the customer perspective as early as possible, testing early and often as code becomes available and stable enough from module/unit level testing.” - A wikipedia definition.
Why Need of Innovations in the Age of Agile Testing?
Global Recession/Economic downtime effect
Current Events are not Current Trends –
When global downturns hit, there is certain inevitability to their impact on information technology and Finance Sectors. Customers become more reluctant in giving software business. Some customers are withdrawing their long term projects and some customers using the opportunities in quoting low price. Many projects that dragged much longer than expected and cost more than planned. So, Companies started to explore how “Agile with different flavors” can help their Enterprises more reliably deliver software quickly and iteratively. The roles and responsibilities of Test Managers/Test Architects become more important in implementing Agile Projects. Innovations are increasingly being fueled by the needs of the testing society at large.
The Challenges in Agile Testing
Agile Testers face lot of challenges when they are working with Agile development team. A tester should be able to apply Root-Cause Analysis when finding severe bugs so that they unlikely to reoccur. While Agile has different flavors, Scrum is one process for implementing Agile. Some of the challenging scrum rules to be followed by every individual are
Obtain Number of Hours Commitment Up Front
Gather Requirements / Estimates Up Front
Entering the actual hours and estimated hours daily.
Daily Builds
Keep the Daily Scrum meetings short
Code Inspections are Paramount
So, in order to meet the above challenges, an agile tester needs to be innovative with the tools that they have. A great idea happens when what you have (tangible and intangible) meets the world’s deepest hunger
How Testers Can be More Innovative in the Age of Agile Testing?
Here are Important Keys to Innovation:
1. Creative
A good Agile Tester needs to be extremely creative when trying to cope up with speed of development/release. For a tester, being creative is more important than being critical.
2. Talented
He must be highly talented and strives for more learning and innovating new ideas. Talented Testers are never satisfied with what they have achieved and always strives to find unimaginable bugs of high value and priority.
3. Fearless
An Agile Tester should not be afraid to look at a developer’s code and if need be, hopefully in extreme cases, go in and correct it.
4. Visionary
He must have a comprehensive vision, which includes client’s expectations and delivery of the good product.
5. Empowered
He must be empowered to work in Pairs. He will be involving in Pair Programming to bring shorter scripts, better designs and finding more bugs.
6. Passionate
Passionate Testers always have something unique to contribute that may be in terms of their innovative ideas, the way they carry day-to-day work, their outputs and improve things around them tirelessly.
7. Multiple Disciplines
Agile Tester must have multiple skills like, Manual, Functional, Performance testing skills and soft skills like Leadership skills, Communication skills, EI, etc. so that agile testing will become a cake walk.
“Innovation is the process of turning ideas into manufacturable and marketable form” - Watts Humpry
