Zero Defect Initiative for Software

Zero Defect Initiative for Software Development

What is a Zero Defect Initiative?

The Goal for a Zero Defect Initiative is to target zero known high severity software defects, a maximum of 10 low severity defects, including 3rd Party Vendor defects at the following SDLC phases:

• Entry in to System Test
• Customer Field Trials (Beta Test)
• Customer Deployment

Why is Zero Defect Complicated?

Because Defect Prediction is complicated

• Defect Arrival from previous Software Releases
• Defect Arrival from Current Software Release (This involves new technology, hardware, Operating Systems etc.)
• Traditional Defect Closure rate, versus, required Defect Closure Rate
• 3rd Party vendor Software issues, their Arrival and Closure Rates

Why do Zero Defect Analysis?

• Every organization usually has some “backlog” defects to contend with, i.e., a number of known Software defects that they have to live with. This “living backlog” defines the productivity of an organization.

• The larger the number of the living backlog, the less productive are the testing phases, as they have to account for the “known scenarios” and are blocked from running their tests, and if they do run their tests, it results in duplicate defects.

• The idea behind the zero defect initiative is to find that “number of steady state software defects”, that an organization can live with, implying a number of open defects, that the test organization can work around and still be effective and productive.

Zero Defect Initiative Team

• Six Sigma Black Belt/Quality Manager on the team
• Project Manager, who monitored incoming arrivals and drove the closure efforts
• Software development managers, who worked very closely with us, to drive defect resolutions by their engineers to meet the quota of their weekly backlog goals
• Director of Engineering, You always need the leadership support
• Senior Leadership team, and the Senior Director of Engineering who introduced the effort for all Engineering Teams.

Task for the Zero Defect Initiative Team

• Predict incoming Arrivals from previous software releases
• Predict incoming Arrivals from current software release
• Determine a closure rate (plan staffing based on peak arrivals)
• Arrival and Closure rates of 3rd Party vendor defects
• Present our results weekly at the organization level

Predicting Incoming arrivals

• Assumption: Arrival rate similar to previous release. To take advantage of the historical data base – the current release entailed;
  • New Hardware
  • New Operating System
  • X% reuse of existing code

(*Let me add another piece of detailed info, we were the platform team for this product, since this was new Hardware and Software, the maximum amount of the changes and challenges were for the Platform team.)

Applications supported by the Platform Team

• Base Station Controller – BSC (also referred to as iCP) (Two Platforms, the main Network Element and the Card Rack, Artesyn)
• Dispatch Application Controller – DAP
• Home Location Register – HLR
• Surveillance Gateway – SG
• SSC – New Controller for the new Platform

Functional Areas in the Platform Team

• The Platform itself had the following Functional Areas:
  • High Availability Functional area – HA
  • Middleware Layer Functional area – MW
  • Input/Output Layer Functional area – IO
  • Operating System Functional area – OS

(*We had to analyze historically the reuse among each FA, and also the defect densities of each of the platforms we supported.)

Tracking with SLIM for our Prediction and Actuals – 1 Month after test start date

Tracking with SLIM for our Prediction and Actuals – 3 Months after test start date

Tracking with SLIM for our Prediction and Actuals – Beta Release

Post Customer Release

Conclusions

• Our final count although close to 600.
  • We had about 40% duplicate defects, because of the new platform.

• Lesson Learned # 1
  • Because of all the products we were supporting, we probably should have factored a higher than usual duplicate defect rate.

• This was one of the most efficient programs, in terms of Cost of testing and meeting the schedule deadlines. Our customers were extremely satisfied.
  • This project is considered as the highest Customer Satisfaction across the company.

• Lesson Learned #2
  • We improved our efficiency/productivity exponentially and had a significant return on investment from the Zero Defect Initiative

• Lesson Learned #3
  • Software can be designed for a level of quality, and it is worth building the quality in your software instead of testing for quality!