Six Sigma and Software Engineering and Reliability

I recently finished reading the book “What is six-sigma?” by Peter Pande, and Larry Holpp. In terms of Software Engineering, Six Sigma is much more than a specific analysis of software reliability. It is a quality improvement framework, and mindset focused on the measurement of process variation as the culprit for lack of quality. I want to point out that the term “six sigma” when used in conjunction with software reliability, has little or nothing to do with statistics, with distributions, with their moments, etc. It is a buzzword and will remain a buzzword until such a time as it is defined in statistically correct ways.

The real Sense for Six Sigma

Six Sigma as the name implies stands for six standard deviations from the mean. Sigma is a statistical measure of variability around the average. The concept of Six Sigma comes from reliability engineering prediction of system or component failure probabilities. For example, the wear out time of a component may be normally distributed – that is meant – standard deviation. So, we want a component having a very small of failure before its design life. If, we set this at one sigma from the mean, we get ~80% reliability, and 2 sigmas gives us ~95%, and 3 sigmas ~99%, and so on. Six Sigma gives us ~99.9997% reliability – near perfect; or, in other ways 3.4 defects per million.

Six Sigma and Software Reliability.

In terms of software engineering, however, it is not so quite clear cut as compared to mechanical or electronic components. Also in case of software reliability, we don’t have very good predictive models, failure models, etc. As somebody suggested, that one approach to this could be to predict faults remaining as a function of faults found in earlier phases. In general terms, for software reliability, Six Sigma would mean that the software process will find ~99.9997% of all the faults before the software is put into service.

What do we need to do?

We need to adjust the design life accordingly. In common terms, the design life of shrink wrapped software is ten seconds before we open the package, and for the custom software ten seconds after the check clears.

In the language of Motorola official release:

”Motorola wants to be free of errors and defects 99.9997% of the time in all that it does. That means no more than 3.4 defects per million units.”  - ‘Electronic Business’, October 16, 1989

Statistical Tools – Improved Software Quality

Use of Statistical Tools to Improve Software Quality and some points to remember regarding this:

• Today, the complexity and size of software has grown substantially, along  with the size and complexity of the silicon processors, perhaps exceeding  Moore’s Law (a doubling of processing power every 18 months).
• The business risk of developing very large software systems has spurred  the development of a very large shrink wrapped software industry, primarily  because of the failure of many very large complex systems.
• Software factories, of which the primary case would be Microsoft, flourish  by delivering very large, internally complex products, at prices consumers  can afford to bear, exclusively by delivering extremely large volumes of like  products. The only technique that has proven effective for quality assurance   is using thousands of volunteer quality inspectors (beta testers) to report  the errors prior to the final release of the product. Because the cost of manufacturing   beta copies is so low, it is far out weighed by the economic benefit the company  receives from this type of testing process.
• Hence, can we ever assume that the software development industry will ever  achieve on standardized uniform measure of software quality, given that to  be relevant, the definition of a software standard must be reached between  the consumer of the software are and the producer of the software ? I would conjecture,  probably no.
• The reason for this is due to the nature of software. An algorithm may  be provably correct but may be implemented in an inefficient manner. (A possible   defect). It might be physically damaged in the duplication of a disk (a manufacturing   problem), which might manifest itself by the consumer being unable to install   and use the product. The cause of the problem, may remain the inefficient   implementation of the algorithm, but it manifests itself in so many potential  ways, it will be in all likelihood, impossible for the consumer to identify  the defect, and unless a defect can be quantitatively measured it will be impossible   to detect.
• At the very core of the problem, the inefficient algorithm might be the  work of one designer or developer, being unaware that more efficient mechanisms   might exist, or it may be the result of a specification error, or perhaps the   algorithm subroutine was purchased from an outside supplier, who provided   poor instructions regarding it’s limitations.
• Statistical tools can be used to analyze overall system quality, such as   a transaction failure. These tools are severely limited in the applicability   to an individual software developer because the development task is typically   to design and write single software modules, as opposed to a large scale software   use.
• We keep learning more and developing new insights, so things will change,   most probably through the use of better software partitioning and packaging   technology.

Conclusion

In the end, the people at large, the **users** does not understand why a concept that is worthy and meaningful in the hardware and manufacturing domain ***does not*** apply to software. Consequently, the **users** might be mislead and ill-served because they are led to believe “six sigma” software is somehow comparable to “six sigma” hardware. Is it?? Does it??

[I am convinced that others who have read the authoritative literature on six sigma and have attended the appropriate training could talk more intelligently about this technology.]

© Manoj Khanna 2003 – 2013.

0

Share