I recently finished reading the book “What is six-sigma?” by Peter Pande, and Larry Holpp. In terms of Software Engineering, Six Sigma is much more than a specific analysis of software reliability. It is a quality improvement framework and mindset focused on the measurement of process variation as the culprit for lack of quality. I want to point out that the term “six sigma” when used in conjunction with software reliability, has little or nothing to do with statistics, with distributions, with their moments, etc. It is a buzzword and will remain a buzzword until such a time as it is defined in statistically correct ways.
The real Sense for Six Sigma
Six Sigma as the name implies stands for six standard deviations from the mean. Sigma is a statistical measure of variability around the average. The concept of Six Sigma comes from reliability engineering prediction of system or component failure probabilities. For example: the wearout time of a component may be normally distributed – that is mean – standard deviation. So, we want a component having a very small of failure before its design life. If we set this at one sigma from the mean we get ~80% reliability, 2 sigmas gives us ~95%, 3 sigmas ~99%, and so on. Six Sigma gives us ~99.9997% relaibility – near perfect. Or in other ways 3.4 defects per million.
In terms of software engineering, however, it is not so quite clear cut as compared to mechanical or electronic components. Also in case of software reliability, we don’t have very good predictive models, failure models, etc. As somebody suggested that one approach to this could be to predict faults remaining as a function of faults found in earlier phases. In general terms for software reliability Six Sigma would mean that the software process will find ~99.9997% of all the faults before the software is put into service. What do we need to do?
We need to adjust the design life accordingly. In common terms, the design life of shrink wrapped software is ten seconds before we open the package, and for the custom software ten seconds after the check clears. In the language of Motorola official release:
“Motorola wants to be free of errors and defects 99.9997% of the time in all that it does. That means no more than 3.4 defects per million units.”
– ‘Electronic Business’, October 16, 1989 Use of Statistical Tools to Improve Software Quality and some points to remember regarding this:
- Today, the complexity and size of software has grown substantially, along
with the size and complexity of the silicon processors, perhaps exceeding
Moore’s Law (a doubling of processing power every 18 months). - The business risk of developing very large software systems has spurred
the development of a very large shrink wrapped software industry, primarily
because of the failure of many very large complex systems. - Software factories, of which the primary case would be Microsoft, flourish
by delivering very large, internally complex products, at prices consumers
can afford to bear, exclusively by delivering extremely large volumes of like
products. The only technique that has proven effective for quality assurance,
is using thousands of volunteer quality inspectors (beta testers) to report
the errors prior to final relase of the product. Because the cost of manufacturing
beta copies is so low, it is far out weighed by the economic benefit the company
receives from this type of testing process. - Hence, can we ever assume that the software development industry will ever
achieve on standardized uniform measure of software quality, given that to
be relevant, the definition of a software standard must be reached between
the consumer of that software and the producer of the software ? I would conjecture,
probably no. - The reason for this is due to the nature of software. An algorithm may
be provably correct, but may be implemented in an inefficient manner. (A possible
defect). It might be physically damaged in the duplication of a disk (a manufacturing
problem), which might manifest itself by the consumer being unable to intsall
and use the product.The root cause of the problem, may remain the inefficent
implementation of the algorithm, but it manifests itself in so many potential
ways, it will be in all likelyhood, impossible for the consumer to identify
the defect. Unless a defect can be quantitatively measured it will be impossible
to detect. - At the very core of the problem, the inefficient algorithm might be the
work of one designer or developer, being unaware that more efficient mechanisms
might exist, or it may be result of a specification error, or perhaps the
algorithm subroutine was purchased from an outside supplier, who provided
poor instructions regarding it’s limitations. - Statistical tools can be used to analyze overall system quality, such as
a transaction failure. These tools are severly limited in the applicability
to a individual software developer, because the development task is typically
to design and write single software modules, as opposed to large scale software
reuse. - We keep learning more and developing new insights, so things will change,
most probably through the use of better software partitioning and packaging
technology.
Conclusion
In the end, the people at large, the **users** does not understand why a concept that is worthy and meaningful in the hardware and manufacturing domain ***does not*** apply to software. Consequently, the **users** might be mislead and ill-served because they are led to believe that “six sigma” software is somehow comparable to “six sigma” hardware. Is it?? Does it??
[I am convinced that others who have read the authoritative literature on six sigma and have attended appropriate training could talk more intelligently about this technology.]
Powered by Dextrus Prosoft, Inc.
