Software Validation Processes - Telecommunications software

[mmclean]

My company sells telecommunications software.

In our Quality Manual, I cover off 7.5.2 by stating:

"Due to the complexity of our software, complete validation of the software against requirements is not possible. As a result, we need to be able to show that the processes we use produce correct results. This is done using:
- Root Cause Analysis to identify and fix faulty processes
- Failure Modes and Effects Analysis (review of Design & Development processes with emphasis on possible types of failures) according to the perceived risks and consequences of failure in design and/or development, followed by appropriate preventive action
- Collection of history of failure modes and effects, and their analysis, followed by appropriate preventive action

in addition to requiring that software designers and developers be qualified to perform their functions."

I'd be interested in your comments.

[Al Rosen]

IEEE Definition
VALIDATION

"Confirmation by examination and provisions of objective evidence that the particular requirements for a specific intended use are fulfilled."

So, you are saying you're not capable of showing that the software is able to fulfill its intrended use. I'm not sure that that is acceptable.

[mmclean]

Originally Posted by Al Rosen

IEEE Definition
VALIDATION
So, you are saying you're not capable of showing that the software is able to fulfill its intrended use. I'm not sure that that is acceptable.

My thought was that in a complex application it is not possible to test all of the feasible combinations of usage in a real-world environment - covering both explicit and implied requirements - before shipping the product. That is, some deficiencies may only become apparent in use in circumstances that were not covered in our test suite.

[Jim Wynne]

Originally Posted by mmclean

My thought was that in a complex application it is not possible to test all of the feasible combinations of usage in a real-world environment - covering both explicit and implied requirements - before shipping the product. That is, some deficiencies may only become apparent in use in circumstances that were not covered in our test suite.

Your specifications should include specific reference to test conditions. You validate the efficacy of the software by verifying that it will do x (and y and z...) under conditions a and b and c...

Your customer should be made aware of the conditions under which the software has been validated. No one expects that there will never be any surprises, as it's not possible to predict all of the possible combinations of hardware, other software, and environmental conditions that your product might be exposed to.

[mmclean]

Originally Posted by Jim Wynne

Your specifications should include specific reference to test conditions. You validate the efficacy of the software by verifying that it will do x (and y and z...) under conditions a and b and c...

Your customer should be made aware of the conditions under which the software has been validated. No one expects that there will never be any surprises, as it's not possible to predict all of the possible combinations of hardware, other software, and environmental conditions that your product might be exposed to.

So are you saying that 7.5.2 does not apply in this case? And that process validation is a 'nice to have'?

[Jim Wynne]

Originally Posted by mmclean

So are you saying that 7.5.2 does not apply in this case?

7.5.2 (of ISO9k2k) says, in part,

Validation shall demonstrate the ability of these processes to achieve planned results

Which pretty much repeats the IEEE quote that Al Rosen gave you. This is a slightly different case, because 7.5.2 applies to "special" manufacturing processes. There's a difference between software validatation and process validation. What are you trying to find out?

Originally Posted by mmclean

And that process validation is a 'nice to have'?

No, it's a "necessary to have." I was simply stating the obvious; it's not always possible to anticipate every possible use of the product.

[Al Rosen]

Originally Posted by mmclean

So are you saying that 7.5.2 does not apply in this case? And that process validation is a 'nice to have'?

You stated that your company sells software. If you design that software, I believe that the validation of the software is covered under 7.3.6 Design and development validation, not 7.5.2 and it does apply. Yes, you might miss something, but you can't say that you can't do it because it is too complex. Your customer is expecting the software to perform the functions that it was intended to perform, so you must check that it does perform them under the defined conditions.

[mmclean]

Originally Posted by Al Rosen

You stated that your company sells software. If you design that software, I believe that the validation of the software is covered under 7.3.6 Design and development validation, not 7.5.2 and it does apply. Yes, you might miss something, but you can't say that you can't do it because it is too complex. Your customer is expecting the software to perform the functions that it was intended to perform, so you must check that it does perform them under the defined conditions.

Right - we do, of course, test our software (validation of the product), and the validation responds to 7.3.6.

However, the original reason I thought that we should say something under 7.5.2 is that complete validation is not always possible - actually, may never be practical.

My reasoning was this: If we manufactured widgets that were required to be 1" +/- 0.1" long, then we could test each widget and determine conclusively if it met its requirement. Then, because production output can be completely verified by subsequent measurement, there would be no need to invoke the provisions of 7.5.2. In the case of non-trivial software we cannot make a conclusive determination, and I thought that the 'gap' should perhaps be bridged by process validation.

If it is sufficient to make the customer aware of the conditions under which the software has been validated (and assuming, of course, that our testing is done to a reasonable level), then it does seem to me that we may not need to validate the process - that is, process validation becomes a 'nice to have'.

Is my logic OK, or am I still missing something here?