How to validate Microsoft Excel Spreadsheet Calculations for the Cpk value

[Bruser]

Good Afternoon Covers;

I have a medical customer that my company has begun sampling product to. We are doing IQ,OQ,PQ for the products and as I submit I had the following question kicked back to me " How did you validate the Microsoft Excel spreadsheet calculations for the Cpk values" in my response that EXCEL calculations is a question for Bill Gates (not my actual response) I received the following-
There?s a special tool called ?CSV? Computer System Validation that is usually used to validate computer software. ?It provides documented proof that the system (e.g. hardware, software, peripherals and network) will repeatedly and reliably do what it is designed to do, is "fit-for-purpose", and complies with the applicable rules and regulations.?
Now I'm looking for a "CSV" to do the validation, so the engineering team can get the process validated. Any thoughts out there?

 

[Mark Meer]

I am also interested in opinions on this topic...

It seems to me that many of these "CSV" tools or services may be unncessary in most cases (with the exception of perhaps highly complicated or customized systems).

Is it not sufficient just to state the intended use, then have a simple set of test-data to validate that your system performs as expected?

For example, if your version of MS Excel (and hardware/OS that runs it) is intended to calculate Cpk values, can you not just have a bunch of known test-data, feed it into your system, and confirm that it handles the data as expected?

Further to this discussion, does the system have to be revalidated every time there is:
1. an OS (e.g. Windows) patch/update?
2. an hardware change (e.g. I get a new computer)?

Seems to me that, in the case of Excel spreadsheets, practically speaking, (1) would be "maybe" and (2) would be "no"...

 

[Steve Prevette]

Personally, I'd say you take a few example calculations and it's data and do it the OLD FASHIONED way with pencil and paper (and calculator) and see if you get the same answer as the Excel file gets. Record all calculations. You should use someone who did not develop the Excel formulae.

 

[Steve Prevette]

Further to this discussion, does the system have to be revalidated every time there is:
1. an OS (e.g. Windows) patch/update?
2. an hardware change (e.g. I get a new computer)?

Seems to me that, in the case of Excel spreadsheets, practically speaking, (1) would be "maybe" and (2) would be "no"...

In my past experience, if this calculation was considered to be "safety significant" (or otherwise critical to the company's profitability) we would have a canned set of numbers that we "knew the answer to", and run them through the computer after significant software upgrades, or upon installation on a new computer.

 

[Darius]

... used to validate computer software. provides documented proof that the system (e.g. hardware, software, peripherals and network) will repeatedly and reliably

How can a tool evaluate your calculations if you don't tell it wich values is the real one?. You said

. “It provides documented proof that the system (e.g. hardware, software, peripherals and network) will repeatedly and reliably do what it is designed to do, is "fit-for-purpose", and complies with the applicable rules and regulations.”

It looks like a tool that you feed the value of your system and the true value, and uses them to calculate the Uncertainty in Measurement (GUM). Of couse is not Excel reliablity..., maybe your customer is telling you that is not sure of the calculations you are obtaining in Excel, Excel doesn't have a Cpk index calculation, but you can use it to make such calculus, did you check it against Minitab calculations or any other tool in the market? (at least ones).

 

[Miner]

See the first post in this thread for the Ford Validation data set. The second tab is for MSA and the third tab is for Cp/Cpk and Pp/Ppk calculations.

 

[Bev D]

Good Afternoon Covers;

I have a medical customer that my company has begun sampling product to. We are doing IQ,OQ,PQ for the products and as I submit I had the following question kicked back to me " How did you validate the Microsoft Excel spreadsheet calculations for the Cpk values"

First the FDA typically requires that all software that is involved in medical device function, patient results, treatment, release testing of material, etc. be validated and that you have a validation system. This includes EXCEL spreadsheets (notoriously frustrating).

Now a Cpk value hardly effects the patient directly so I'm a bit confused as to why your Customer is invoking the software validation? Unless of course you are using Cpk to get out of 100% testing in which case the formulas in EXCEL are the least of their worries...

Another thing to consider is that when data are submitted to the FDA, they look askance at anything coming from EXCEL. The statistical reviewers typically look for either SAS or R and the logs that show what the software did (real time validation). As an aside Minitab and JMP are also looked upon with disdain...) some products of course don't have high powered stats reviewers and results analyzed in EXCEL are acceptable, but typically there is some proof of validation of the formula used. (a simple data set with known results) OR the raw data is submitted and the reviewer redoes the analysis themselves.

in any case for Cpk I would expect a simple validation data set is all that is required.

 

[Bruser]

A little clarification, we are a packaging supplier to a medical device manufacturer. We make foam trays and foam pouches that implants slip into. These pouches are discarded prior to implant.

The Cpk values are for the width and length capability of the pouches and trays. This seems like overkill to validate the software that calculates the Cpk value for the measurement of the pouch / tray length & width.

The software doing the calculations is a purchased excel add on program.

Thanks all for the help, I'll be running the calculations through the program and comparing to the "Ford MSA " spreadsheet suggested above. Lets see what happens.

 

[Bev D]

yes that is overkill.
have you asked your Customer why they are requiring this validation?
It might be that the customer is just blindly applying the software or data validation requirement without thought...this may or may not be a conscious decision.

 

[Steve Prevette]

A little clarification, we are a packaging supplier to a medical device manufacturer. We make foam trays and foam pouches that implants slip into. These pouches are discarded prior to implant.

The Cpk values are for the width and length capability of the pouches and trays. This seems like overkill to validate the software that calculates the Cpk value for the measurement of the pouch / tray length & width.

The software doing the calculations is a purchased excel add on program.

Thanks all for the help, still investigating..

I don't think any of us sitting here miles away from you can judge if doing the validation is "overkill" or not. Hopefully your facility already has Software QA standards that would list what software is "critical" and requires what for validation. But if you do decide you need to do the validation, you've got some ideas here.
UPDATE NOTE: I wrote this comment simultaneously with Bev's so don't mean to contradict her.