How to validate Microsoft Excel Spreadsheet Calculations for the Cpk value

Bruser

Starting to get Involved
#1
Good Afternoon Covers;
:frust:
I have a medical customer that my company has begun sampling product to. We are doing IQ,OQ,PQ for the products and as I submit I had the following question kicked back to me " How did you validate the Microsoft Excel spreadsheet calculations for the Cpk values" in my response that EXCEL calculations is a question for Bill Gates (not my actual response) I received the following-
There?s a special tool called ?CSV? Computer System Validation that is usually used to validate computer software. ?It provides documented proof that the system (e.g. hardware, software, peripherals and network) will repeatedly and reliably do what it is designed to do, is "fit-for-purpose", and complies with the applicable rules and regulations.?
Now I'm looking for a "CSV" to do the validation, so the engineering team can get the process validated. Any thoughts out there?
 
Elsmar Forum Sponsor

Mark Meer

Trusted Information Resource
#2
I am also interested in opinions on this topic...

It seems to me that many of these "CSV" tools or services may be unncessary in most cases (with the exception of perhaps highly complicated or customized systems).

Is it not sufficient just to state the intended use, then have a simple set of test-data to validate that your system performs as expected?

For example, if your version of MS Excel (and hardware/OS that runs it) is intended to calculate Cpk values, can you not just have a bunch of known test-data, feed it into your system, and confirm that it handles the data as expected?

Further to this discussion, does the system have to be revalidated every time there is:
1. an OS (e.g. Windows) patch/update?
2. an hardware change (e.g. I get a new computer)?

Seems to me that, in the case of Excel spreadsheets, practically speaking, (1) would be "maybe" and (2) would be "no"...
 

Steve Prevette

Deming Disciple
Staff member
Super Moderator
#3
Personally, I'd say you take a few example calculations and it's data and do it the OLD FASHIONED way with pencil and paper (and calculator) and see if you get the same answer as the Excel file gets. Record all calculations. You should use someone who did not develop the Excel formulae.
 

Steve Prevette

Deming Disciple
Staff member
Super Moderator
#4
Further to this discussion, does the system have to be revalidated every time there is:
1. an OS (e.g. Windows) patch/update?
2. an hardware change (e.g. I get a new computer)?

Seems to me that, in the case of Excel spreadsheets, practically speaking, (1) would be "maybe" and (2) would be "no"...
In my past experience, if this calculation was considered to be "safety significant" (or otherwise critical to the company's profitability) we would have a canned set of numbers that we "knew the answer to", and run them through the computer after significant software upgrades, or upon installation on a new computer.
 

Darius

Quite Involved in Discussions
#5
... used to validate computer software. provides documented proof that the system (e.g. hardware, software, peripherals and network) will repeatedly and reliably
How can a tool evaluate your calculations if you don't tell it wich values is the real one?. You said

. “It provides documented proof that the system (e.g. hardware, software, peripherals and network) will repeatedly and reliably do what it is designed to do, is "fit-for-purpose", and complies with the applicable rules and regulations.”
It looks like a tool that you feed the value of your system and the true value, and uses them to calculate the Uncertainty in Measurement (GUM). Of couse is not Excel reliablity...:bonk:, maybe your customer is telling you that is not sure of the calculations you are obtaining in Excel, Excel doesn't have a Cpk index calculation, but you can use it to make such calculus, did you check it against Minitab calculations or any other tool in the market? (at least ones).
 

Bev D

Heretical Statistician
Staff member
Super Moderator
#7
Good Afternoon Covers;
:frust:
I have a medical customer that my company has begun sampling product to. We are doing IQ,OQ,PQ for the products and as I submit I had the following question kicked back to me " How did you validate the Microsoft Excel spreadsheet calculations for the Cpk values"
First the FDA typically requires that all software that is involved in medical device function, patient results, treatment, release testing of material, etc. be validated and that you have a validation system. This includes EXCEL spreadsheets (notoriously frustrating).

Now a Cpk value hardly effects the patient directly so I'm a bit confused as to why your Customer is invoking the software validation? Unless of course you are using Cpk to get out of 100% testing in which case the formulas in EXCEL are the least of their worries...

Another thing to consider is that when data are submitted to the FDA, they look askance at anything coming from EXCEL. The statistical reviewers typically look for either SAS or R and the logs that show what the software did (real time validation). As an aside Minitab and JMP are also looked upon with disdain...) some products of course don't have high powered stats reviewers and results analyzed in EXCEL are acceptable, but typically there is some proof of validation of the formula used. (a simple data set with known results) OR the raw data is submitted and the reviewer redoes the analysis themselves.

in any case for Cpk I would expect a simple validation data set is all that is required.
 

Bruser

Starting to get Involved
#8
A little clarification, we are a packaging supplier to a medical device manufacturer. We make foam trays and foam pouches that implants slip into. These pouches are discarded prior to implant.

The Cpk values are for the width and length capability of the pouches and trays. This seems like overkill to validate the software that calculates the Cpk value for the measurement of the pouch / tray length & width.

The software doing the calculations is a purchased excel add on program.

Thanks all for the help, I'll be running the calculations through the program and comparing to the "Ford MSA " spreadsheet suggested above. Lets see what happens.
 

Bev D

Heretical Statistician
Staff member
Super Moderator
#9
yes that is overkill.
have you asked your Customer why they are requiring this validation?
It might be that the customer is just blindly applying the software or data validation requirement without thought...this may or may not be a conscious decision.
 

Steve Prevette

Deming Disciple
Staff member
Super Moderator
#10
A little clarification, we are a packaging supplier to a medical device manufacturer. We make foam trays and foam pouches that implants slip into. These pouches are discarded prior to implant.

The Cpk values are for the width and length capability of the pouches and trays. This seems like overkill to validate the software that calculates the Cpk value for the measurement of the pouch / tray length & width.

The software doing the calculations is a purchased excel add on program.

Thanks all for the help, still investigating..:frust:
I don't think any of us sitting here miles away from you can judge if doing the validation is "overkill" or not. Hopefully your facility already has Software QA standards that would list what software is "critical" and requires what for validation. But if you do decide you need to do the validation, you've got some ideas here.
UPDATE NOTE: I wrote this comment simultaneously with Bev's so don't mean to contradict her.
 
Thread starter Similar threads Forum Replies Date
Anonymous16-2 21 CFR Part 11 - Steps to take if we want to validate an electronic system Pharmaceuticals (21 CFR Part 210, 21 CFR Part 211 and related Regulations) 2
T How to “validate/qualify” a new measurement equipment? Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 4
R Validation of Medical Device Hardware containing Software - How many to Validate ISO 13485:2016 - Medical Device Quality Management Systems 1
S Do we need to validate Software used in Drug discovery and development process? Qualification and Validation (including 21 CFR Part 11) 2
kisxena Memo to File - Failure to Validate Supply Chain Security Management Systems 1
A ISO 13485:2016 - Validate our use of software that impacts on the QMS ISO 13485:2016 - Medical Device Quality Management Systems 8
A Process to Validate the Accuracy of Test Reports (AS9100) AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 38
C Do we need to Validate Off the Shelf Software? Manufacturing and Related Processes 5
M How to validate hundreds or even thousands of parts? Qualification and Validation (including 21 CFR Part 11) 7
M How to validate a system to make it compliant with 21 CFR Part 11 Qualification and Validation (including 21 CFR Part 11) 2
B Do I need to validate my database software? Calibration and Metrology Software and Hardware 1
K COTS Validation: Should we validate Windows after each Update? IEC 62304 - Medical Device Software Life Cycle Processes 3
S How to validate a homegrown software 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 5
A How to verify and validate the effectiveness of CAPA with regards to employees ? 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 5
A When to verify and when to validate Design Changes 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 4
H How To Validate Electronic Part From Non Authorized Distributor Process Maps, Process Mapping and Turtle Diagrams 7
G Prototype Manufacturing Equipment - Validate or not to validate? Quality Manager and Management Related Issues 2
SteveK How to validate Device Shelf Life? CE Marking (Conformité Européene) / CB Scheme 8
M Destructive VTMV Subgroups - A Destructive Variable Test Method to Validate Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 11
M How to validate a Dye Test method used to evaluate Package Integrity Pharmaceuticals (21 CFR Part 210, 21 CFR Part 211 and related Regulations) 8
I When to Validate and when not to Validate US Food and Drug Administration (FDA) 24
C How do you Validate Solderability of Components Prior to Soldering Operations Manufacturing and Related Processes 5
M Welding - to Validate or Not? If so, How? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 8
V Do we need to Validate a Server/Back-End System? Qualification and Validation (including 21 CFR Part 11) 2
I Using validated SAS JMP to validate Excel for Statistical Analysis? Qualification and Validation (including 21 CFR Part 11) 11
E How to Validate Master Equipment that has been brought to onsite General Measurement Device and Calibration Topics 12
E When to Validate a Process - 21 CFR Part 820.75.(a) 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 8
R How to bond PVC (Tygon) Tubing and Validate the Process ISO 13485:2016 - Medical Device Quality Management Systems 13
T Do I need to validate a basic Excel spreadsheet Quality Assurance and Compliance Software Tools and Solutions 10
R Process Validation - Adhesive Bonding, what to validate and how? Qualification and Validation (including 21 CFR Part 11) 10
N How to Validate a Gage R&R Spread Sheet General Measurement Device and Calibration Topics 3
A How to Validate Effectiveness of Mitigations? ISO 14971 - Medical Device Risk Management 13
C Validate the idea of a Composite Performance Index Quality Tools, Improvement and Analysis 4
M How to determine/validate the "REQUIRED MANPOWER" in an office environment? Service Industry Specific Topics 4
R How to Validate Vision Inspection Systems Inspection, Prints (Drawings), Testing, Sampling and Related Topics 4
C Third-party lab needed to validate cleaning & disinfection? EU Medical Device Regulations 12
Proud Liberal How to validate the geometry of a 8 mm pitch, 22 tooth Gates HTD sprocket Inspection, Prints (Drawings), Testing, Sampling and Related Topics 4
T How does a tool-shop validate molds? Verification vs. Validation ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 5
D Validate 99.5% reliability 90% confidence - Sample size calculation Reliability Analysis - Predictions, Testing and Standards 4
J Protocol to validate the use of Electronic Records like DHRs, NCMRs, Validations ISO 13485:2016 - Medical Device Quality Management Systems 3
N Periodically validate test reports for raw material - AS9100 7.4.3 AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 15
V Calibration of a Digital Vernier - Need to check/calibrate/validate all 3 functions? General Measurement Device and Calibration Topics 9
P Last-off-part comparison used to validate the set-up (TS16949 Clause 7.5.1.3) IATF 16949 - Automotive Quality Systems Standard 3
E How to validate product if gage is determined out of calibration Document Control Systems, Procedures, Forms and Templates 19
M Digital Filling System - What do I need to Validate Here? Software Quality Assurance 1
M Seat Cover Product Validation for PPAP - How can I validate my product? APQP and PPAP 12
N Do we need to Validate Outsouced Processes? ISO13485 ISO 13485:2016 - Medical Device Quality Management Systems 12
E GMW3172 - Thermal Shock and PTC cycles required to validate an electronic system Reliability Analysis - Predictions, Testing and Standards 2
A Software Observation during ISO 9001 Audit - Validate and Re-validate Software ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 13
R Validation of outsourced processes - Calibration - How do we validate that? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 12

Similar threads

Top Bottom