Calculating Ppk for Design Verification - Variable Sampling

ga2qa23

Involved In Discussions
#1
Hi all, I sure could please use some advice on how to perform variable sampling for design verification of a product. Too be honest, I'm not even sure how you'd do it, because you need to calculate your Ppk but design verification should occur BEFORE you start ramping up significant numbers of product to use for test samples.

If I manufactured a batch of 20 products and I had performed a 100% measurement/verification check (of continuous/variable data) so that I don't need to validate the manufacturing process, but I have to put all 20 products through a shipping test and confirm they still work with a variable sampling plan (for design verification), then can I calculate my Product Performance Index (Ppk) using data from the earlier 100% check from my manufacturing process?


To give some more details (if you're interested in my tale)...

My product is a medical device, but this question should still apply to non-medical device products. Basically, my product is super expensive and my company will only be making 20 of them to start with. We're doing a 100% verification check after assembly, so thus the manufacturing process will not be validated. However, one of our product requirements is that the finished product must survive shipping in our package and remain functional. Our product uses a gyroscope sensor to detect angles. So our functional test (after shipping) is to confirm that the product can detect angles within acceptable parameters of accuracy.

The only hang-up is that because this design verification activity is a true "test", not just a simple inspection/demonstration/analysis. Thus, the both the US FDA and European Union will expect us to have a statistical approach for this test (per the standard ISO 13485 - basically required by EU and FDA is currently transitioning to use the standard as well).

Sounds great, right? My test is measuring the angles that my product measures, which is continuous (variable) data. So I'd use a variable sampling plan, which allows for lower sample size, and I already know my specification limit. But I can't calculate a Product Performance Index (Ppk) without previous measurements! I have no historical data from similar products and a small population because the cost is so high. Can I just take the data from my earlier 100% check of the manufacturing process, and use that to calculate the Ppk in this new shipping test?
 
Elsmar Forum Sponsor
#2
I have not heard of that. Design verification sample size is based on risk. If your product is low risk, using 20 samples may be sufficient. Typically, design verification testing is done prior to design transfer. The manufacturing process may not be set and certainly is not required to be under any kind of statistical control. The only exception to this is a process that itself affects the design and where the output is unable to be verified.
 

ga2qa23

Involved In Discussions
#3
Thanks for responding! Our product is technically Class 2 but on the lower end. In my understanding, design transfer isn't set in stone and can start even before the design verification stage. This is because product can be designed/redesigned for manufacturing. Please let me know if I'm mistaken.

This isn't just a feature demonstration. In this situation, I'm treating the shipping/transit validation as a "Design Verification" activity. I also thought that "Design Verification" performed by testing requires a statistical approach/rationale. I'm working off of clauses in ISO 13485 and the book "Statistical Procedures for the Medical Device Industry" by Dr. Wayne A. Taylor.

For the shipping test, the prototype parts we'll ship in the test are high-fidelity (near-final) parts that are manufactured by the same vendor we chose for final product manufacturing. After shipping, we must ensure that the sensor can measure within X degrees of accuracy, and thus it's a "Design Verification" test that requires a sampling plan.
 

Jim Wynne

Leader
Admin
#4
I don't see any way that Ppk (a dubious statistic to begin with) will help you. Ppk is a process metric, not product. I can't speak to the requirements for medical devices, but I think you're barking up the wrong tree with using a process capability approach.
 

ga2qa23

Involved In Discussions
#5
Doesn't Design Verification by test require a statistical approach/rationale, whether it's medical devices or the automotive industry?

I just chose a variable sampling plan because we're measuring how accurate a sensor is. Should I instead just use a t-test?

Both options assume a normal distribution of data. So I assumed that I'd still need to measure the data for all 20 units pre-shipping test, to confirm that the data is normal.
 

Bev D

Heretical Statistician
Leader
Super Moderator
#6
Shipping tests do not typically require a statistical sample as they are usually what are called ‘directed’ tests. The standard approach is to submit the product to worst case temperature and stress conditions. Measure before and after and look for degradation or loss of accuracy.

Will you be submitting the verification data to your regulatory agency for review or will you be just filing the report to be bailable for review upon request?
 

ga2qa23

Involved In Discussions
#7
That is very interesting. Is the term "directed test" generally used outside of the semiconductor industry? I'm hesitant to apply that logic to medical devices. I'm not asking you for the answer, I'm just wondering if you've heard that term used in other industries.

I understood the aspect of worst-case / stress conditions testing. I was instead applying that logic to reduce my initial sample size. For example, I started with a sample size of 15 for variable sampling, and then reduced it by a factor of 5 to get a sample size of 3. This is because the shipping testing was determined to be a five-times stress test.

Our report would be filed away, for review upon request. For a medical device, you just tell the FDA in the 510(k) submission that you validated the shipping per a standard but don't give details. However, FDA still is supposed to audit you every 2 years afterward and has full access to your reports, and could potentially write up a "483 Report" on you if they find issue with it. Your failure can then become public knowledge through a FOIA (Freedom of Information Act) Request.
 

Bev D

Heretical Statistician
Leader
Super Moderator
#8
I used to work in the medical device industry (Veterinary but based on FDA requirements). Not sure if directed testing is used in all industries but it is the basis of most of these tests that are required and covered in many many standards. and in many of these the sample size is one.

by the way it isn't valid to reduce your sample size by 5 because the stress is 5X....also you shouldn't use arbitrary stresses. understand what can actually happen in the worst case and use those stresses...that is what you have to justify to the auditors not the sample size.

Sample sizes are used when you don't know what the conditions are or how they vary. So you come up with a statistical sample size (based on confidence and reliability) and then ship those parts to diverse locals under diverse seasons...remember a statistical sample is only valid if it is randomly sampling the entire population of possible conditions.
 

ga2qa23

Involved In Discussions
#9
Thank you, I really appreciate your input. My only hangup is that if sample size for ship/transit testing should be one, then what about high-volume / low cost medical devices? Those manufacturers make millions or billions of "units" (medical device products) each year. So then they conduct testing, and they have a "shipping configuration" of say 30 units in a single box, then do they only strictly need to do a shipping test on one box? I find that very interesting.
 
#10
It all depends on risk. A sample size of one is not typical in my experience.

What are the risks associated with a failed product? If the risk to the user/patient is minimal, you may be comfortable with a lower sample size. You can also do a modified shipper with dummy devices in the low risk locations. Using your example of 30 units, a manufacturer could use multiple shippers that only have real devices in the corners and bottom, with equivalent weight in all other locations. There is no one-size-fits-all approach to this. You need to figure out what your risks are and what makes sense for your organization.
 
Thread starter Similar threads Forum Replies Date
S Is a Stable Process (within Control Limits) required for Calculating Pp, Ppk? Capability, Accuracy and Stability - Processes, Machines, etc. 6
J Need help calculating actual Ppk - Result of % GR&R=11.8% when conducting MSA Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 11
D Use of an outlier in calculating Cpk/Ppk Capability, Accuracy and Stability - Processes, Machines, etc. 8
S Calculating Cpk/Ppk Simultaneously in SPC software Capability, Accuracy and Stability - Processes, Machines, etc. 6
A Calculating Risk Estimation ISO 14971 - Medical Device Risk Management 28
S Help needed in choosing the method of calculating the minimum sample size Internal Auditing 12
M True position, calculating theoretical distances General Measurement Device and Calibration Topics 0
M Lean: Conducting Capacity Study, calculating Cycle times on laser cutting machines Lean in Manufacturing and Service Industries 2
A Calculating Carbon Footprint in the Organization Miscellaneous Environmental Standards and EMS Related Discussions 2
C Calculating your KPIs ISO 13485:2016 - Medical Device Quality Management Systems 7
K Calculating Heat Dissipation Manufacturing and Related Processes 1
Q Calculating Defect Rates ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 4
S Formula for Calculating NoGo Major Diameter for UN gages Inspection, Prints (Drawings), Testing, Sampling and Related Topics 4
P Calculating a weight for a machine for the CE label CE Marking (Conformité Européene) / CB Scheme 0
R Remote Support - Calculating Number of Employees IATF 16949 - Automotive Quality Systems Standard 2
G Calculating Tolerance of a Panel Meter with a 2 degrees of an Arc General Measurement Device and Calibration Topics 0
T Calculating LCL, UCL, Cp, and Cpk in an Excel Spreadsheet Statistical Analysis Tools, Techniques and SPC 3
V Product Development - When to start calculating Process Capability Capability, Accuracy and Stability - Processes, Machines, etc. 3
J Calculating Reliability for Subsystems in Series Reliability Analysis - Predictions, Testing and Standards 15
Stijloor Calculating (3rd Party) Audit Days for Company with Seasonal Employees General Auditing Discussions 3
T Formulas for Calculating Coefficients for RTD / PRTD Calibration According to ITS-90 General Measurement Device and Calibration Topics 5
J Calculating part variation from historic data for GRR study Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 1
P Purpose of calculating Ta, Tb & R Square in Linearity Study Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 1
P Purpose of calculating Uncertainty value in calibration study Measurement Uncertainty (MU) 5
M Calculating MTBF for plug-in PCI cards Reliability Analysis - Predictions, Testing and Standards 2
J Calculating impact force, expressed in lbs, of a 5 lb weight dropped 48 inches Inspection, Prints (Drawings), Testing, Sampling and Related Topics 1
M Calculating Cpk when sample size equals to 1 Capability, Accuracy and Stability - Processes, Machines, etc. 12
D Calculating Cpk on Non-Normal Data Distribution Capability, Accuracy and Stability - Processes, Machines, etc. 10
K Calculating Capability of a process when data is skewed in the upper range Capability, Accuracy and Stability - Processes, Machines, etc. 4
A Calculating Accuracy for a Multimeter General Measurement Device and Calibration Topics 1
M Calculating Coefficients for an RTD (Resistance Temperature Detector) Probe General Measurement Device and Calibration Topics 3
T Calculating Plating/Coating Weight for IMDS RoHS, REACH, ELV, IMDS and Restricted Substances 3
D Calculating Lower and Upper 3 Sigma Control Limits Statistical Analysis Tools, Techniques and SPC 2
M Calculating Adequate Receiving Inspection Sample Size Statistical Analysis Tools, Techniques and SPC 2
M Calculating Capability of Delivery Performance Capability, Accuracy and Stability - Processes, Machines, etc. 5
K Advice on Calculating Control Chart Control Limits Statistical Analysis Tools, Techniques and SPC 13
B Calculating Combined DPMO and Sigma Level for Two or More Different Work Areas Six Sigma 3
G Calculating the Fraction Defective Confidence Interval for a Lot Statistical Analysis Tools, Techniques and SPC 2
A Calculating Combined Measurement Uncertainty - VDA 5 Measurement Uncertainty (MU) 1
B Calculating Precision to Tolerance Ratio Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 9
D DPPM - Calculating the Defect Rate of a Software Test Statistical Analysis Tools, Techniques and SPC 9
L FMEA - Calculating RPN = S*O*D FMEA and Control Plans 3
I Shewhart Constants vs Central Limit Theorem in calculating Control Limits Statistical Analysis Tools, Techniques and SPC 18
B Please share a template for calculating Cp Cpk Document Control Systems, Procedures, Forms and Templates 3
M Calculating the IATF TS 16949 Recertification Audit Man Days - Clarification ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 6
J IEC60825 - Laser Safety - Calculating Maximum Permissible Exposure Correctly Other ISO and International Standards and European Regulations 1
P Calculating Process Capability from Data of attached spreadsheet Statistical Analysis Tools, Techniques and SPC 4
I AQL Formula in Excel - Calculating Sample Size based on an AQL Chart AQL - Acceptable Quality Level 8
D Formula for Calculating Warp Measurement of Corrugated Boxes General Measurement Device and Calibration Topics 1
S Need Help Calculating and Implementing Cs - Wright's Index Statistical Analysis Tools, Techniques and SPC 11

Similar threads

Top Bottom