Calculating your KPIs

#1
Dears,

in our recertification audit we got the following deviation concerning chapter 8.4:
" The current procedure for KPIs does not state any methods, statistical techniques nor the extent of their use."
This is correct, we only use ratios, but:
- where can I find the requirement to use statistical methods
- what KPIs do you use (CpK and Six Sigma are too big for our company)

Any help is highly appreciated!
 
Elsmar Forum Sponsor

Jen Kirley

Quality and Auditing Expert
Staff member
Admin
#3
Dears,

in our recertification audit we got the following deviation concerning chapter 8.4:
" The current procedure for KPIs does not state any methods, statistical techniques nor the extent of their use."
This is correct, we only use ratios, but:
- where can I find the requirement to use statistical methods
- what KPIs do you use (CpK and Six Sigma are too big for our company)

Any help is highly appreciated!
Welcome to The Cove!

8.4 of ISO 13485:2016 includes the words "including statistical techniques" but does not say "The organization shall use SPC and process capability statistical techniques."

It is important to select the proper type of data analysis method(s) and ensure correct use. If you choose to use statistics for some applications (they certainly are not the best for every data analysis need), there are many. The course outline Introduction to Basic Statistical Methods lists a number of statistical methods. The course document has hyperlinks to individual lesson documents that look very good to me.

I hope this helps!
 
#4
Sounds questionable, but you don't tell us what the "current procedure" says. Also, you should post the entirety of the NC statement.
The complete NC says above plus "Therefore, it could not be assured that the procedure leads to appropriate data to demonstratw the basis for the evaluation of the QMS system.

The procedure is our SOP that defines our KPIs as ratios, e.g. nonconforming parts/purchased parts.
 

Tagin

Trusted Information Resource
#5
The complete NC says above plus "Therefore, it could not be assured that the procedure leads to appropriate data to demonstratw the basis for the evaluation of the QMS system.

The procedure is our SOP that defines our KPIs as ratios, e.g. nonconforming parts/purchased parts.
Nothing wrong with using a ratio as a KPI.

Do you indicate anywhere a threshold value or other method by which this KPI is used to determine if the system is functioning well or not? (E.g., "if the ratio of NC/purchased is greater than 2%, then the following action will be taken.....")
 

levatorsuperioris

Involved In Discussions
#6
It sounds like it could be a questionable finding.

Statistical techniques can be as simple as ratios but the ratios must be defined. CpK, DPM, CPM are all defined ratios that are acceptable. But you have to be able document how the ratio is calculated. For example next years complaint rate is a 15% reduction in Complaints per Million, CPM - then define CPM as number of complaints / total number of products shipped (Last month 1 year ago to last month) or (2019 total shipped units) or (2020 total shipped units thus far) its up to you to determine which number to use depending on your product shelf life, sales volume increasing , decreasing etc .... (not suggesting CPM is a good metric, most regulatory bodies do not like it since it is in the public health interest to increase the numerator in reporting.)

As you can see here you can end up defining CPM very differently if you change how you define the denominator. So a pure ratio for example could be non conforming if the method to determine the numerator or denominator is not defined. At a global company with multiple sites the QMS would need to do this or every site may have a varying definition of the numerator and denominator for purposes of reporting.
 

Steve Prevette

Deming Disciple
Staff member
Super Moderator
#7
If you need help with having a procedure for metrics and statistical analysis, I have one posted here https://elsmar.com/Cove_Members/Steve's stuff/Generating_and_Using_Control_Charts.pdf

And there are several other files related to choice of KPI's in the library Index of /Cove_Members/Steve's stuff Generally I do find ratios to be unreliable as they are non-linear, such a 1 A and 4 B's is a ratio of 1/4 but 4 B's to 1 A is 4.0. Much better to trend of my A's and B's, I have 20% A's and 80% B's.

I will say there are several recommendations to use statistical methods out there -as pointed out by Jen. Some customers (such as Ford did for some time) require the use of Statistical Process Control.
 

yodon

Staff member
Super Moderator
#8
I believe @Tagin hit the nail on the head. They didn't say you had to use statistical techniques. Use what works for you but define how you collect, how you assess the data, and when you'll take actions. The approach and thresholds should all be risk-driven. For example, conformity to product requirements is probably something you're measuring. You probably want to create different buckets: those NCs that could result in risk and those that may not. Your thresholds for taking actions on NCs that could pose risk would be lower than those that don't.

While there's, no doubt, a lot of bogus audit findings, I tend to try to give the auditor the benefit of the doubt first. If you don't have your approaches defined, it could well be a valid finding.
 
Thread starter Similar threads Forum Replies Date
M True position, calculating theoretical distances General Measurement Device and Calibration Topics 0
M Lean: Conducting Capacity Study, calculating Cycle times on laser cutting machines Lean in Manufacturing and Service Industries 2
A Calculating Carbon Footprint in the Organization Miscellaneous Environmental Standards and EMS Related Discussions 2
K Calculating Heat Dissipation Manufacturing and Related Processes 1
Q Calculating Defect Rates ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 4
S Formula for Calculating NoGo Major Diameter for UN gages Inspection, Prints (Drawings), Testing, Sampling and Related Topics 4
P Calculating a weight for a machine for the CE label CE Marking (Conformité Européene) / CB Scheme 0
R Remote Support - Calculating Number of Employees IATF 16949 - Automotive Quality Systems Standard 2
G Calculating Tolerance of a Panel Meter with a 2 degrees of an Arc General Measurement Device and Calibration Topics 0
T Calculating LCL, UCL, Cp, and Cpk in an Excel Spreadsheet Statistical Analysis Tools, Techniques and SPC 3
V Product Development - When to start calculating Process Capability Capability, Accuracy and Stability - Processes, Machines, etc. 3
J Calculating Reliability for Subsystems in Series Reliability Analysis - Predictions, Testing and Standards 15
Stijloor Calculating (3rd Party) Audit Days for Company with Seasonal Employees General Auditing Discussions 3
T Formulas for Calculating Coefficients for RTD / PRTD Calibration According to ITS-90 General Measurement Device and Calibration Topics 5
J Calculating part variation from historic data for GRR study Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 1
P Purpose of calculating Ta, Tb & R Square in Linearity Study Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 1
P Purpose of calculating Uncertainty value in calibration study Measurement Uncertainty (MU) 5
M Calculating MTBF for plug-in PCI cards Reliability Analysis - Predictions, Testing and Standards 2
J Calculating impact force, expressed in lbs, of a 5 lb weight dropped 48 inches Inspection, Prints (Drawings), Testing, Sampling and Related Topics 1
M Calculating Cpk when sample size equals to 1 Capability, Accuracy and Stability - Processes, Machines, etc. 12
D Calculating Cpk on Non-Normal Data Distribution Capability, Accuracy and Stability - Processes, Machines, etc. 10
K Calculating Capability of a process when data is skewed in the upper range Capability, Accuracy and Stability - Processes, Machines, etc. 4
A Calculating Accuracy for a Multimeter General Measurement Device and Calibration Topics 1
M Calculating Coefficients for an RTD (Resistance Temperature Detector) Probe General Measurement Device and Calibration Topics 3
T Calculating Plating/Coating Weight for IMDS RoHS, REACH, ELV, IMDS and Restricted Substances 3
D Calculating Lower and Upper 3 Sigma Control Limits Statistical Analysis Tools, Techniques and SPC 2
M Calculating Adequate Receiving Inspection Sample Size Statistical Analysis Tools, Techniques and SPC 2
M Calculating Capability of Delivery Performance Capability, Accuracy and Stability - Processes, Machines, etc. 5
K Advice on Calculating Control Chart Control Limits Statistical Analysis Tools, Techniques and SPC 13
B Calculating Combined DPMO and Sigma Level for Two or More Different Work Areas Six Sigma 3
G Calculating the Fraction Defective Confidence Interval for a Lot Statistical Analysis Tools, Techniques and SPC 2
A Calculating Combined Measurement Uncertainty - VDA 5 Measurement Uncertainty (MU) 1
B Calculating Precision to Tolerance Ratio Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 9
D DPPM - Calculating the Defect Rate of a Software Test Statistical Analysis Tools, Techniques and SPC 9
L FMEA - Calculating RPN = S*O*D FMEA and Control Plans 3
I Shewhart Constants vs Central Limit Theorem in calculating Control Limits Statistical Analysis Tools, Techniques and SPC 18
B Please share a template for calculating Cp Cpk Document Control Systems, Procedures, Forms and Templates 3
S Is a Stable Process (within Control Limits) required for Calculating Pp, Ppk? Capability, Accuracy and Stability - Processes, Machines, etc. 6
M Calculating the IATF TS 16949 Recertification Audit Man Days - Clarification ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 6
J IEC60825 - Laser Safety - Calculating Maximum Permissible Exposure Correctly Other ISO and International Standards and European Regulations 1
P Calculating Process Capability from Data of attached spreadsheet Statistical Analysis Tools, Techniques and SPC 4
I AQL Formula in Excel - Calculating Sample Size based on an AQL Chart AQL - Acceptable Quality Level 8
D Formula for Calculating Warp Measurement of Corrugated Boxes General Measurement Device and Calibration Topics 1
S Need Help Calculating and Implementing Cs - Wright's Index Statistical Analysis Tools, Techniques and SPC 11
S Calculating an Uncertainty Budget for an Optical Comparator General Measurement Device and Calibration Topics 4
D One-Two-Three Sigma Control Limits - Calculating Control Limits for X-bar Charts Statistical Analysis Tools, Techniques and SPC 3
K Calculating (Approximate) Parallelism Inspection, Prints (Drawings), Testing, Sampling and Related Topics 10
S Calculating MTBF (Mean Time Between Failures) for the whole Production Unit Reliability Analysis - Predictions, Testing and Standards 2
J Need help calculating actual Ppk - Result of % GR&R=11.8% when conducting MSA Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 11
M Calculating Cp and Cpk - Is my calculation correct? Capability, Accuracy and Stability - Processes, Machines, etc. 7

Similar threads

Top Bottom