PPM Defect Rate as a Performance Metric - Are there Flaws?

leftoverture

Involved In Discussions
#1
Our company, as with many others, has historically tracked our quality performance largely by tracking our PPM (parts per million). I see PPM as a flawed metric, though, for two primary reasons:

1) It is very dependent on how many parts you actually produce and/or ship. So if you have a low shipping month, for instance, a relatively small rejection can result in an unusual spike in the metric.

2) It is not cost-related. As a job shop, we have many high-volume, low cost products that tend to influence the metrics more than the lower volume, high cost products.

Coming up with a better metric probably isn't that hard, but measuring PPM is pretty much expected by our customers and can be a hard sell to those for whom PPM "is they way we've always done it".

I would be interested in knowing if anyone else has found the same flaws in the PPM metric and how have you approached it?

Thanks!
 
Elsmar Forum Sponsor

hogheavenfarm

Quite Involved in Discussions
#2
This problem plagues other metrics too. Some of our customers require ppm, others do not, but regardless what measure you utilize, there will be things that skew it. When I was in extrusion, we had a better handle on this as the machines ran 24/7 at a specific rate, the ppm here was actually pretty valid. Now we manufacture to print, maybe one item, maybe 30, maybe 1000. This is where skews become evident. When possible, use what best reflects your particular reality, when required to show ppm, I add a disclaimer, "*ppm may not reflect historical data from this product line". This always generates questions which show that a particular product line may have very good defect numbers when measured against itself. Of course, you had better be prepared and have historical data handy (and better than the ppm). In the opposite case I leave well enough alone.
 

Sebastian

Trusted Information Resource
#3
I see PPM as a flawed metric, though, for two primary reasons:
1) It is very dependent on how many parts you actually produce and/or ship. So if you have a low shipping month, for instance, a relatively small rejection can result in an unusual spike in the metric.
"Dependent" sounds negative here. Why? Counting rejected pieces irrespectively of production volume is a wrong idea. Comparing reject value to production volume, as PPM does, allows comparison between production lots. It is fair. PPM bases on assumption that process has some nonconforming product ratio (process capability), which results in increase of rejected quantity proportional to production quantity increase.

2) It is not cost-related. As a job shop, we have many high-volume, low cost products that tend to influence the metrics more than the lower volume, high cost products.
In previous company I worked for, I had started from PPM metric and then moved to cost, as 10 rejected pieces of Product A resulting in 1234 PPM has higher cost than 1200 rejected pieces of Product B resulting in 5678 PPM, who would be normally No.1 target for improvement.
Reject cost is better, but PPM is not that bad.
 

Bev D

Heretical Statistician
Staff member
Super Moderator
#4
There isn't really any problem with a direct ppm calculation*. Ppm is just like percentages except the normalized base is a million instead of a hundred. The issue most people encounter comes from the non-homogenous distribution of defects in the process stream. There are 2 types of defect distributions: one is a random, homogenous distribution of defects (such as from a common cause system) and the other is a clustering of defects from an assignable cause such as a set up error or a defective batch of raw material. The latter creates a spike in defects. If your denominator for ppm comes from a single lot you will show a fairly large spike in ppm. Which is completely accurate for that short period of time but is not representative of long term defect rates IF you correct the assignable causes AND put permanent preventive solutions in place.

An additional issue is that Customers and Suppliers have difficulty separating manufactured defect rates and shipped defect rates. Both parties really need to understand the various effects of these two different things.

As for cost remember that a even an inexpensive part (for you) can be an expensive failure for your customer...


*There are major issues with a theoretical ppm calculation based on a distributional model such as with Cpk indices - these are usually just bogus made-up numbers....
 

leftoverture

Involved In Discussions
#5
"Dependent" sounds negative here. Why? Counting rejected pieces irrespectively of production volume is a wrong idea. Comparing reject value to production volume, as PPM does, allows comparison between production lots. It is fair.
A few reasons it is negative. 1) we are a job shop and make many different parts for many different customers. If I look at it for just one part number, it is a good measure of our defect rate, but across many different part numbers it starts to get skewed and loses it's significance.

2) Our quality incentive is based on our return rate. Ours is very, very low compared to most companies, but still, a return of 500 pieces could have varying impact on our incentive depending on how many parts we shipped that month.

3) We do monitor our company wide PPM, both internal and external, and we track trends, but if customer A orders part B which has a very high volume, it will cause a short term skew in the data. And if customer A is slow and doesn't order part B, the PPM will likewise artificially go up.

So, in the end, it is an imperfect measure. We understand the pitfalls, and try to interpret the data accordingly, but I still would like to find a better way (since continual improvement is practically our middle name here!). So I was just wondering if others have found a better way of measuring defect rates.
 

Sebastian

Trusted Information Resource
#6
It seems to me, that nevertheless various products are coming from a same manufacturing process, for whom one common PPM value is calculated, there are dramatic differences between products individual PPM values.
Normally, your manufacturing process design process activities shall build process without significant differences, but somehow it failed in some cases. Additionally probably during designed process validation there was not noticed performance drop for some products and you put them all in one bag and calculated common PPM. Maybe someone had noticed that, but it did not change anything anyway.
I would suggest for "bad boys" identified during process validation, setting individual PPM targets and then monitoring them separately during serial production.
 
Last edited:

hogheavenfarm

Quite Involved in Discussions
#7
But that is the issue, the products do not come from the same manufacturing process. When your product line contains everything from delicate electronic devices to armor plate, there are vastly different processes involved. We have one product with a 30% fallout rate. There is little we can do to improve it, (and I always try every time we run it), since the product is customer material, cut to customer prints (which we are not allowed to even see), and given a customer supplied check spec. It either falls within tolerance or not. The customer pays for all the material as well. When we run this, our internal PPM goes off the chart, and sometimes borderline cases like this escape, causing our external PPM to spike as well. Other products can be run by the thousands with virtually no defects at all. Yes , the PPM is an accurate average overall, but is not useful for determining capability of a particular process or product line. I cringe when customers simply want to see our "PPM defect rate" to determine if we are "good enough" to do business with.
 

leftoverture

Involved In Discussions
#8
But that is the issue, the products do not come from the same manufacturing process. When your product line contains everything from delicate electronic devices to armor plate, there are vastly different processes involved. We have one product with a 30% fallout rate. There is little we can do to improve it, (and I always try every time we run it), since the product is customer material, cut to customer prints (which we are not allowed to even see), and given a customer supplied check spec. It either falls within tolerance or not. The customer pays for all the material as well. When we run this, our internal PPM goes off the chart, and sometimes borderline cases like this escape, causing our external PPM to spike as well. Other products can be run by the thousands with virtually no defects at all. Yes , the PPM is an accurate average overall, but is not useful for determining capability of a particular process or product line. I cringe when customers simply want to see our "PPM defect rate" to determine if we are "good enough" to do business with.
Exactly! Glad to see I am not the only one who has this battle.

Sent from my LG-TP260 using Tapatalk
 

Golfman25

Trusted Information Resource
#9
A few reasons it is negative. 1) we are a job shop and make many different parts for many different customers. If I look at it for just one part number, it is a good measure of our defect rate, but across many different part numbers it starts to get skewed and loses it's significance.

2) Our quality incentive is based on our return rate. Ours is very, very low compared to most companies, but still, a return of 500 pieces could have varying impact on our incentive depending on how many parts we shipped that month.

3) We do monitor our company wide PPM, both internal and external, and we track trends, but if customer A orders part B which has a very high volume, it will cause a short term skew in the data. And if customer A is slow and doesn't order part B, the PPM will likewise artificially go up.

So, in the end, it is an imperfect measure. We understand the pitfalls, and try to interpret the data accordingly, but I still would like to find a better way (since continual improvement is practically our middle name here!). So I was just wondering if others have found a better way of measuring defect rates.
Since you're a job shop and have spikes and variation, you might try using a rolling average. That will help smooth everything out.
 

hogheavenfarm

Quite Involved in Discussions
#10
We do this internally (rolling avg), but unfortunately most customers want us to report PPM for the current year to date. Its hard to make them understand that there is a difference between product lines. I may start separating the lines into categories, and then just report PPM on those lines. We'll see. Currently PPM is 3,900 , but can take a big tumble if certain product lines come up.
 
Thread starter Similar threads Forum Replies Date
S Six sigma and defect rate (6 sigma = 3.4 PPM) Six Sigma 6
R What's the difference between using PPM or Defect Rate in a process? Quality Tools, Improvement and Analysis 1
Gman2 Automotive PPM Requirements - Our customer is trying to demand a 100% 0 defect rate IATF 16949 - Automotive Quality Systems Standard 3
Marc Calculating Defect Rate (PPM) from a Cpk value Capability, Accuracy and Stability - Processes, Machines, etc. 8
L Typical defect ppm level for barrel surface coating process? Supplier Quality Assurance and other Supplier Issues 1
J Calculating PPM for a Supplier on a Specific Defect - Excel .xls Spreadsheet Wanted Statistical Analysis Tools, Techniques and SPC 7
S DPM (Defect parts per million opportunity) vs. PPM Statistical Analysis Tools, Techniques and SPC 3
M Targets / limits for PPM/ Scrap Manufacturing and Related Processes 2
M Help me improve the definition for internal PPM Manufacturing and Related Processes 3
A ISO 2859-1 - 9.3.3.2 Switching score - Sampling Plans and PPM AQL - Acceptable Quality Level 0
S Software for Supplier Charge back and internal PPM General Information Resources 2
L PPM - wrong parts or defective parts? IATF 16949 - Automotive Quality Systems Standard 18
0 PPM calculation of customer complaints for PMS Customer Complaints 10
F Determining what type of scrap to include in my internal PPM calculation Quality Manager and Management Related Issues 5
R PPM, Pictorial Process Mapping, Templates Manufacturing and Related Processes 5
V PPM - Internal PPM / External PPM Quality Manager and Management Related Issues 4
N Converting ETO Residual Limits from PPM to mg ISO 13485:2016 - Medical Device Quality Management Systems 6
J Internal PPM's in a medium sized metal stamping plant Quality Manager and Management Related Issues 6
S What is appropriate PPM level for a Plastic Injection Moulding Supplier Quality Tools, Improvement and Analysis 4
Q PPM - Filtering Defects out of calculation Quality Tools, Improvement and Analysis 5
M Is PPM a good metric for gathering companywide complaints ? Customer Complaints 2
L Oil Field Bearing Production PPM Standard Benchmarking 2
V How is PPM (Parts Per Million) Calculated? Inspection, Prints (Drawings), Testing, Sampling and Related Topics 4
L Help with concept of PPM in SPC Statistical Analysis Tools, Techniques and SPC 6
Z Predicting External PPM Capability from Internal PPM Capability, Accuracy and Stability - Processes, Machines, etc. 1
S Cell Rubber Stamping PPM for Scrap Analysis Quality Tools, Improvement and Analysis 1
R Supplier Deviations - Should this be counted in the supplier ppm? Supplier Quality Assurance and other Supplier Issues 3
S How to calculate Supplier PPM at Receiving Inspection Document Control Systems, Procedures, Forms and Templates 2
Z Is There a Benchmark for PPM for Medical Devices? Benchmarking 4
K Proper YTD PPM Supplier Calculation Six Sigma 3
M California Proposition 65 Compliant (no more than 600 ppm touch area) IEC 60601 - Medical Electrical Equipment Safety Standards Series 5
M What is an acceptable PPM for large volume fasteners? APQP and PPAP 6
G Sample Size for 2900 ppm Visual Defects Inspection Requirement Inspection, Prints (Drawings), Testing, Sampling and Related Topics 4
V Calculation of PPM & Sigma Level in an Excel Spreadsheet Statistical Analysis Tools, Techniques and SPC 3
D Suitability of Attribute Control Charts for ppm Level Control Statistical Analysis Tools, Techniques and SPC 6
M Start Up Scrap vs. ppm (Parts per Million) in an FMEA IATF 16949 - Automotive Quality Systems Standard 17
H PPM Prediction for a New Automotive Electronic Product Reliability Analysis - Predictions, Testing and Standards 7
I Defectives (ppm) coming from Visual Inspection Inspection, Prints (Drawings), Testing, Sampling and Related Topics 1
G How do I calculate PPM from a Rolled Wire Suppllier Supplier Quality Assurance and other Supplier Issues 3
C Comparing PPM Year over Year - Determining the PPM decrease from 2009 to 2010 Benchmarking 10
5 Calculating PPM - Multiple product lines and multiple type of parts Six Sigma 2
B Automotive Supplier Benchmarking - Supplier PPM Data Benchmarking 5
O Estimating the dpm and ppm of the process from Cp and Cpk Statistical Analysis Tools, Techniques and SPC 4
S "Acceptable" PPM Level for Aluminum Die Casting Benchmarking 3
L Quality PPM (parts per million) Tracking - Stats for our Management Review Meeting Benchmarking 8
A What is the appropriate PPM Level for Sheet Metal Fabrication? Benchmarking 5
D Internal or External PPM form or template wanted Document Control Systems, Procedures, Forms and Templates 5
B Are Parts in a Positive Recall w/o Customer Complaint considered as External PPM? Quality Manager and Management Related Issues 6
P How to "translate" PPM and On Time Delivery in Monetary Terms Quality Manager and Management Related Issues 7
A How do I set up a PPM with Total pcs shipped vs. Total Scrap for the month Statistical Analysis Tools, Techniques and SPC 10

Similar threads

Top Bottom