PPM calculation of customer complaints for PMS

01mercy

Involved In Discussions
#1
I'm seeking advise on how to calculate ppm levels of customer complaints for PMS in a B2C market. EU medical and non-medical.

Theoretically (normal) the calculation would be #complaints(from_sales(mthX))/#sales(mthX)*1.000.000, monthly figure

What is preventing us from getting this figure is:
- There is a big delay between complaint date and the sales date. This delay is a distribution therefore we can't use one figure for correction.
- Only from a little fraction of the complaints we get the device serial nr to track back to the sales date.

Solutions that have been proposed are:
- PPM calculation by cumulative numbers -> #complaints(CUM)/#sales(CUM)*1.000.000, monthly figure
- PPM calculation by moving annual total -> #complaints(MAT)/#sales(MAT)*1.000.000, monthly figure
- The three calculations above but with the individual complaints corrected for av (sales->complaint) delay to het the amount of complaints approximately at the right month that correlates to the sales

The problem with the solutions is that they all create their own artifacts in graphs either at the start of sales or at the end of sales.
Needles to say the straight forward PPM calculation of complaints per month against sales per month does not give a realistic number due to the delays between sales and complaints.
The delays can be in the range of a few days up to several months.

Does anyone have encountered such a situation and have implemented a solution to come up with a number that will show if complaints are rising or lowering independent of sales?

I would love to hear about it :)
 
Elsmar Forum Sponsor

Johnny Quality

Involved In Discussions
#2
It seems to me that you're putting a lot of effort into generating a PPM formula. Like you, our business doesn't work with a typical PPM calculation due to the delay between manufacturing, sales and complaints - it can be many months.

I have abandoned calculating PPM's and have redirected my efforts into reducing complaints and meaningful metrics to me, it has worked wonders.
 

Pau Calvo

Starting to get Involved
#3
It seems to me that you're putting a lot of effort into generating a PPM formula. Like you, our business doesn't work with a typical PPM calculation due to the delay between manufacturing, sales and complaints - it can be many months.

I have abandoned calculating PPM's and have redirected my efforts into reducing complaints and meaningful metrics to me, it has worked wonders.
I agree with Johnny, if it is not a must have KPI I would change for one more focus to the number of claims or response time.
I work on automotive industry and also have moths of delays between the claim and manufacturing but i must have Customer PPM KPI so we monitor them by month but evaluated in quarters.
 

01mercy

Involved In Discussions
#4
Johnny and Pau,
Thank you for your reply. I agree with how you look at this.

However it is an obligation for us to do so. We have to monitor the risk to the consumer in the market and compare it with our risk files and if needed update them or act on the risk in the market if needed if the ppm levels become to high compared to the accepted ppm levels in the risk file.
This way of PMS is obligated for medical devices and we do this also for non-medical devices.
Since I work for a big global company these metrics are also made mandatory by the business.
In our company the business groups need to develop this per business group and our business group is relatively young therefore we are looking into this how to calculate this with the challenges we face.

Therefore I'm not looking for different approaches on way of working (but thank you for the feedback) but I'm looking for the best way to calculate ppm levels with the difficulties we're facing, as explained.
 

Johnny Quality

Involved In Discussions
#5
Hmmm...

Would it be possible to split your PPM's your PPM's by product line, process or any sort of classification relevant to you? I admit I'm in Automotive like Pau and I'm also in a small business and have no idea what it's like to be in a global company.

Failing that is it at all feasible to discuss your issue with whoever is responsible for dictating you must use PPM's? You mention your business group is young and you are facing a challenge with the system that has been dictated to you.
 

01mercy

Involved In Discussions
#6
Hi Johnny it is mandatory for our type of device on the market to monitor risk by ppm.
Therefore business demands this of us, not because they know how but because they know that it is mandatory, if we don't is is also a business risk. Just to say that an alternative way (not monitoring this and work on the complaint causes only) is not accepted

We do have a device which is split into several sub types with all kind of different features. However for each sub type we still have the same problem for calculating the ppm levels.
 
Last edited:

Bev D

Heretical Statistician
Staff member
Super Moderator
#7
ppm is not a requirement of any standard. is this a Customer (?) requirement or an 'internal' requirement?

ppm is just a proportion multiplied by a million just like percentage is a proportion multiplied by 100. that's just fun with math.
ppm is often casually used to refer to 'defect rate'. ppm and percentage etc. are just 'units of measure' of the defect rate.
anybody who really understands monitoring risk to the Customer has to understand that the count of defects (numerator) must come from the area of opportunity (denominator). so the misalignment of the sales data in a time period to the reporting of defects in the same time period are NOT aligned and this variation is not indicative of the true variation and defect occurrence that the Customer is experiencing. so you are misstating the 'risk' by definition.
 

01mercy

Involved In Discussions
#8
Bev D,
Indeed ppm is not a requirement. Monitoring risk in the market is.

The figure that is used is ppm.
Yes it is a ratio, anyone with a lab and quality background would know that ppm and % are resp. 10^6 and 10^2 multiplication of a fraction in the range 0-1.

Yes defect count per area of opportunity indeed and yes it is miss aligned.
Please read my post carefully because the whole reason I'm asking the question is to deal with the miss alignment due to the nature of the feedback process (only few cust are sharing back device serial nr info).
 
Last edited:

Bill Levinson

Industrial Statistician and Trainer
#9
Bev D,
Indeed ppm is not a requirement. Monitoring risk in the market is.

The figure that is used is ppm.
Yes it is a ratio, anyone with a lab and quality background would know that ppm and % are ratios.

Yes defect count per area of opportunity indeed and yes it is miss aligned.
Please read my post carefully because the whole reason I'm asking the question is to deal with the miss alignment due to the nature of the feedback process (only few cust are sharing back device serial nr info).
It doesn't look like a good metric because of, as you pointed out, the long lag time between the actual trouble and the time you become aware of it (customer complaint) plus the variation in the reporting time that you mentioned. Also, if not every customer who is affected complains, the results are not even accurate.

If the customer wants a metric of this nature (ppm per month or per quarter, possibly broken down by product line), go ahead and report it, but I would be more interested in whether customer complaints result in corrective and preventive action (CAPA) that goes to some kind of conclusion to prevent recurrence; any customer complaint can initiate a CAPA (for which I recommend AIAG's CQI-20 Effective Problem Solving process or the similar 8D process, although anything similar will work).

"Top FDA Form 483 Observations for Medical Device Companies in 2015" cites CAPA, complaint files, and nonconforming product as the 2nd, 3rd, and 4th biggest contributors for a total of 31.3%. 9 Top FDA 483 and Warning Letter Problems for Device Companies in 2015 Complaint files constitute 12% of Form 483 observations. I am not familiar with FDA but it looks like ineffective or no CAPA for complaints is the issue.

  1. Complaint Files

• CFR Clause 820.198
• QS Subsystem: CAPA
• Number of 483 Observations: 435
• Percent of total 483s: 12%

CFR - Code of Federal Regulations Title 21 requires a CAPA process.
CFR - Code of Federal Regulations Title 21 for complaint files

Again, this is not my area of experience or expertise (FDA and medical devices) but it looks pretty similar to what ISO 9001 and IATF 16949 require; a process for handling complaints that includes CAPA.
 

Bev D

Heretical Statistician
Staff member
Super Moderator
#10
01mercy: simply track the number of complaints. You can look at each ‘defect’ separately prioritized by severity. As long as your sales are not increasing or decreasing steeply (by about 15% up or down) the trends will be fairly accurate - at least to signal that investigation is warranted. You could use a simple c chart to do this. Without any way to align the numerator and denominator this is your only option.

Bill is also correct that some failure modes of high severity dont’ need to be trended - their occurence and report is sufficient to alert you to a serious problem. I use the analogy of seeing a mouse in your kitchen. You really don’t need an “accurate” count of how many mice are in your house or even traendign of the ‘sightings’ to know that you need to begin eradication immediately.

By the way I did read your post carefully. My responses are often general in nature to all readers. You would be surprised that many people in the quality profession are not well trained (not their fault) and the whole ppm thing is quite controversial with some.
 
Thread starter Similar threads Forum Replies Date
F Determining what type of scrap to include in my internal PPM calculation Quality Manager and Management Related Issues 5
Q PPM - Filtering Defects out of calculation Quality Tools, Improvement and Analysis 5
K Proper YTD PPM Supplier Calculation Six Sigma 3
V Calculation of PPM & Sigma Level in an Excel Spreadsheet Statistical Analysis Tools, Techniques and SPC 3
S Supplier ppm calculation issue - Disregard minor NC parts? Supplier Quality Assurance and other Supplier Issues 6
M Formula for the Calculation of PPM (Parts Per Million) Statistical Analysis Tools, Techniques and SPC 15
B Logistical or Quality PPM's? Calculation of PPM relevant parts Quality Tools, Improvement and Analysis 28
Sean Kelley Is ppm calculation valid or invalid for bulk materials? IATF 16949 - Automotive Quality Systems Standard 4
T Sample Size Calculation for PPM (Parts Per Million) Prediction Statistical Analysis Tools, Techniques and SPC 6
M Targets / limits for PPM/ Scrap Manufacturing and Related Processes 2
M Internal PPM Manufacturing and Related Processes 1
A ISO 2859-1 - 9.3.3.2 Switching score - Sampling Plans and PPM AQL - Acceptable Quality Level 0
S Software for Supplier Charge back and internal PPM General Information Resources 2
L PPM - wrong parts or defective parts? IATF 16949 - Automotive Quality Systems Standard 18
R PPM, Pictorial Process Mapping, Templates Manufacturing and Related Processes 5
V PPM - Internal PPM / External PPM Quality Manager and Management Related Issues 4
L PPM Defect Rate as a Performance Metric - Are there Flaws? Quality Manager and Management Related Issues 9
N Converting ETO Residual Limits from PPM to mg ISO 13485:2016 - Medical Device Quality Management Systems 6
J Internal PPM's in a medium sized metal stamping plant Quality Manager and Management Related Issues 6
S What is appropriate PPM level for a Plastic Injection Moulding Supplier Quality Tools, Improvement and Analysis 4
M Is PPM a good metric for gathering companywide complaints ? Customer Complaints 2
L Oil Field Bearing Production PPM Standard Benchmarking 2
V How is PPM (Parts Per Million) Calculated? Inspection, Prints (Drawings), Testing, Sampling and Related Topics 4
L Help with concept of PPM in SPC Statistical Analysis Tools, Techniques and SPC 6
Z Predicting External PPM Capability from Internal PPM Capability, Accuracy and Stability - Processes, Machines, etc. 1
S Cell Rubber Stamping PPM for Scrap Analysis Quality Tools, Improvement and Analysis 1
R Supplier Deviations - Should this be counted in the supplier ppm? Supplier Quality Assurance and other Supplier Issues 3
S How to calculate Supplier PPM at Receiving Inspection Document Control Systems, Procedures, Forms and Templates 2
Z Is There a Benchmark for PPM for Medical Devices? Benchmarking 4
M California Proposition 65 Compliant (no more than 600 ppm touch area) IEC 60601 - Medical Electrical Equipment Safety Standards Series 5
M What is an acceptable PPM for large volume fasteners? APQP and PPAP 6
G Sample Size for 2900 ppm Visual Defects Inspection Requirement Inspection, Prints (Drawings), Testing, Sampling and Related Topics 4
D Suitability of Attribute Control Charts for ppm Level Control Statistical Analysis Tools, Techniques and SPC 6
M Start Up Scrap vs. ppm (Parts per Million) in an FMEA IATF 16949 - Automotive Quality Systems Standard 17
H PPM Prediction for a New Automotive Electronic Product Reliability Analysis - Predictions, Testing and Standards 7
I Defectives (ppm) coming from Visual Inspection Inspection, Prints (Drawings), Testing, Sampling and Related Topics 1
G How do I calculate PPM from a Rolled Wire Suppllier Supplier Quality Assurance and other Supplier Issues 3
C Comparing PPM Year over Year - Determining the PPM decrease from 2009 to 2010 Benchmarking 10
5 Calculating PPM - Multiple product lines and multiple type of parts Six Sigma 2
B Automotive Supplier Benchmarking - Supplier PPM Data Benchmarking 5
O Estimating the dpm and ppm of the process from Cp and Cpk Statistical Analysis Tools, Techniques and SPC 4
S "Acceptable" PPM Level for Aluminum Die Casting Benchmarking 3
S Six sigma and defect rate (6 sigma = 3.4 PPM) Six Sigma 6
L Quality PPM (parts per million) Tracking - Stats for our Management Review Meeting Benchmarking 8
A What is the appropriate PPM Level for Sheet Metal Fabrication? Benchmarking 5
D Internal or External PPM form or template wanted Document Control Systems, Procedures, Forms and Templates 5
B Are Parts in a Positive Recall w/o Customer Complaint considered as External PPM? Quality Manager and Management Related Issues 6
P How to "translate" PPM and On Time Delivery in Monetary Terms Quality Manager and Management Related Issues 7
A How do I set up a PPM with Total pcs shipped vs. Total Scrap for the month Statistical Analysis Tools, Techniques and SPC 10
J Are there PPM statistics kept by Industry Manufacturing and Related Processes 4
Similar threads


















































Top Bottom