Complaints time series significant change

Steve Prevette

Deming Disciple
Leader
Super Moderator
#11
Actually, SPC does NOT depend upon normality, at least according to the originator of it, Dr. Shewhart. Dr. Wheeler also agrees.

What would I do to find out on an increasing trend? Well, hopefully you have characterization data on each complaint - what was the item complained about, what was the failure mode, what company originated the complaint, did they assign a severity rating (such as 1 to 5 stars). One can slice and dice that database with histograms and Pareto charts to identify any big pockets, and or compare each source before and after the trend to see which one(s) changed.

Tell you what. If you are willing to share your data in a spreadsheet I'll put a few hours into it as a freebie. If you want it kept private, you can send me a private message here on the Cove, but would hope could share the results (maybe with replacing company names or product names with company 1, 2, 3; project A, B, C).
 
Elsmar Forum Sponsor

01mercy

Involved In Discussions
#12
@Steve Prevette thank you for your offer. I've got an ok to share an anonymized file.
I did look at my sixsigma course book to determine which SPC tool it would point me to.
I determined
type of data = discreet
what to count = defects per unit when I would say #complaint per device in market per time period, because defect units would not make sense since we have continuously one product in market and not a situation of selling make product from a production line
Area of defect possibility = constant
That brings me to a C chart
However this would mean the sample size would always be the same and on that part I don't really know what to choose since there is one product in the market but the amount of users vary over time.
I'm not sure what to choose to take as the population where the sample size is taken from
- 1 product
- x amount of customers
- y amount of users
- z amount of instances that users use the product
 

Bev D

Heretical Statistician
Leader
Super Moderator
#13
A C chart might work as it is intended for a population where the area of opportunity is not calculable but is ‘large’ compared to the number of occurrences (As you describe). In this case you will absolutely have to recalculate the control limits for every revision change As they area of opportunity will change. One thing to watch is the number of software ‘units’ are sold. It only makes sense that as you sell more units there will be more complaints. If your sale are not increasing substantially it won’t make much difference. When I have had an increasing sales base I simply divide the count of complaints by the cumulative number of units sold for that version each month. this would be a u chart…

This situation is not uncommon (I have literally faced it with hundreds probably thousands of cases from product level tracking to specific defect level tracking) and it absolutely makes the standard statistical tests inappropriate. I have generally found that the I, MR chart works better but a C chart or u chart can work. You have to try it out - there is no real way to know before plotting the data and looking at it.

I again strongly recommend the book by Donald Wheeler and his articles available at SPC Press or Quality Digest. It is one thing to get (absolutely excellent) advice from Steve for this specific incident, but there is no substitute for truly understanding what this ‘SPC stuff’ is all about and how to use it in different situations.
 

Steve Prevette

Deming Disciple
Leader
Super Moderator
#14
I would start with a c-chart and see what that tells you. Also doing some Pareto charts for characteristics I noted before may lead you to want to do some defect rates - the u-chart - with complaints per item sold, or dollar value of items sold. Bev runs through some ideas in the previous post. The idea is primarily to start with something (and might as well choose something easy), see what it is telling you, and then get more complex if it appears you need to. For example at one location I trended union grievances, and a simple c chart was useful, and usually when there was a change in union grievances we were able to find the source and learn from it.
 

01mercy

Involved In Discussions
#15
@Bev D and @Steve Prevette thank you for your replies.
Steve thank you for your suggestion to have a look at my data.
I've attached it here :thanx:

ProductA, B and C are the main products, ProductD is a bin of some smaller non-med products
I think I anonymized everything, if not pls let me know because it's on this open forum :)

@Bev D I have looked for the book but it seems I can't find a source that sells it (I'm in Europe). If you know of one pls let me know.
Does the book go into how to deal with non-standard situations, because that's what I most often run into in my work. On my previous workplace for example there was a big discussion how to deal with make product of which for a part we did and a part we did not get the serial number back from the consumer and thus couldn't trace it back to the batch and time made. I do have my statistics sixsigma book and also my data science course notes but indeed a book that guides me with examples of non-standard situations and how to make choices would be great.

I will take both your advise and also start working to try to get these graphs made myself.
Is it ok to share if I've set them up, to have a look how far it is correct what I did or if I run into probs?
 

Attachments

Miner

Forum Moderator
Leader
Admin
#16
First off, Bev and Steve are providing good advice. There is an alternate method that you can consider that works well when you know the approximate date (day, week, or month) of manufacture, particularly when your returns are reliability returns rather than out of the box failures. That method is called a Kaplan-Meier analysis. It is more complex and complicated, but provides a much clearer picture of what is happening when you have a lot of variation in the time to failure.

This is one of several graphs. I sanitized it, but it included known issues above the date codes.
1663934324595.png
 
Last edited:

01mercy

Involved In Discussions
#17
First off, Bev and Steve are providing good advice. There is an alternate method that you can consider that works well when you know the approximate date (day, week, or month) of manufacture, particularly when your returns are reliability returns rather than out of the box failures. That method is called a Kaplan-Meier analysis. It is more complex and complicated, but provides a much clearer picture of what is happening when you have a lot of variation in the time to failure.
Thanks I didn't know of that, I will have a look at it, might come in handy in the future if I run in such situation again. :thanx:
 

Bev D

Heretical Statistician
Leader
Super Moderator
#18
You won’t find any helpful info in you six sigma book or other stats books. Those cover what are called enumerative statistics and you are in the analytic statistics realm. (Occasionally enumerative statistics will not hurt but in these cases they are mathematical overkill.).
You can order the Wheeler book from SPCpress.com. It is under the Books tab.
 

Bev D

Heretical Statistician
Leader
Super Moderator
#20
In my experience this isn't unusual. The c chart can work but with the number of uses of various parts of the software that might generate complaints being susceptible to non-homogenous variation the I, MR chart is more likely a better indicator of stability - it certainly matches what our eyes tell us.

As Dr. Wheeler pointed out the c chart and p chart are subject to many 'assumptions' that are really requirements and so the I, MR chart is better choice.
 
Thread starter Similar threads Forum Replies Date
B Documenting Medical Device Complaints after End of Life? Medical Device and FDA Regulations and Standards News 0
J Are complaints applicable to development of medical devices? Customer Complaints 2
G Too many customer complaints Customer Complaints 16
G Supplier doesn't accept the complaints Supplier Quality Assurance and other Supplier Issues 20
S RMAs & Closing Complaints ISO 13485:2016 - Medical Device Quality Management Systems 7
D ISO 13485 8.2.1 and 8.2.2 - Customer Feedback and Customer Complaints ISO 13485:2016 - Medical Device Quality Management Systems 5
V Quality review Meeting with Customer for complaints we received Customer Complaints 6
A ISO 10002:2018 Checklist Needed (Complaints Handling) Customer Complaints 5
Q Refusal to Document Complaints Customer Complaints 39
R Do Medical Device Complaints include Clinical Trials? Medical Device and FDA Regulations and Standards News 3
A Complaint alert levels - Industry standards pertaining to alert levels for complaints Customer Complaints 5
0 PPM calculation of customer complaints for PMS Customer Complaints 10
J Complaints and not the legal manufacturer? 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 20
S Best software for customer support/complaints? Customer Complaints 0
N ISO 9001:2015 Customer Complaints Requirements ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 4
tnorton Lessons learned from implementing Customer Complaints Customer Complaints 1
D Volkswagen Complaints Management Rules Customer and Company Specific Requirements 0
Q Documenting Customer Complaints (Records) Customer Complaints 6
K Risk Analysis Updates due to complaints ISO 14971 - Medical Device Risk Management 10
Ashok sunder Is it possible to reduce FMEA Occurrence and Detection Ranking after corrective action taken for customer complaints? FMEA and Control Plans 6
Marc Complaints - How to Complain (Xenforo Software October 2018) Elsmar Xenforo Forum Software Instructions and Help 0
S What is the process of capturing "oral" customer complaints? Customer Complaints 9
J MDR reporting and CAPA thoughts? Classifying Complaints on Risk 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 13
C Backorders and delayed packages complaints? Preventive Action and Continuous Improvement 5
C UDI and complaints ISO 13485:2016 - Medical Device Quality Management Systems 6
J Medical Device End of Life and Complaints 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 5
Q Does ISO 9001 require CARs for all customer complaints? Customer Complaints 2
Marc Customer Complaints Discussions in All Elsmar Forums Customer Complaints 0
G App for logging complaints. Customer Complaints 2
C Complaint Training - FDA - How to recognize complaints and what actions to take 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 8
Q Managing Internal/External RNC and Complaints in same Format ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 4
L Implantable Medical Device Manufacturer Going Out of Business - Complaints EU Medical Device Regulations 2
T Are internally found Medical Device Software "Bugs" Complaints? 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 3
A Difference Between Complaints and Feedbacks in terms of Follow-Up Actions Needed Customer Complaints 3
J ISO 13485 Complaints (what is a complaint) Customer Complaints 5
M Is PPM a good metric for gathering companywide complaints ? Customer Complaints 2
A How to handle complaints after end of specified medical device life-time ISO 13485:2016 - Medical Device Quality Management Systems 5
L Creating a xlsx Customer Complaint file to track Complaints Excel .xls Spreadsheet Templates and Tools 2
D Can anyone share a Customer Complaint procedure not requiring logging complaints ? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 5
shimonv Classification of Complaints from Beta Sites - Complaints Definition per QSR 820.3 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 5
A Help me understand the applicability of 21 CFR Part 820.198 (Customer Complaints) 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 3
R Risk Management for Complaints ISO 14971 - Medical Device Risk Management 2
A SOP for Internal Complaints between Sites Customer Complaints 2
J Why are there Customer Complaints when the process is in control? Six Sigma 24
S FDA terms - Adverse Events, Medical Device Reporting, MAUDE and Customer Complaints US Food and Drug Administration (FDA) 1
G Corrective Action against Customer Complaints ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 9
S Customer Complaints - Categorization help requested Customer Complaints 2
W Investigating Medical Devices Customer Complaints US Food and Drug Administration (FDA) 5
T How many complaints, defects and non-conformances are typical? Customer Complaints 5
J Quality Manager's role in Customer and Supplier Complaints Customer Complaints 10

Similar threads

Top Bottom