Complaints time series significant change

[Steve Prevette]

Actually, SPC does NOT depend upon normality, at least according to the originator of it, Dr. Shewhart. Dr. Wheeler also agrees.

What would I do to find out on an increasing trend? Well, hopefully you have characterization data on each complaint - what was the item complained about, what was the failure mode, what company originated the complaint, did they assign a severity rating (such as 1 to 5 stars). One can slice and dice that database with histograms and Pareto charts to identify any big pockets, and or compare each source before and after the trend to see which one(s) changed.

Tell you what. If you are willing to share your data in a spreadsheet I'll put a few hours into it as a freebie. If you want it kept private, you can send me a private message here on the Cove, but would hope could share the results (maybe with replacing company names or product names with company 1, 2, 3; project A, B, C).

 

[01mercy]

@Steve Prevette thank you for your offer. I've got an ok to share an anonymized file.
I did look at my sixsigma course book to determine which SPC tool it would point me to.
I determined
type of data = discreet
what to count = defects per unit when I would say #complaint per device in market per time period, because defect units would not make sense since we have continuously one product in market and not a situation of selling make product from a production line
Area of defect possibility = constant
That brings me to a C chart
However this would mean the sample size would always be the same and on that part I don't really know what to choose since there is one product in the market but the amount of users vary over time.
I'm not sure what to choose to take as the population where the sample size is taken from
- 1 product
- x amount of customers
- y amount of users
- z amount of instances that users use the product

 

[Bev D]

A C chart might work as it is intended for a population where the area of opportunity is not calculable but is ‘large’ compared to the number of occurrences (As you describe). In this case you will absolutely have to recalculate the control limits for every revision change As they area of opportunity will change. One thing to watch is the number of software ‘units’ are sold. It only makes sense that as you sell more units there will be more complaints. If your sale are not increasing substantially it won’t make much difference. When I have had an increasing sales base I simply divide the count of complaints by the cumulative number of units sold for that version each month. this would be a u chart…

This situation is not uncommon (I have literally faced it with hundreds probably thousands of cases from product level tracking to specific defect level tracking) and it absolutely makes the standard statistical tests inappropriate. I have generally found that the I, MR chart works better but a C chart or u chart can work. You have to try it out - there is no real way to know before plotting the data and looking at it.

I again strongly recommend the book by Donald Wheeler and his articles available at SPC Press or Quality Digest. It is one thing to get (absolutely excellent) advice from Steve for this specific incident, but there is no substitute for truly understanding what this ‘SPC stuff’ is all about and how to use it in different situations.

 

[Steve Prevette]

I would start with a c-chart and see what that tells you. Also doing some Pareto charts for characteristics I noted before may lead you to want to do some defect rates - the u-chart - with complaints per item sold, or dollar value of items sold. Bev runs through some ideas in the previous post. The idea is primarily to start with something (and might as well choose something easy), see what it is telling you, and then get more complex if it appears you need to. For example at one location I trended union grievances, and a simple c chart was useful, and usually when there was a change in union grievances we were able to find the source and learn from it.

 

[01mercy]

@Bev D and @Steve Prevette thank you for your replies.
Steve thank you for your suggestion to have a look at my data.
I've attached it here

ProductA, B and C are the main products, ProductD is a bin of some smaller non-med products
I think I anonymized everything, if not pls let me know because it's on this open forum

@Bev D I have looked for the book but it seems I can't find a source that sells it (I'm in Europe). If you know of one pls let me know.
Does the book go into how to deal with non-standard situations, because that's what I most often run into in my work. On my previous workplace for example there was a big discussion how to deal with make product of which for a part we did and a part we did not get the serial number back from the consumer and thus couldn't trace it back to the batch and time made. I do have my statistics sixsigma book and also my data science course notes but indeed a book that guides me with examples of non-standard situations and how to make choices would be great.

I will take both your advise and also start working to try to get these graphs made myself.
Is it ok to share if I've set them up, to have a look how far it is correct what I did or if I run into probs?

 

[Miner]

First off, Bev and Steve are providing good advice. There is an alternate method that you can consider that works well when you know the approximate date (day, week, or month) of manufacture, particularly when your returns are reliability returns rather than out of the box failures. That method is called a Kaplan-Meier analysis. It is more complex and complicated, but provides a much clearer picture of what is happening when you have a lot of variation in the time to failure.

This is one of several graphs. I sanitized it, but it included known issues above the date codes.

 

[01mercy]

First off, Bev and Steve are providing good advice. There is an alternate method that you can consider that works well when you know the approximate date (day, week, or month) of manufacture, particularly when your returns are reliability returns rather than out of the box failures. That method is called a Kaplan-Meier analysis. It is more complex and complicated, but provides a much clearer picture of what is happening when you have a lot of variation in the time to failure.

Thanks I didn't know of that, I will have a look at it, might come in handy in the future if I run in such situation again.

 

[Bev D]

You won’t find any helpful info in you six sigma book or other stats books. Those cover what are called enumerative statistics and you are in the analytic statistics realm. (Occasionally enumerative statistics will not hurt but in these cases they are mathematical overkill.).
You can order the Wheeler book from SPCpress.com. It is under the Books tab.

 

[Steve Prevette]

Attached is the file, with a c-chart and a x-ImR chart included. The overall complaint rate is stable - but interestingly the spread of the data is much higher than what the c-chart predicts. It would be easy to link the same logic used to any other column.

 

[Bev D]

In my experience this isn't unusual. The c chart can work but with the number of uses of various parts of the software that might generate complaints being susceptible to non-homogenous variation the I, MR chart is more likely a better indicator of stability - it certainly matches what our eyes tell us.

As Dr. Wheeler pointed out the c chart and p chart are subject to many 'assumptions' that are really requirements and so the I, MR chart is better choice.