Hello...first time on this, so bear with me. I need to develop a statistical test to measure the effectiveness of 100% inspection on product to prove the old 'industry standard' of that 100% inspection is 85% effective. I need more than the 'count the f's or c's or o's tests'. Obviously, I know I could go with a standard 'seeded' test and I could do a Kappa test which measures the attribute agreement (kappa)...both which I am planning on doing but I was hoping I could find something that will give me a measure of 'effectiveness' that has a solid statistical basis. Any suggestions?:thanx:

Hello...first time on this, so bear with me. I need to develop a statistical test to measure the effectiveness of 100% inspection on product to prove the old 'industry standard' of that 100% inspection is 85% effective. I need more than the 'count the f's or c's or o's tests'. Obviously, I know I could go with a standard 'seeded' test and I could do a Kappa test which measures the attribute agreement (kappa)...both which I am planning on doing but I was hoping I could find something that will give me a measure of 'effectiveness' that has a solid statistical basis. Any suggestions?:thanx:

What I'd do is set up a blind test with some known "good" items and some known "bad" items and give them to the inspectors. I'd track two p-chart control charts - what percent of known "good" items got declared as bad, and what percent of known "bad" items got declared as good. I'd keep these as two separate indicators as a person may be very good at detecting bad items, but perhaps is too quick to delare something bad (leading to discarding good items needlessly). You could then use the control chart to show improvement with time, and also to identify if there are any issues (outside of normal variation) with any inspector.

Be prepared though. Many times an exercise like this will derail at the very beginning, with an inability to get a firm answer on the test parts as to which is "good" and which is "bad" and why.

Hello...first time on this, so bear with me. I need to develop a statistical test to measure the effectiveness of 100% inspection on product to prove the old 'industry standard' of that 100% inspection is 85% effective. I need more than the 'count the f's or c's or o's tests'. Obviously, I know I could go with a standard 'seeded' test and I could do a Kappa test which measures the attribute agreement (kappa)...both which I am planning on doing but I was hoping I could find something that will give me a measure of 'effectiveness' that has a solid statistical basis. Any suggestions?:thanx:

Hello Triana,

Welcome to The Cove. :bigwave:

Look at this thread: http://www.elsmar.com./Forums/showthread.php?t=23041&highlight=inspectors

You may find something of use to you.

Stijloor.

Triana,

First of all, welcome to the Cove.

You pose an interesting problem. In my opinion, you must be careful about using parts that are classified as good or bad and rating your inspectors on whether or not they picked the part as good or bad. The reason is somebody had to pick the parts as good or bad before you got them and that person stands the same probability of being wrong as your test inspectors.

Your concern is whether or not all of your inspectors rated the part the same (all good or all bad).

The way I would proceed is go to the product line and get 80 good parts from the good parts box and 20 bad parts from the bad parts box. I do not know how much your parts cost, your production rate, or how long it takes to inspect the parts, so you may want to vary the number of parts and the distribution to fit your situation.

I would clearly number each part and have 10 different inspectors inspect the parts and record the results. Again you may want to adjust this number, but the smaller your sample and fewer your inspectors, the less statistically significant your answer.

I have attached a spreadsheet for use in conducting this test. You may wish a different design. It will work for any number of parts and inspectors, just enter the number of parts and number of inspectors at the top.

There are many things you can do with this method.

1. Make it more statistically powerful by repeating it with different inspectors.
2. Show the difference if the inspectors are rotated frequently by having each inspector only inspect 5 parts per day for 20 days, or 5 parts per hour for 20 hours.
3. Show the effect of tedium by having each inspector inspect all the parts at one sitting.

If I were to guess where your are headed with this, you are building a case to demonstrate for senior management that random sampling techniques (either SQC or SPC) are more effective than a 100% sampling technique.

This is a very good first step.

I hope this helps.

Best regards,

Jim Shelor

First of all, welcome to the Cove.
Ditto. :D

You pose an interesting problem. In my opinion, you must be careful about using parts that are classified as good or bad and rating your inspectors on whether or not they picked the part as good or bad. The reason is somebody had to pick the parts as good or bad before you got them and that person stands the same probability of being wrong as your test inspectors.

This is not necessarily true; there could be a significant difference in the expertise of the screener(s) and the method used for screening the experimental parts.

Your concern is whether or not all of your inspectors rated the part the same (all good or all bad).

Not if the object is to tell whether or not inspectors can tell good from bad, although establishing continuity between inspectors helps to establish whether they actually understand good from bad, and the techniques required to make the distinction.

I think I'm clear on what Triana wants to do, but I'm not sure why. If we knew that we might be able to save her some time and aggravation.

Jim,

This is not necessarily true; there could be a significant difference in the expertise of the screener(s) and the method used for screening the experimental parts.

Yes, "same probability" was probably a poor choice of words, however, there is a non-zero probability that some number of the parts have been classified incorrectly by the inspector who picked them.

Not if the object is to tell whether or not inspectors can tell good from bad, although establishing continuity between inspectors helps to establish whether they actually understand good from bad, and the techniques required to make the distinction.

I do not believe the object is to see if the inspectors know the difference between good and bad, I think the object is to see how many parts may have been classified as bad when they were good and vice versa. In other words, the effectiveness of 100% inspection.

Best regards,

Jim Shelor

Hi all ;) ,

Thank you so much for your responses! This is really neat to be able to bounce questions off to such knowledgeable people and get all your valuable opinions! I'm sorry I didn't know about this place sooner!

I work in a regulated industry and there is a requirement for 100% inspection. We've had some complaints of some defects going through and customers are asking how effective our 100% inspection process is. I know Juran has been quoted as saying it's ~70% effective and 200% inspection is only about 85% effective (or something like that). I was asked to prove it statistically and quantifiably.

Your ideas did point me in a good direction though. I found what I needed in Juran's 4th edition QC handbook, page 18.95. Interesting assessment in case you're interested:

Accuracy of inspector = (d-k)/(d-k+b)
where: d=defects reported by inspector, k=# of defects reported by inspector but not really defects, b=defects missed by the inspector.

I'll do this (as well as follow your much appreciated suggestions) and let you know how it works! :cfingers:

Thank you all for your fantastic feedback! I love this forum!!!

:applause::thanks:

Jim,
I do not believe the object is to see if the inspectors know the difference between good and bad, I think the object is to see how many parts may have been classified as bad when they were good and vice versa. In other words, the effectiveness of 100% inspection.


Well, what's the alternative? Isn't the object actually to show that sampling is at least as good as 100% inspection? The method you shared helps to determine whether 100% inspection works, but it doesn't address the alternative. Don't get me wrong--I'm not down on your contribution here; I just wanted to point out that it's only half the job.:D

I also wanted to mention the fact that sampling and 100% inspection aren't mutually exclusive practices. The former almost always begets the latter when a defect turns up.

Well, what's the alternative? Isn't the object actually to show that sampling is at least as good as 100% inspection? The method you shared helps to determine whether 100% inspection works, but it doesn't address the alternative. Don't get me wrong--I'm not down on your contribution here; I just wanted to point out that it's only half the job.:D
Jim,

That is true. It is only half the problem. I intentionally stopped at the first half because you need the answer to the first half before you can do the second half. Additionally, addressing the second half at this point would only detract from the original question.

Respects,

Jim Shelor