Effectiveness of Human Inspection - Reality vs. the 80% 'Rule of Thumb Acid Test'

E

engjane

#1
I admit, Im being a bit lazy....I am Sure this topic has been discussed before but I dont want to spend too long searching.

We are looking for some wise words on the effectiveness of human visual inspection.
I know that it is about 80% and I know you can test it using the sentence with all the F's - most people only spot half of them. However, we wanted to find a paper or written comment somewhere about the actual effectiveness of vis. insp. to try and use it as a weapon!

Sounds ominous i know.

We have lots of vis and double vis inspection and we know its "pants" (translates as bad....) :bonk:
but we want to support the "pants" comment with something more solid.

Anybody know a good book, web link or something else that could help?

:thanx:
Jane
 
Elsmar Forum Sponsor
D

D.Scott

#2
Jane, I will do some research to try to find something for you but it will have to wait a bit. In the meantime, the 80% effective theory you are quoting was based on a study done in a unique inspection situation. Without finding my references I will relate what I remember in sort of general terms. You will get the idea but please don't count on the value of the "actual information". I will make up names just to relate the story.

Let's use Motorola - wanted to do an inspection of circuit boards and found that the inspection found only about 80% of the defects. The theory that visual inspection is only 80% effective stemmed from that. This may have some basis of truth when you are looking for multiple defects on a highly technical circuit board but would the theory hold true if you were asked to inspect a line of white refrigerators and identify any red ones? The analogy is, of course, absurd because of the extreme ends but the point is still valid in that it depends on what you are inspecting for. There is also the thought that if inspection is truly 80% effective, all you have to do is inspect them 3 times. This would mean you end up with less than 1 un-identified defect after the third sort.

You can probably tell that I don't buy the 80% theory. I will see if I can find some information and post it for you. In the meantime, why not run some reliability/capability tests on your sorted parts and see what it comes out when applied to your specific inspection criteria.

Dave
 

Jim Wynne

Staff member
Admin
#3
Human inspection as opposed to what? It goes without saying, I think, that non-human forms of inspection are generally more likely to be successful in finding defects (assuming, of course, that the efficacy of the device has been established). But at what cost? The application of machine-vision solutions, for example, must be balanced against the risk of the difference between human error and machine error, and that risk will be different for different applications. And the risk of error is not the only factor; it could be that an automated inspection system would be cheaper to operate than its human alternative.

The "find the "f's" exercise, while useful for providing a gross example of human error in visual inspection, shouldn't be taken as a scientific experiment unless the inspectors are examining text all day in an effort to find all of the "f's."
 
R

Rob Nix

#4
Slightly :topic:

In JSW05's reply I visually inspected and found 30 "f"s; then I did an automation assisted inspection (i.e. I copied the reply to "Word" and did a "find" for all "f"s) which yielded 32. So my manual visual inspection was 30/32 = 94%! :eek:
 
D

David Hartman

#5
I believe that it was in one of the earlier versions of Dr. Juran's Handbook that it is stated that is approximately 85% effective (in fact this version of the Handbook provides directions for performing an Inspector Effectiveness audit).
 
R

ralphsulser

#6
I think you are right about it being in Dr. Juran's earlier handbook. However, I recall that it was quted as being "at best 75% effective". I know we did trials in Q101 for the class, and don't remember anyone getting above 75% using the colored beads. Then we did a re inspected using sampling paddles at increased sample sizes, which exhibited a more effective likelyhood of rejecting a defective lot size.
 

Mike S.

Happy to be Alive
Trusted Information Resource
#7
I think Dave nailed it -- it depends. The only way to get an accurate idea of how effective it is in your particular application is probably to test it at your place with your people.
 

Bev D

Heretical Statistician
Staff member
Super Moderator
#8
agreed. the literature will provide examples of specific situations, but each situation is different and with the appropriate training and environemtnal conditions, humn visual inspection can be quite effective.

The approach that I have used quite successfully over the years is to perform a KAPPA test (a search on the internet will yield many papers on the math behind this fairly simple test.)

I collect actual defect samples from the line and more samples that are not defective - per the inspectors. I ensure that the ratio of 'defectives' to 'not defective' is equivelant to the reported rate over time. If there are certian conditions that I know are tricky - inspectors 'seem' to have a tough tiem detecting them or they are high on the 'escape' list I make sure I incldue those as well - but maintain the 'normal' defect rate (dont' over salt defectives, it biases the test)

Then I have each inspector (usually a small subset of the task team 3-4 people) inspect the test set twice recording which ones they reject and pass. I typically have the retest session doen a couple of weeks later stating to the inspectors that it's a different set.

Teh Kappa looks at agreement within the inspector and between inspectors and demonstrates the current effectiveness.

From this test I knwo which inspectors are better - if any. and which defect types are 'easier' to call. From there I formulate improvement actions for th einspection techniques including training, sampels and limit samples and a NEW retest to determine if the inpectors improved. Usually they do.

Of course soem of the improvent actions are poke yokes to detect certain things where possible and upstream corrective action to remove the defects all together. Some times there are defects wehre we can't improve the visual inspection effectiveness for but no I have the data that says so. not somebody else's paper on their defects! data is mightier than the pen - at least in this case.
 

Marc

Hunkered Down for the Duration with a Mask on...
Staff member
Admin
#9
Effectiveness of visual inspection is completely dependent upon what is being inspected and for what. I have cited the case of the anodized part where the customer's inspectors would accept parts which were previously rejected and 'secretly' mixed in with parts off the line and fed through inspection again. In that case we were looking at cosmetic issues. In cases such as this the 80% 'rule' doesn't hold water.

Let's look at an extreme on the other end of the spectrum - is there a weld nut where there is suposed to be one. This is not subjective as in the case of the anodized part being inspected for cosmetic issues. In this case the 80% 'rule' would be more 'applicable'.

It is true that it is difficult, and in many cases impossible, to poke yoke a visual inspection for cosmetic / appearance issues, but even in the case of a nut being present there may not be a solution which is cheap enough.

Just a few thoughts.
 
Thread starter Similar threads Forum Replies Date
S Nonconformance to a Note in ISO13485 clause 6.2 Human Resources Effectiveness ISO 13485:2016 - Medical Device Quality Management Systems 13
K Visual Inspection Effectiveness Inspection, Prints (Drawings), Testing, Sampling and Related Topics 16
M ISO14971:2019 - Verification of implementation and effectiveness of risk control ISO 14971 - Medical Device Risk Management 3
C Quality, efficiency and effectiveness of daily work routines Misc. Quality Assurance and Business Systems Related Topics 4
V CAPA effectiveness check Nonconformance and Corrective Action 3
N How to monitor the effectiveness of the CAPA system ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 8
qualprod ISO 9001 5.1.1 - How to measure System effectiveness ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 4
qualprod Add new action plans in CA, while waiting effectiveness - Same problem reappears ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 13
qualprod Best criteria to measure Corrective Action effectiveness - Poor Maintenance ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 10
C CMDR Essential (Safety & Effectiveness) Requirements Checklist? Canada Medical Device Regulations 9
J AQL for Effectiveness Check - What AQL tiers to use on this type of sampling? Nonconformance and Corrective Action 9
M Medical Device News FDA - Transformative new steps to modernize FDA’s 510(k) program to advance the review of the safety and effectiveness of medical devices Other US Medical Device Regulations 0
J Recall Effectiveness - Accessory to Medical Devices shipped through Distributors Customer Complaints 3
qualprod To Evaluate Effectiveness in All Improvements? (ISO 9001:2015) ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 9
M Corrective Action Effectiveness Nonconformance and Corrective Action 5
Uriel Alejandro Evaluation of the effectiveness of correction actions General Auditing Discussions 15
L Verifying training effectiveness for procedure changes ISO 13485:2016 - Medical Device Quality Management Systems 4
T Hot dogs and process operation and control effectiveness? 4.1.3 a) ISO 13485:2016 - Medical Device Quality Management Systems 2
S ISO 9001:2015 4.4.1 (c) Process Effectiveness ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 17
M Effectiveness Measuring and Efficiency Measuring In ISO 9001:2015 ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 2
A IATF 16949 - Non-conformity on 5.1.1.2 Process effectiveness and efficiency IATF 16949 - Automotive Quality Systems Standard 5
S What are some ways to demonstrate Effectiveness of Training? ISO 13485:2016 - Medical Device Quality Management Systems 29
B IATF 16949 Cl. 7.3.2 - Documented Process (Effectiveness & Efficiency) IATF 16949 - Automotive Quality Systems Standard 7
R Documenting Self-Training and Effectiveness - ISO 13485:2016 Training - Internal, External, Online and Distance Learning 4
B Auditing Senior Management to determine the Effectiveness of Management Processes IATF 16949 - Automotive Quality Systems Standard 6
J Process Effectiveness and Efficiency (IATF 16949) IATF 16949 - Automotive Quality Systems Standard 29
P Is ISO 9001 Model tested for its effectiveness by ISO before its release? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 1
D Do people read PEARS (Process Effectiveness Assessment Report)? AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 15
B IATF 16949 Cl. 9.3.2.1 - Management Review Inputs - Process Effectiveness and Efficie IATF 16949 - Automotive Quality Systems Standard 14
Colin Effectiveness of IRCA Registered Training ASQ, ANAB, UKAS, IAF, IRCA, Exemplar Global and Related Organizations 4
R ISO 13485:2016 Clause 6.2 - Effectiveness Check ISO 13485:2016 - Medical Device Quality Management Systems 1
Q Evaluating Effectiveness of a Preventive Action after Closure Preventive Action and Continuous Improvement 7
M Training Effectiveness TS 16949 Clause 6.2.2c Training - Internal, External, Online and Distance Learning 5
Q Corrective Action Effectiveness also for correction? Nonconformance and Corrective Action 11
S AS9100C 8.2.2 - Selected Tools & Effectiveness of IA Process and Performance AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 3
A Performance vs. Effectiveness - ISO 9001:2015 par. 9.1.1 ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 11
S How to Monitor Training Effectiveness Training - Internal, External, Online and Distance Learning 13
Q Effectiveness Plan for CAPA initiated from External Audit US Food and Drug Administration (FDA) 3
Buckyb Are KPIs a requirement to determine Process Effectiveness? AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 14
M Auditing ISO9001/AS9100 7.3 Design and Development Quality System Effectiveness Manufacturing and Related Processes 12
I Not having "effectiveness" in the Quality Policy ISO 13485:2016 - Medical Device Quality Management Systems 8
M Can OEE (overall equipment effectiveness) exceed 100%? Manufacturing and Related Processes 2
A Effectiveness of Risk Control Measures ISO 14971 - Medical Device Risk Management 4
M Where does OEE (Overall Equipment Effectiveness) belong? Quality Tools, Improvement and Analysis 11
R The Value and Effectiveness of In Process Inspection Quality Manager and Management Related Issues 17
L OEE (Overall Equipment Effectiveness) and Cost Savings Manufacturing and Related Processes 2
G 510(k) Pre-market approval Application Requirements - Effectiveness of the product Other US Medical Device Regulations 1
S ISO9001:2008 - Effectiveness Tracking for Nonconformance ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 1
D Metrics on effectiveness of Customer Returns Misc. Quality Assurance and Business Systems Related Topics 9
J Reviewing Corrective Action Effectiveness ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 8

Similar threads

Top Bottom