SBS - The best value in QMS software

AQL for Effectiveness Check - What AQL tiers to use on this type of sampling?

JeantheBigone

Quite Involved in Discussions
#1
Good day Covers!
We're looking at doing an effectiveness check to see if a procedure is being followed.
Does anyone have a rule of thumb to share on what AQL tiers to use on this type of sampling?
 
Elsmar Forum Sponsor

Bev D

Heretical Statistician
Staff member
Super Moderator
#2
AQL inspection plans are not intended for this type of assessment.

a better focus is to think about how you actually assess compliance.

some examples:
Are you going to watch the "operators"? then your sample size needs to be 'long enough' that the operators lose that 'I'm being watched' behavioral bias. And you'll need to watch everyone who performs the process - a sample won't really suffice in this case.
Do you have some electronic rail that can be monitored? (Like electronic batch records) then all you can assess a large number without observation bias. just make sure you hit all of the operators at least twice.

A few other considerations: sample size isn't as important as the representativeness of the observations.

Your observations should be spread over time: operators will often comply early after training and drift off course over time.
Are there conditions that make compliance more difficult? such as volumes, looming deadlines, first shift managerial chaos over third shift calm, different organizational cultures that support or denigrate compliance (R&D vs Operations for example).
The important thing is observe or assess during the most stressful times and not the easy times.
 

Mike S.

Happy to be Alive
Trusted Information Resource
#4
AQL inspection plans are not intended for this type of assessment.
It depends. If the check is to detect a large percentage of defects or lack thereof, for example to see if the assemblers are adding a new clamp to an assembly that did not have this clamp in the past, it could be fine.

If you are trying to detect a small percentage of defects (i.e. to determine if a process change eliminated a 2% defect rate) an AQL check is not likely to be effective.
 

JeantheBigone

Quite Involved in Discussions
#5
Sorry!

I should have been clearer that this is an effectiveness check on a quality process procedure, not a production procedure.

Without going into too much detail, we want to do an effectiveness check using records that should have been signed off on by certain authorized persons. If the record was appropriately approved, then the record "passes." If the individual performing the work did not get the proper approval, the record "fails."
 

Al Rosen

Staff member
Super Moderator
#6
Sorry!

I should have been clearer that this is an effectiveness check on a quality process procedure, not a production procedure.

Without going into too much detail, we want to do an effectiveness check using records that should have been signed off on by certain authorized persons. If the record was appropriately approved, then the record "passes." If the individual performing the work did not get the proper approval, the record "fails."
FDA uses this sampling plan for records.
 

Bev D

Heretical Statistician
Staff member
Super Moderator
#7
If you are looking at the compliance of a single person - that it appears you suspect of ‘wrong doing’ - why wouldn't You look at all of them over some period? This doesn’t sound like a situation where any ‘defects’ are allowed and no sampling plan can handle that - just look at all of them.

Again you might think about the time period to check for some conditions that might result in this individual’s non-compliance.
 

JeantheBigone

Quite Involved in Discussions
#8
If you are looking at the compliance of a single person - that it appears you suspect of ‘wrong doing’ - why wouldn't You look at all of them over some period? This doesn’t sound like a situation where any ‘defects’ are allowed and no sampling plan can handle that - just look at all of them.

Again you might think about the time period to check for some conditions that might result in this individual’s non-compliance.
I'm not sure why it appears we suspect one single person of "wrongdoing." This does not describe the situation at all.

This involves a procedure which applies to multiple individuals in one department who had become accustomed to not getting approval from a different department for certain documents. They were made aware of the requirement and given training last year and we want to do an effectiveness check. In the past year there were over 700 such documents, and given the nature of the documents (which I prefer not to divulge) if they have not been signed off on, we do not consider it "critical" but only "major." What I was asking for was an appropriate AQL.

This is not shift work, does not involve operators standing at a machine, and the only person who have to comply are all in the same department.
 

Bev D

Heretical Statistician
Staff member
Super Moderator
#9
my apologies - your description of the nature of not getting the appropriate approvals and the use of the term certain led me to use the air quoted 'wrong doing'. no intention to imply illegal or immoral acts. wrong doing in the sense of not doing the required thing....it is always difficult to discern the nature of vague situations.

so 700 documents. the AQL 'tiers' are related to how critical the defect is. as you know this requires an understanding of the probability of missing the non-compliance vs the effect of the non-compliance should it occur. and all AQL testing has a probability of missing some level of non compliance. This isn't a straightforward thing for the AQL tables as they were negotiated AND they are based on the lot/population size. neither of which result in statistically valid estimates of the probability of missing the non-conformance. This is exacerbated by the fact that non-homogeneity of the non-conformance renders any such estimates unreliable. human behavior is notoriously non-homogenous. it is quite possible that for a short time after the training they all complied. as time goes on and the causal element of what prompted them to non-compliance recurs they are more likely to not comply. as time goes on and they do not get caught they will be tempted to increase their level of non-compliance. Another consideration is that AQL sampling is not intended for one time use - it's probabilities are based on a continual stream of sampling...so again I would advise that AQL sampling is not the appropriate tool for this job. *I* would sample 100% of the documents form a few time periods when they are most likely to have not complied, or from when you have other reasons to suspect that they didn't comply.

However, since you have stated that the non-compliance is a major: in your mind which tier is appropriate for an AQL based sample? then you must determine what defect level is ACCEPTABLE because that is the defect level that drives the sample size within your selected tier.
 

Mike S.

Happy to be Alive
Trusted Information Resource
#10
Sounds like you may want to audit the process a couple different times. As Bev said, you need to weigh how much work (sampling/auditing) you want to do vs. how much risk you can tolerate.

For example, based on the “rule of 3’s” (p=3/n) if you inspected 50 pieces (samples) and found 0 defects you could say you are 95% sure you have </= 6% defects in the lot. Is that acceptable risk? Only you can say.

If you inspected only 20 and found 0 defects you could say you are 95% sure you have </= 15% defects in the lot.

If you inspected 100 and found 0 defects you could say you are 95% sure you have </= 3% defects in the lot. And so forth….
 
Thread starter Similar threads Forum Replies Date
W LTPD, AQL, Ppk and Cpk validation sampling plan table Inspection, Prints (Drawings), Testing, Sampling and Related Topics 0
R AQL, Consumer Risk and MA Statistical Analysis Tools, Techniques and SPC 2
A AQL - How to count samples with defects for each defect class AQL - Acceptable Quality Level 17
T Defining sampling plan for different AQL AQL - Acceptable Quality Level 3
S ARMY AQL - Requirements which don't have an AQL associated with them Manufacturing and Related Processes 2
M AQL table - I need to sample 1250pcs AQL - Acceptable Quality Level 3
Nicole Desouza Sampling plan for a customer who wants AQL 1.0 (per ANSI Z1.4) AQL - Acceptable Quality Level 5
V How to interpret AQL sampling tables AQL - Acceptable Quality Level 5
A AQL sampling bags of parts Inspection, Prints (Drawings), Testing, Sampling and Related Topics 5
I Sampling processes - Who must define the AQL level? AQL - Acceptable Quality Level 9
S AQL Inspector's Rule - bring into CQE Exam? Professional Certifications and Degrees 5
N The question asks about sampling plans and AQL Reliability Analysis - Predictions, Testing and Standards 5
K Defining Acceptance Quality Level, I need clarity on AQL 1.5, 2.5, 4.0 AQL - Acceptable Quality Level 5
V Receiving Inspection AQL - Am I allowed to just pick a AQL? Inspection, Prints (Drawings), Testing, Sampling and Related Topics 10
L Established a c=0 "AQL" or "RQL" table AQL - Acceptable Quality Level 5
A AQL Sampling Plan SOP / WI - Medical Device Industry Inspection, Prints (Drawings), Testing, Sampling and Related Topics 14
C Example Work Instruction/Procedure for AQL (Acceptable Quality Level) AQL - Acceptable Quality Level 4
L How to calculate the Probability of Deficiency of the AQL of an Inspection AQL - Acceptable Quality Level 16
P Setting Acceptance Quality Limits (AQL) for New Medical Devices ISO 13485:2016 - Medical Device Quality Management Systems 4
K AQL for Sterile Product - Laminate Tubes 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 5
M How to Mathemetically Derive the AQL Tables given Lot Sizes AQL - Acceptable Quality Level 5
L AQL ANSI z1.4 & z1.9 - What are the differences between these 2 standards? AQL - Acceptable Quality Level 5
P RQL/AQL from Attribute Data Six Sigma 1
B Designing a Sampling Plan - AQL AQL - Acceptable Quality Level 19
S Determining Sample Size - AQL & LQ AQL - Acceptable Quality Level 10
V AQL for Component Level and Assembled Finished Product AQL - Acceptable Quality Level 1
alonFAI AQL and Sampling Size for Components Solderability Test Statistical Analysis Tools, Techniques and SPC 3
A Inspection Plans - How to choose AQL (MIL STD 105E) Excel .xls Spreadsheet Templates and Tools 1
U AQL, Lot Size vs Sampling Plans AQL - Acceptable Quality Level 8
R Formula to tie together - Confidence/Reliability Level AQL? LTPD? Reliability Analysis - Predictions, Testing and Standards 3
P Which Sampling Tables for AQL? Statistical Analysis Tools, Techniques and SPC 6
K Can I use ANSI/AQL for the Total Number of Containers for the entire Lot? Inspection, Prints (Drawings), Testing, Sampling and Related Topics 1
M Determining c=0 AQL numbers Misc. Quality Assurance and Business Systems Related Topics 3
L Who and when to determine AQL for our product inspection AQL - Acceptable Quality Level 1
K Verification Sampling vs. AQL - Sampl Sizes Inspection, Prints (Drawings), Testing, Sampling and Related Topics 1
E Translating AQL to a Production Line Inspection Plan AQL - Acceptable Quality Level 14
S Class III Sampling Plan - Explain AQL vs. 95%C & 90%R Inspection, Prints (Drawings), Testing, Sampling and Related Topics 1
K Training the absolute beginner to understand AQL Training - Internal, External, Online and Distance Learning 1
S What are typical Acceptable Quality Levels (AQL) adopted in Food industry? Food Safety - ISO 22000, HACCP (21 CFR 120) 1
D What AQL level to use to test LEDs Inspection, Prints (Drawings), Testing, Sampling and Related Topics 5
B AQL Sampling Tables Question Inspection, Prints (Drawings), Testing, Sampling and Related Topics 4
C Constructing an AQL matrix for our QC operators in a flexographic printing outfit AQL - Acceptable Quality Level 4
N Visible particles inspection in powders for injections. Approaches to AQL level Pharmaceuticals (21 CFR Part 210, 21 CFR Part 211 and related Regulations) 1
S Justifying the AQL for a Zero (C=0) Acceptance Plan (Medical Device) AQL - Acceptable Quality Level 6
W Application of AQL (Acceptable Quality Level) Concept AQL - Acceptable Quality Level 15
P Deviating from AQL in a small Retail Apparel Company AQL - Acceptable Quality Level 5
M AQL (Acceptable Quality Level) for Mixed Part Defect IATF 16949 - Automotive Quality Systems Standard 1
S AQL (Acceptable Quality Level) Report Form example wanted AQL - Acceptable Quality Level 1
H Determining the Correct Sample Size when AQL points to two Sample Sizes AQL - Acceptable Quality Level 7
F AQL - ISO 3951-4:2011 Sampling Plan Requirements AQL - Acceptable Quality Level 2

Similar threads

Top Bottom