Sampling plan for destructive tests on small lots

Statistical Steven

Statistician
Staff member
Super Moderator
#11
Hi all

I was wondering if someone could maybe help. We are a medical device manufacturing company that needs to revise our sampling plans on our finished products. At the moment we have a AQL of 1% (which we were advised might be a little high), by reducing the AQL the amount of samples for testing increases.

We are currently using ISO 2859-1 for our sampling plan and according to this if we use Inspection level II and an AQL of 1% we need to send 13 samples for destructive testing, however the problem with this is that we make batches of quantity 20.

Does anybody have any advise?

Thanking you in advance. :)
First things first....if you only make a quantity of 20, the smallest AQL you can have is 5% (1 of 20 is defective).

Let's start from the beginning. You validated your process originally at a certain risk level. Assuming you passed, you have confidence that your process is capable of meeting the requirements at that level of risk. With small batch sizes, you cannot commit a large sample size to destructive testing, and it is inappropriate to do sampling on small lot sizes as the AQL is a long term process average. So how do we approach this problem? Since you said you use 2859-1, it is an attribute test, can it be converted to a variable output? Are your lots of 20 truly independent? If not, can I do sampling across multiple lots to gain confidence? Can a sample be tested more than once (in other words, if it is destructive like a pull test, can I cut my sample and test it twice)?

Now, the advanced part of the discussion. Sampling plans are risk based concepts which includes both the AQL and LTPD. If we exhausted your entire lot of 20 units, we would only have 95% confidence with 86% reliability, but of course we have 100% confidence since we did 100% inspection. So sampling plans are not intended for small lots as their OC curves to not reflect the actual state. You can use the Hypergeometric, but I doubt that will help.

In general, you must ask yourself, what is the risk? If I sample one piece per lot and it passes, what is the likelihood that the other 19 will fail? What is the risk that if one piece passes the other 19 are good? I rely on my validation data to help to define my risk.
 
Elsmar Forum Sponsor

enorht

Starting to get Involved
#12
I am working on a sterile packaging validation as per ISO 11607. I have already defined my OQ and PQ tests:
- Performance tests (ISTA 2A)  Attribute data
- Sealing strength (ASTM F88)  Variable data
- Pealability test (EN 868-5)  Attribute data
- Dye penetration (ASTM F1929) )  Attribute data
- Sterility test (European Pharmacopeia) )  Attribute data

My routine batch size is from 10 to 90 around. In the past, we had a sample size of 5 per test which was not justified by a valid statistical rational. We got a non-conformity from our notified body. So now, I try to find a sample size statically acceptable and economically feasible. Available data: last validation but not performed on the worst-case, 1 pealability test performed per batch.

Which sampling plan can I use for my attribute tests and my variable tests? If I use the ISO 2859-1 (as mentioned in a note in ISO 11607), I have a sample size of 13 for a batch size of 51-90 and a level of II and a AQL of 1%... If I want to use a AQLof 0.1%, I need a sample size of 125 per test which is not feasible…

Thanks,
 

Mike S.

Happy to be Alive
Trusted Information Resource
#13
Much of the advice given earlier to the original poster can apply to your situation as well. Read it over carefully.

For example, what do your customers (internal and external) require as far as testing and expected quality levels? What process controls do you have in place, and how well do they work? What does your historic sampling data look like? What is the failure rate in the field?

If you want to use an "AQL-type plan" you are going to need to increase the AQL level to cut your sample sizes. Maybe consider a C=0 plan at 10% AQL -- there your sample sizes would be 2-4 and you could at least show OC curves and calculate some statistical quality parameters like AOQL, ERP, or LQ. Take a look a Zero Acceptance Number Sampling Plans, Fifth Edition, by N.L. Squeglia, ASQ Quality Press.
 

Bev D

Heretical Statistician
Staff member
Super Moderator
#14
So the issue here is a lack of critical thinking. sorry to be so blunt but I do want to pull everyone up from following the AQL ritual and really think about what the true need is. As Quality professionals we too often blindly revert to the same old things and don't really think about them. the two OPs here are asking really good questions. they are thinking - something is amiss and they can sense it. they just don't know how to get an answer that makes sense.

First AQL sampling (and it's better sibling RQL sampling) is NOT intended for validation (except during PQ) and it certainly isn't intended for small batches or special processes. It only works (and even then not very well) when the defects are randomly distributed and are therefore from a process with a random distribution. It is also intended for an ongoing process and not a single event like validation. The nature probabilistic nature of this type of sampling plan means that the process should also be better than the AQL/RQL level most of the time. When the process is at the AQL/RQL defect rate one can't say with certainty that any given lot meets the AQL/RQL criteria - some will and some won't. Additionally even if you trust the statistical probabilities you can really only say that the sample shows that the process isn't worse than some level but it doesn't tell you how good it is. And then the BIG issue is that AQL plans (even those that aren't based on the lot size (another zombie fallacy) result in HUGE sample sizes for today's quality levels of less than 1%. This is certainly then not appropriate for validation.

There are many different approaches that can be taken here that are statistically based, including those fro a deterministic process. we can use directed testing and matched pairs etc. also continuous data instead of attribute data. but we'll need more details about the process...

And yes I hear the howls: but the 'agency' wants the AQL thing! they really don't. some may be zombies and adhere to rituals since they are easy and don't require thinking but I've found that most of the time there is someone who gets it and really wants to ensure good quality. A well presented case based on physics, data and the appropriate statistics usually settles the question.
 

enorht

Starting to get Involved
#15
Hi,

I have no issues not using ISO 2859-1. I know that it is not adapted for process validation (OQ and PQ).
My principai aim is respect the ISO 11607-1 "Sampling plans should be based on a
valid statistical justification." to validate my packaging process. (bag sealer, vacuum sealer...).

I have 3 tests to be performed and i need 3 batches (yes, i know that 3 batches are not a good practice either). So it means that i will need 3 X 3 X sample size... And it will be destructive for my items.

The issue about the validation is that the last validation is a mess : no traceability of the test article (we do not know what was tested), no process monitoring.

What I did, i perform a PFMEA and i already identified my critical risks. If I want to use the ISO 2859-1, i classify these risks as AQL 0.1% and i will need 125 samples per test.
If I use the C=0 sampling plan with confidence at 95% (Beta risk) and a LTPD of 0.1% (i am not very sure how to define the LTPD value), i get 52 samples which is still too much. As i want to validate my process with critical defects, AQL of 10% is not acceptable.
By the way is it possible to divide the sample size by the three lots of PQ?

Do you have any other sampling plan which is adapted to my case (OQ and PQ validation of special process : packaging, no monitoring data, small batch size, tests to be performed: pull test (quantitative), dye penetration (attribute), visual inspection (attribute), pealability (attribute))? This sampling plan must be based on a valid statistical justification

here the tests to be performed with the acceptance criteria and the old sampling plan (without justification, so i need one or i need to change the sample size):


Thanks,
 
Last edited:

Statistical Steven

Statistician
Staff member
Super Moderator
#16
Just an FYI, a sampling plan that is explicitly called out in an internationally recognized standard does not need additional justification.

Hi,

I have no issues not using ISO 2859-1. I know that it is not adapted for process validation (OQ and PQ).
My principai aim is respect the ISO 11607-1 "Sampling plans should be based on a
valid statistical justification." to validate my packaging process. (bag sealer, vacuum sealer...).

I have 3 tests to be performed and i need 3 batches (yes, i know that 3 batches are not a good practice either). So it means that i will need 3 X 3 X sample size... And it will be destructive for my items.

The issue about the validation is that the last validation is a mess : no traceability of the test article (we do not know what was tested), no process monitoring.

What I did, i perform a PFMEA and i already identified my critical risks. If I want to use the ISO 2859-1, i classify these risks as AQL 0.1% and i will need 125 samples per test.
If I use the C=0 sampling plan with confidence at 95% (Beta risk) and a LTPD of 0.1% (i am not very sure how to define the LTPD value), i get 52 samples which is still too much. As i want to validate my process with critical defects, AQL of 10% is not acceptable.
By the way is it possible to divide the sample size by the three lots of PQ?

Do you have any other sampling plan which is adapted to my case (OQ and PQ validation of special process : packaging, no monitoring data, small batch size, tests to be performed: pull test (quantitative), dye penetration (attribute), visual inspection (attribute), pealability (attribute))? This sampling plan must be based on a valid statistical justification

here the tests to be performed with the acceptance criteria and the old sampling plan (without justification, so i need one or i need to change the sample size):


Thanks,
 

enorht

Starting to get Involved
#17
Statistical Steven, i explain in my post that the sample size is not justified. No sample size is explicitly called in these standards, just the test methodology. That's why i need to justify them or to design another sample sizes.
 

Statistical Steven

Statistician
Staff member
Super Moderator
#18
Statistical Steven, i explain in my post that the sample size is not justified. No sample size is explicitly called in these standards, just the test methodology. That's why i need to justify them or to design another sample sizes.
My apologies, I assumed the the sampling plans in your table were from the standard. ASTM usually refers to ASTM 133 for sample size calculation, though again it needs to be justified. If the standard has an acceptance criteria without a sample size, many organizations have assumed (right or wrong I will not opine), single unit sampling. That is if the standard says must support a minimum of 200 lbs, they would assume that is just test a single unit, if it supports 200 lbs you've met the standard. I typically would use a tolerance interval to tell me what percent of the units will meet the specifications at a given confidence.
 

Mike S.

Happy to be Alive
Trusted Information Resource
#19
Maybe I am inferring what was not implied, but it sounds to me like perhaps enorht has used a sampling plan that works for him and his process but he doesn't have "valid statistical justification" for it.

Some customers/standards require that when you use sampling you formally “justify” the sampling plan somehow, and sometimes this justification includes calculating quality parameters (ERP, LQ, AOQL, etc.) or showing other statistical characteristics of the plan.

If you have a plan that has proven to work for you in real life use, maybe all you need to do is show an auditor that you have calculated descriptive statistics on the plan, and that it was approved by the appropriate persons. This may be going about things in reverse, justifying the plan after selecting and using it, but you are where you are and you need to move forward from here.
 

Wolf.K

Quite Involved in Discussions
#20
Dear enorht,

I wonder if you came up with a solution for the number of sample tests you have to perform for validation of ASTM F1929?

I want to use this test in our new laboratory, but unfortunately the standard (as so often) does not state any sampling numbers.

:) Wolf
 
Thread starter Similar threads Forum Replies Date
V Sampling Plan for Destructive Testing Inspection, Prints (Drawings), Testing, Sampling and Related Topics 2
M Manufacturing Process Validation Destructive Testing Sampling Plan Advice Inspection, Prints (Drawings), Testing, Sampling and Related Topics 1
A Destructive Testing Failure Mode Sampling Plan Inspection, Prints (Drawings), Testing, Sampling and Related Topics 5
A Sampling plan for destructive test Inspection, Prints (Drawings), Testing, Sampling and Related Topics 2
E Sampling Plan for Destructive Testing - Personal Body Armor Plates Inspection, Prints (Drawings), Testing, Sampling and Related Topics 8
S Which Sampling Plan(s) Should I Use? Inspection, Prints (Drawings), Testing, Sampling and Related Topics 7
L Sampling Plan Risks AQL - Acceptable Quality Level 6
A Sampling plan for in-process QC (medical devices) Inspection, Prints (Drawings), Testing, Sampling and Related Topics 13
W LTPD, AQL, Ppk and Cpk validation sampling plan table Inspection, Prints (Drawings), Testing, Sampling and Related Topics 0
E Change in control plan - Do I have to do sampling? IATF 16949 - Automotive Quality Systems Standard 1
S Is using ANSI/ASQ Z1.4-2008 the correct sampling plan to determine Pass/Fail of Apparel measurements? AQL - Acceptable Quality Level 4
N Sampling Plan for Internal Audits - ISO 2859 or 3951 - Or Neither? Internal Auditing 6
R Reduced sampling plan for sterial products APQP and PPAP 0
T Defining sampling plan for different AQL AQL - Acceptable Quality Level 3
B AS9100D Sampling Plan AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 2
Nicole Desouza Sampling plan for a customer who wants AQL 1.0 (per ANSI Z1.4) AQL - Acceptable Quality Level 5
M Sampling Plan for Alumin High Pressure Die Castings Manufacturing and Related Processes 0
E Sampling plan for orthopedic implant - Process Validation Inspection, Prints (Drawings), Testing, Sampling and Related Topics 4
S How to make a Sampling Plan - Conveyor belts Inspection, Prints (Drawings), Testing, Sampling and Related Topics 4
S Help develop sampling plan for prototype samples (medical devices) Statistical Analysis Tools, Techniques and SPC 7
M Zero Defect Sampling Plan Standards Inspection, Prints (Drawings), Testing, Sampling and Related Topics 4
L ISO 2859-2 LQL sampling plan linked to %confidence/%reliability Inspection, Prints (Drawings), Testing, Sampling and Related Topics 2
K Sampling Plan for transparent tubular plastic parts from 32 different cavities Inspection, Prints (Drawings), Testing, Sampling and Related Topics 3
A AQL Sampling Plan SOP / WI - Medical Device Industry Inspection, Prints (Drawings), Testing, Sampling and Related Topics 14
J Sampling Plan Questions from the CQE Practice Exam Inspection, Prints (Drawings), Testing, Sampling and Related Topics 2
M Sampling Plan for for Chemical Analysis Testing of Medical Devices EU Medical Device Regulations 3
J Can ANSI Z1.4 sampling plan be used for sampling alone without the Acc/Rej criteria? Inspection, Prints (Drawings), Testing, Sampling and Related Topics 5
B Designing a Sampling Plan - AQL AQL - Acceptable Quality Level 19
N Use of arrows in ISO 2859-1 Table 1 and 2-A sampling plan Inspection, Prints (Drawings), Testing, Sampling and Related Topics 1
S AS9100 Sampling Plan Requirements - 8.2.4 AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 5
S Military Sampling Plan 105E vs. ISO 2859-1 AQL - Acceptable Quality Level 2
kedarg6500 Acceptance Sampling Plan for Variables - Product Performance Testing Inspection, Prints (Drawings), Testing, Sampling and Related Topics 5
S Where could I get reference to an OQA Sampling Plan of 1 per 1000 ? Quality Manager and Management Related Issues 4
R What to Report when Using C=0 Sampling Plan Supplier Quality Assurance and other Supplier Issues 15
R Results: Monte Carlo Simulation of Acceptance Sampling Plan AQL - Acceptable Quality Level 3
G Sampling Plan according to SAE ARP9013 C=0 Inspection, Prints (Drawings), Testing, Sampling and Related Topics 1
G Sampling Plan for Inspection of Metal Parts after CNC Machining Inspection, Prints (Drawings), Testing, Sampling and Related Topics 34
S Class III Sampling Plan - Explain AQL vs. 95%C & 90%R Inspection, Prints (Drawings), Testing, Sampling and Related Topics 1
G Sampling Plan for a Performance Qualification of an Inflation System Qualification and Validation (including 21 CFR Part 11) 2
S How to establish a Sampling Plan from the Rework percentage? AQL - Acceptable Quality Level 6
Q Sampling Plan for Paper and Tape Industry Inspection, Prints (Drawings), Testing, Sampling and Related Topics 7
C Medical Device Tracking Sampling Plan using MIL-STD-105E Other US Medical Device Regulations 4
P Lot Acceptance Plan that specifies BS 6001 Double Sampling Plans, general inspection Inspection, Prints (Drawings), Testing, Sampling and Related Topics 1
B Best Attribute Data Sampling Plan - Early Production Containment Inspection, Prints (Drawings), Testing, Sampling and Related Topics 2
B In-Process Sampling Plan for a Secondary Package Adjustment Process Inspection, Prints (Drawings), Testing, Sampling and Related Topics 1
Proud Liberal Making a simple Sampling Plan by Variables Inspection, Prints (Drawings), Testing, Sampling and Related Topics 2
E How to determine the Lot Size for the OQ Sampling Plan Inspection, Prints (Drawings), Testing, Sampling and Related Topics 11
L Electrical Cable Harness Quality Control - Sampling Plan for Pull Test Manufacturing and Related Processes 4
G Sampling Plan - Sample Size Justification for a Bottling Cleaning Process Validation Qualification and Validation (including 21 CFR Part 11) 2
I Alpha and Beta in the Sampling Plan for ISO 2859 & ISO 3951 Inspection, Prints (Drawings), Testing, Sampling and Related Topics 13

Similar threads

Top Bottom