Use of statistical techniques for Process Validation

[Mastapain]

Hi All,

We have received a Minor NC by Notified Body as follows:
"The process validation performed by the manufacturer does not follow the requirement of the ISO 13485:2016 standard to use statistical techniques and rationale for sample sizes. The sample size selected for the above mentioned validation report is not justified. The procedure (SOP) related to process validation is also not addressing the requirement for the usage of "statistical techniques with rationales for sample sizes"."

While I do not argue we should have a statement in Validation SOP on statistical techniques use if apropiate as one of the solutions, it seems for me a ISO 13485:2016 standard does not require that as obligatory to be used (see a standard quote below: section 7.5.6) and our "mistake" was rather we have not applied justification for sampling plan we chose during validation rather than not using actual statistical techniques.

Section 7.5.6 of ISO13485:2016 quote:

The organization shall document procedures for validation of processes, including:
(…)
d) as appropriate, statistical techniques with rationale for sample sizes(...)"

Having said that:
Is my way of thinking correct that ISO13485:2016 does not make obligatory to use statistical techniques for sampling plans but is leaving it as option?
If so what conditions sampling plans need to meet in order to comply with standard yet not using statistical techniques actually?

Thanks

 

[Steve Prevette]

Two thoughts. "As appropriate" I will admit has some vagueness to it - but an auditor can still ask - what is the logic you used to choose to "not use statistical techniques". Personally, I will say - I don't see how you can do a sample without some form of "statistics". It would help to know exactly what your plan called for. For example, I know there is a common but flawed general statement by non-statistical folks - sample 10% and you'll be fine. One time where it might not be appropriate is if you did a 100% sample.

 

[Mastapain]

Thanks Steve,

Let me think of an example:

We plan to validate the process that is very similar/ or close to identical to one used in our plant for decades.
And for all this time we have used the sample size that was not established with statistical techniques (there was no such strict statement in place then).
So validating now new process I might refer to sampling plan used back then and present data demostrating there was not major quality escapes, process is stable and under control, no medical incidents and no significant risk for customer detected since then up to now.
So in the nutshell even thought it is not statistically significant seems to be applicable and working well.
Can I present it as rationale for selected sampling size for validation?

Thanks

 

[Zero_yield]

I think you're kind of conflating two different topics.

Can you partially lean on comparing on long term data vs. new process data showing stability as part of the evidence showing your new process is under control? Yes, absolutely.

Should you have a firm rationale for whatever sample sizes you choose for new studies? Also yes.

The fact that the process is working well now does not mean that the sample size chosen back then was appropriate. Maybe the sample size was drastically too small and the data generated was essentially meaningless. Maybe the sample size was drastically too large, and you'd waste a lot of time and effort trying to repeat an enormous study that could be completely satisfactory with a much smaller sample size.

Once you create, justify, and document a reasonable rationale, then you can keep going back to the old well for future studies (assuming the needs of the new studies match up with the documented rationale).

 

[Tidge]

"as appropriate" has some intentional vagueness, but generally such a statement is to be interpreted as "as appropriate, and if not appropriate document the reason why it is not appropriate."

In the case of Process Validations, I agree with @Steve Prevette that some form of "statistics" (derived from a hypothesis test of some sort) will always be behind the choices made in sample sizes... for a well-established process there may be a different hypothesis being tested than for a never analyzed process. It is typically valuable to incorporate (appropriate) historical data into the study designs which establish sample sizes.

From my experience in Process Validations, the simplest "easy outs" I have used for using the "as appropriate" clause have been when:

1. The lot size was smaller than what the study design would require; this simply leads to 100% (no sampling)
2. The method of determining conformance would be destructive of 100% of process outputs; this requires some thought and analysis (and generally a reconfiguration of the null hypothesis) and/or use of a consensus methodology which establishes sample sizes.

In the second case, my mind always goes to something like destructive ASTM test methods for materials and product classes.

 

[Steve Prevette]

Some good responses, and thanks for the example of the issue. I would say that using an "outdated" (not meeting current requirements) sampling plan would not pass muster. I would suggest asking - how many more "outdated" sampling plans are out there, and what is the risk if they are re-used?

Now, it does sound like you have some decent data. What I would do is go back to the original sample size and results and backfit the statistical significance. Especially assuming if the sample found no defects, you can take the sample size and say - I am 5% confident (or in a pinch you could go to 10%) that no more than X percent are defective. That at least let's you know where you are at (and lets the auditor know that you know). And if you have produced some of the new mod since then, throw that in.

 

[Mastapain]

Thanks All,

That makes great sense and brings more light to crude text of standard.
We would perform assessment on how many sample plans are there not meeting current requirements and backfit statistical difference excercise for available historical data.

Regards

 

[Ed Panek]

You are correct that ISO 13485:2016 does not make it obligatory to use statistical techniques for sampling plans, but it does state that the organization shall document procedures for validation of processes and, as appropriate, use statistical techniques with a rationale for sample sizes.

This means that while statistical techniques are not required, they are considered a best practice and should be used when appropriate. The standard leaves the determination of when statistical techniques are appropriate to the discretion of the organization.

If an organization chooses not to use statistical techniques for sampling plans, it should have a clear rationale for this decision and should ensure that the sample size is sufficient to ensure that the process is validated, and that the product meets the requirements for quality and safety.

The sampling plan should be based on the product and process characteristics, the intended use of the product, and the level of risk associated with the product. It is important that the sampling plan is reviewed and approved by the organization's quality management representative and that it is followed consistently throughout the process.

Additionally, ISO13485:2016 requires the organization to establish, implement and maintain procedures for the validation of processes, these procedures should be suitable for the specific process and should be based on risk management. Therefore, the organization must have a process validation plan which should include the sampling plan, and that must be reviewed and approved by the relevant parties.

 

[planB]

To my knowledge, there are only two situation when you can be confident that you are not challenged for a "non-statistical" sampling plan, namely when

1) a standard gives you "hard" sample sizes, and/or
2) you validate when your process "fails" and thus, providing additional assurance under which conditions the process actually "passes".

Examples are scare to my knowledge and typically sterilization-related:
- ISO 11135 for EO (ethylene oxide) sterilization tells you to perform consecutively at least 2 (worst-case) half cycles and 1 (nominal) full cycle when using the half-cycle overkill approach to demonstrate reproducible cycle effectiveness; you also have to perform at least 1 additional sub-lethal cycle, which is actually an intended "fail" process.
- ISO 11137-2 for gamma irradiation tells you as one option to validate the process according to the "VDmax25 method" by irradiating 10 products out of 3 batches, respectively.
- ISO 10993-7 for EO residuals ("aeration process") tells you as one option for release of product to establish a dissipation curve of gaseous EO residuals out of a minimum of 3 data points gained over different aeration times. With the empirical knowledge that dissipation of EO typically follows first-order kinetics, you would then extrapolate the resulting exponential curve, that may include "fail" data, until you hit the (time) point from where on your product would "pass" EO residual limits.

 

[somashekar]

Thanks All,

That makes great sense and brings more light to crude text of standard.
We would perform assessment on how many sample plans are there not meeting current requirements and backfit statistical difference excercise for available historical data.

Regards

The standard text is not so crude. If you see the clause 0.2 - Clarification of concepts ., you can find
When a requirement is qualified by the phrase “as appropriate”, it is deemed to be appropriate unless the organization can justify otherwise. A requirement is considered appropriate if it is necessary for: — product to meet requirements; — compliance with applicable regulatory requirements; — the organization to carry out corrective action; — the organization to manage risks.
I have seen cases where the Organization has not used statistical technique and provided due written justification.
I have seen cases where the machine capacity for a load is the number of samples used and the same is justified and documented.
What is your's ... ??
Good Luck