Unrealistic Packaging Validation Sample Size

PutDownBoars

Registered
Hello all,

I'm working for a small med device company where my current project is to perform the packaging validation necessary to support a 510k submission. I performed a risk analysis of the packaging, and based on the risk I've determined that the sampling plan will need to show 95% confidence that 95% is conforming/pass the ISTA 2 Series: Partial Simulation Performance Tests.

To achieve the requirements of the above sampling plan, I've calculated that I need a sample size of n=59. 59 packages is a massive sample size! If I were packaging a prohibitively expensive device such as a robotic surgical unit, how could the FDA realistically expect me to test 59 packaged products without burning through all my company's cash? I've spoken to some colleagues at some of the larger established med device companies, and they've told me that they've been able to get good packaging validation results for their products with only 15 samples, though they've been unable to provide me with how they provided valid rationale for that smaller sample size.

I've heard of companies using 3 or 5 samples as the magic number for verification/validation, so I feel like I'm missing something here? I'd appreciate anyone who can provide input.
 
What is the risk you are concerned about? If your device is very large, you probably won't be shipping your device under a regular shipping method. How will your device be shipped? Do you have a sterile barrier? There may be workarounds for doing the package testing, but you have not provided much info to go off of. For small devices, it is common to have 30 to 60 units undergo packaging validation. Not all units need to be fully functioning devices though.
 

yodon

Leader
Super Moderator
We've always generally assumed that there really isn't any variability in packaging (of course, that's assuming it's done per spec) so the sample size has always been 1. We engage with FedEx to put it through their tests, get a report, used it as validation, and have yet to get any push-back from reviewers / inspectors.

Not to say that is bulletproof. I'm curious, though, what would the rationale for using more be?
 

PutDownBoars

Registered
What is the risk you are concerned about? If your device is very large, you probably won't be shipping your device under a regular shipping method. How will your device be shipped? Do you have a sterile barrier? There may be workarounds for doing the package testing, but you have not provided much info to go off of. For small devices, it is common to have 30 to 60 units undergo packaging validation. Not all units need to be fully functioning devices though.


The biggest risks we are concerned about with the packaging relate to:
  • The product inside the packaging getting damaged (which is why we'll be doing vibration/drop tests)
  • The primary package breaking its seal, which may lead to the product becoming unclean (which is why we're doing bubble burst test)
  • The product/packaging weakening because of extreme conditions during transport (which is why we're doing environmental conditioning)
Our device is large-ish (size of a human skull) and will be shipped using a standard single parcel delivery system. It is cleaned before packaging, though it is not provided sterile and will be sterilized by the end-user prior to use. Therefore we don't need a sterile barrier, though there is the risk of the product becoming "unclean" if the packaging opens.


We've always generally assumed that there really isn't any variability in packaging (of course, that's assuming it's done per spec) so the sample size has always been 1. We engage with FedEx to put it through their tests, get a report, used it as validation, and have yet to get any push-back from reviewers / inspectors.

Not to say that is bulletproof. I'm curious, though, what would the rationale for using more be?


Well the tests are pass/fail attribute data, so I'm surprised that regulators have accepted a sample size of 1. What if you were to test it again and the package broke? That would be a 50% conformance rate for a sample size of 2. I don't know how you can assign any type of confidence statement there, which is confusing to say the least. Is the risk of packaging failing considered "low" in your packaging & shipping risk analysis/PFMEA?
 

Steve Prevette

Deming Disciple
Leader
Super Moderator
Certainly 3 to 5 samples is insufficient. However, one can ask - at what rate should we be sampling to get to the "magic" number of 59 samples without failures? Could you do 3 to 5 per week? per month? per quarter (well, that might be too slow) . . . Keep in mind that this is basically a very common c=0 test, but ANY failure within the 59 implies a failure of the overall test.
 

yodon

Leader
Super Moderator
The way we've looked at it is that if the shipping test fails, the packaging is not sufficient to adequately protect the device (and that would be the case if one failed out of any number of samples). Indeed, historically, the devices we worked with have fairly low risk for shipping.
 
The number of samples for packaging validation is directly related to the level of risk. If your medical device has a sterile barrier and is an implantable device, a minimum of 29 samples is expected, and 59 units is preferred. If your device is not sterile and is a low risk device that is unlikely to be damaged during shipping, you are going to need a lot fewer samples. There should be a rationale for sample sizes that fits in with your product development and risk management plans. If your device undergoes some sort of installation activity, this could affect risk, and therefore, the sample size selected for package testing. If your device is a multiple-use device, you can obtain your sample size by reusing the sample devices for multiple rounds of testing. As you can see, there are a lot of factors that go into this.
 

planB

Super Moderator
Concurring with yodon ("there really isn't any variability in packaging"), shipping tests can be regarded as Type Tests with a sample size = 1. This is also permissible by ISTA 2A itself as minimum sampling (one samples, no replicate testing).

In my experience, this approach has been accepted for submissions of sterile Class III devices worldwide.

HTH,
 
Concurring with yodon ("there really isn't any variability in packaging"), shipping tests can be regarded as Type Tests with a sample size = 1. This is also permissible by ISTA 2A itself as minimum sampling (one samples, no replicate testing).

In my experience, this approach has been accepted for submissions of sterile Class III devices worldwide.

HTH,
I find this hard to believe for a Class III sterile medical device. You simply cannot have statistical significance with a sample size of 1. Can you tell us which country accepted this, what year, and what kind of packing material this was?

Also, ISTA 2A is not a US FDA consensus standard, so I would caution anyone from using this standard for a US FDA submission.
 

Bev D

Heretical Statistician
Leader
Super Moderator
Of course you can have statistical significance (whatever you think that is) with a sample size of 1. I do it quite frequently. Packaging strength tests are not the same as typical validation testing where the variation of the process, materials and products must be assessed by statistical means. In the case of packaging strength we have a stress-strength system. The variation of the packaging and product strength is quite small in comparison with the variation of the stresses that might be experienced. So it makes physics - and variation - sense to test 1 package with product in it under the worst case stresses. If it survives then the design has plenty of margin for the range of variation of stresses. Of course when the packaging and product variation might be larger than I’d like I may use the weakest combination of strength and product to test at the worst case conditions. And yes, I. May need. More than 1 unit to test if I have to perform several different destruct tests, but still 1 unit per test. (This by the way is the whole concept of OQ validation at the product level). It is this concept that is applied and accepted by most standards and regulatory agencies as it makes sense. Simply applying “statistical sampling plans” to all situations is black box thinking. Remember that statistics must match the physics of the situation. There is more than one kind of statistics in the world...
 
Top Bottom