Defining Reliability and Confidence Levels

M

mschaller

Hello,

I've always worked at medical device companies where an SOP (Standard Operating Procedure) defines the reliability and confidence levels for me based on a given risk index. For example, based on the output of an FMEA I will have the estimated severity, occurrence, and detection which combine to form some risk index (sometimes an RPN is used). And based on that, I look at a table in the SOP which says I need .9900 reliability at 95% confidence, or perhaps it's a different level.

How are these levels defined? Is there a standard which defines what typical reliability/confidence levels should be used?

Typically, I have used .99 reliability as tightest control, .975 reliability as medium control, and .95 as loosest control (all with 95% confidence).

What do other companies use?

Thanks for the help!
 

Bev D

Heretical Statistician
Leader
Super Moderator
:bigwave: welcone to the cove!

there is no set standard for the choice of reliability and confidence levels. some companies will create their own standard. some individuals (such as a regulatory statistical reviewer or a QA manager) may have their own 'favorites'.

the tradition of using 95% confidence - or 5% alpha risk - dates back to Sir Ronald Fisher. Although this is often misquoted as Fisher suggested that a 5% alpha risk woudl be sufficient for an analysis IF the experiment were replicated several times with the same results being statistically significant at a 5% alpha level each time. The Mil Std acceptance tables used a confidence level of 95% for AQL based plans and since then 95% has been traditionally used for confidence levels. Reliability is not traditionally anchored...
 
M

mschaller

Thank you Bev, that is very helpful. It sounds like keeping confidence at the industry standard of 95% is the safe thing to do, but reliability levels have a little more wiggle room.

If anyone else has any thoughts about what reliability levels they've seen across companies, chime in!
 

Bev D

Heretical Statistician
Leader
Super Moderator
Remeber that Reliability (in the context of sampling plans based on a stated reliability at a stated confidence level) has a very specific definition: it is the minimum acceptable quality level. OR better stated it is 1-the defect proprotion that you wish to reject.

These plans typically require no defects in the sample. We then say "the process - or lot - has a quality level of XX% with XX% confidence." where the 'quality level is the reliability. in essesnce this is a c=0, RQL sampling plan.

if you pick a reliability of 90% you are saying that you will accept up to 10% defects...

if you are dealing with true reliability testing there is a time - or stress - component to the test and you are saying that at least XX% of the product will still be working (will be reliable) at the stress or time point with XX% confidence.


typically the reliability is chosen by your customer OR is dictated by what you want ot claim or can afford in your warranty program.

The choice of reliability is not 'customary', 'standard', mythological, or what others do, unlike the confidence level. It has a meaning and it must be based on what is important to you....
 

Apex Hao

Starting to get Involved
Dear covers, could I kindly ask for some opinions here :) This is the closest thread that I could find.

We have a part which needs to pass a mechanical robustness test. The test is destructive. The risk classification is major, thus we are following our aligned target of 95% confidence for 95% reliability.
  • During validation, we have tested 59 samples and no failure is found. We are stating that our part is at least 95% reliable with 95% confidence.
  • After few months, during a routine surveillance checking (3 samples every 3 months), one of the samples is failing the the test (though not terribly failing, it is still a fail). I am in the dilemma if we should stop the production and determine if something is wrong. Essentially, could I expect all the surveillance checking samples to pass the test, since previously we are only validating to 95% reliability, and this failure is perhaps from the balance 5%?
Thanks in advance for your response.
 

Bev D

Heretical Statistician
Leader
Super Moderator
well 3 is a pretty small sample on it's own. a 95% reliability means that you could have up to 5% defective. so you might take the 1 failure and divide by all of the samples you have done since validation - but this probably won't tell you much. categorical data ahs so little resolution.

Since you said that the part is "only failing a little" I assume you are measuring continuous data of some sort and then just comparing the results to ta specification for pass/fail? You will have much better insight and response if you plot the continuous data in time series. you could try a control chart with n=3 subgroups. But it's better at first to plot all of the individual data points in time series in mulit-vari format. this will show you if the process is stable and the one failure is just to be expected on occasion , or if you have a shift or trend that is a result of a loss of process capability.
 

Apex Hao

Starting to get Involved
Hi Bev, thanks for the reply!

The testing unfortunately is a qualitative one - we subject the part to mechanical impacts and then assess its functionality. Sometimes the part deforms/cracks/shifts and causes unwanted leakage. A leak in this case is a failure (unlikely to quantify), but from observation we know that the part isn't damaged too seriously (again such damage is unlikely to quantify).

I agree with you that 3 sample is pretty small and it might take years to collect sufficient sample to obtain a meaningful percent defective. After this failure occurrence I might try to increase the surveillance checking frequency, and perhaps implement some sort of proper process control.

I love the idea of monitoring the process by having at least a run chart, however my hands are pretty much tied with such categorical pass/fail data. I don't expect a proportion control chart to be much useful here, isn't it?
 

Bev D

Heretical Statistician
Leader
Super Moderator
A proportion control chart won’t help much.
BUT i dont think this is a ‘job for statistics’, it’s job for physics. At a sample size of 3 you probably should treat each failure as something to be investigated. Ho well do you understand the processes and material properties that create the ‘strength’ you are testing? how are you monitoring those characteristics? What is different about the part that was damaged and leaked from other parts?
 
Top Bottom