Capability or Gage R&R Study for Leak Tester?

Kenny

Registered
Hi guys,

I have a question to ask.
I have 3 Helium Leak Tester with Chamber Vacuum type.

1) I would like to get the repeatability of each chamber. However, I understand the measurements values obtained fluctuates and have variations among each samples i test. Its difficult to get the values close to each other. May I know what is the acceptable range for the fluctuations? Is it possible to perform R&R for this?

2) The 3 chambers have variation among each other. How can I reduce the variations to ensure the 3 chambers measured results are close to each other? What is the acceptable range for chamber to chamber variation?
 

Miner

Forum Moderator
Leader
Admin
I have some dated experience with this type of test, so the technology may have changed in the interim. The type I was familiar with had little operator involvement limited to loading the chamber, closing it and pressing a button. The rest of the test was automated. Is yours similar?

If that is the case, I would treat the three chambers the same as three operators. A standard R&R study should give you repeatability (within chamber variation) and reproducibility (between chamber variation).

For determining the acceptable range of variation, you would compare the measurement variation against your maximum leakage specification. My blog post on MSA and 1-sided tolerances may help.

Are you well versed in R&R studies? From your questions, I get the impression that this may be new to you. If so, there is a lot of information on the cove covering this topic. I recommend starting here.

One more thing that I encountered: The company that handled the calibration of our testers would always calibrate the mass spectrometer in isolation. However, a measurement device is the total system, not one piece of it. Another example: we had a robotic sniffer that would move across all of the welded seams and sniff for leaks. However, it would check itself by pausing to sniff a calibrated leak. The results when paused are not the same as the results when moving. When calibrating or performing an MSA, it is very important to test the total system.
 

Kenny

Registered
I have some dated experience with this type of test, so the technology may have changed in the interim. The type I was familiar with had little operator involvement limited to loading the chamber, closing it and pressing a button. The rest of the test was automated. Is yours similar?
Yes, that's right. My machine is automated.

If that is the case, I would treat the three chambers the same as three operators. A standard R&R study should give you repeatability (within chamber variation) and reproducibility (between chamber variation).
Can I don't treat this 3 chambers as 3 operators? Because all these 3 chambers have its own variation which is large. By doing this, I'm afraid its don't show the repeatability and reproducibility of the particular chamber. I am looking on the repeatability and reproducibility of EACH particular chamber. Possible to focus on 1 chamber to do the R&R? What I meant is by using 10 samples X 3 operators X 3 sessions (although operators is undefined since its automated) ??

For determining the acceptable range of variation, you would compare the measurement variation against your maximum leakage specification. My blog post on MSA and 1-sided tolerances may help.
What does this meant? For example, my maximum leakage specification for my product is 2e-5 cc/m, the results obtained by 1 pass sample may between 1e-6 to 1.99e-5 cc/m. Does this meant the acceptable variations is between 1.99e-5 to 1e-6 cc/m ?? If that's the case, its more like detecting pass/fail sample?

One more thing that I encountered: The company that handled the calibration of our testers would always calibrate the mass spectrometer in isolation. However, a measurement device is the total system, not one piece of it. Another example: we had a robotic sniffer that would move across all of the welded seams and sniff for leaks. However, it would check itself by pausing to sniff a calibrated leak. The results when paused are not the same as the results when moving. When calibrating or performing an MSA, it is very important to test the total system.
Ya understand this. Each chamber have its own calibrated leak bottle to calibrate the system.
From your statement, do you meant that the tested results in the chamber is not comparable with the tested results using sniffer?


 

Miner

Forum Moderator
Leader
Admin
Can I don't treat this 3 chambers as 3 operators? Because all these 3 chambers have its own variation which is large. By doing this, I'm afraid its don't show the repeatability and reproducibility of the particular chamber. I am looking on the repeatability and reproducibility of EACH particular chamber. Possible to focus on 1 chamber to do the R&R? What I meant is by using 10 samples X 3 operators X 3 sessions (although operators is undefined since its automated) ??

You have several options, each of which will tell you something different about your testers:
  • A standard 3 x 3 x 10 R&R study on each tester. However, if the operator truly has no impact on the results, you will end up with a Repeatability study.
  • A Type 1 gage study on each tester. This will evaluate the repeatability, bias and Cg/Cgk capability of the testers.
  • A standard 3 x 3 x 10 R&R study substituting testers for operators. This will evaluate the repeatability of the testers and the reproducibility between testers.
  • An expanded R&R study, which adds between tester variation to the standard R&R study. This will breakout repeatability, reproducibility and between tester out as separate line items.

What does this meant? For example, my maximum leakage specification for my product is 2e-5 cc/m, the results obtained by 1 pass sample may between 1e-6 to 1.99e-5 cc/m. Does this meant the acceptable variations is between 1.99e-5 to 1e-6 cc/m ?? If that's the case, its more like detecting pass/fail sample?

The normal metric for determining acceptability of a gage used for inspection is % Tolerance. However, this metric assumes a 2-sided tolerance. You have a 1-sided tolerance. Did you read my linked blog on 1-sided tolerances? The Cgk metric may be suitable.

Ya understand this. Each chamber have its own calibrated leak bottle to calibrate the system.
From your statement, do you meant that the tested results in the chamber is not comparable with the tested results using sniffer?
I probably should not have included that last statement. That was a different type of tester than the chamber tester. I just mentioned it to emphasize the point about looking at the measurement device as a whole system.
 

Kenny

Registered
ou have several options, each of which will tell you something different about your testers:
  • A standard 3 x 3 x 10 R&R study on each tester. However, if the operator truly has no impact on the results, you will end up with a Repeatability study.
  • A Type 1 gage study on each tester. This will evaluate the repeatability, bias and Cg/Cgk capability of the testers.
  • A standard 3 x 3 x 10 R&R study substituting testers for operators. This will evaluate the repeatability of the testers and the reproducibility between testers.
  • An expanded R&R study, which adds between tester variation to the standard R&R study. This will breakout repeatability, reproducibility and between tester out as separate line items.
Based on your experienced in Helium tester, can i get a good repeatability by doing this?? Meaning that all the values are close to each other in the 3 x 3 x 10 R&R study... This is because, from my understanding, there is a difficulties to get a consistent measurements in leak tester since the whole system have lots of factors which will affect the testing. such as the sealing, vacuum chamber, ion source and etc.
 

Miner

Forum Moderator
Leader
Admin
I suspect that there will be very poor repeatability within a single tester as well as a lot of variation across testers. The leak tester is a highly complex piece of equipment with a lot of variables that introduce variation. In my previous experience, I did not even trust the calibration.
 
Top Bottom