Difference between Test Method Validation and Gage R&R

[Frypot]

Hi Bev, if I repeat the tests and find no significant difference, between the two sets of data (performed by two sets of people on different equipment)?

 

[Bev D]

Frypot - the t-test is to see if there is a difference between two sets of independent data. of course the data are not independent as they are the same parts. but the bigger reason is that measurement error is confounded with part to part variation in a t-test analysis. since measurement error is typically random about the true value of the part, the means of the data sets will be the same. mathematically they have to be. unless of course there is systemic bias in the measurement process. Then if the bias is large enough you will see a statistically significant difference between the means. I have attached 3 examples. they are all 30 parts selected randomly from across the specification range and were measured randomly twice. they all 3 pass the t-test but have very different repeatability that you can only see when you do the appropriate analysis.

quality statistics are like your tool box. there are many tools all with different purposes. you should not use a nail file to cut down a tree and you shouldn't use a chain saw to trim your toe nails...

 

[Frypot]

Thanks, Bev I appreciate your efforts but I'm certainly not comfortable with stats! (Shows doesn't it?)
I already know that the variation of the material is a big factor.
My big problem is the testing is destructive, therefore I cannot repeat a measurement. So all tests are on individual packs (made to the same specification).

Sorry to be a pest, but what tool would you suggest?

 

[Bev D]

Of course you can. you can cut the part in two, you can sample from the same well mixed batch. OR you can use a nested study that utilizes 'matched parts' that are made under the same conditions and get multiple parts from different conditions, lots or batches. Many of us do this all of the time. check out my MSA papers in the resources section here to see different examples. Also Miner discusses this in his blogs. If you tell us what you are measuring and the test method we can probably give direct advice...

 

[Frypot]

Thanks for your patience.
The product is a wound dressing pack.
The pack consists of an upper paper coated with a layer of LDPE (on the inside of the pack) and a lower layer of paper coated with a heat seal adhesive lacquer coated in a grid pattern.
The pack has a left, right and bottom seal. Top top has a chevron seal (pointing upwards) giving the shape of a child's drawing of a house (i.e. a pentagon) this gives a 'Top left' and 'Top right' seal.
This 'grid lacquer' paper is notorious for being variable in coating weight.
The seals of the pack are tested by cutting a 15mm strip (perpendicular to the seal) and pulling the two webs apart. This part of the seal cannot be tested again, a sample close to this area can easily have a seal strength value of 10% to even 100% more, due to the variability of the coating (?).

The pass strength is > 104gf/15mm strip.

 

[William55401]

I have been asked by a customer to validate our test method for seal strength (as in ASTM F88)

If there is a published standard, I am not sure there is a need to validate. I do not have the authoritative source, but I have been trained that test method validation is not needed when a recognized standard is used. Disclaimer: I am not expert in TMV-- just sharing my quick take.

 

[Bev D]

percentages can be very misleading. it is always best to perform the MSA and see if the paired measurements -even tho destructive - actually are good enough. I've seen a lot of people get surprised! don't talk yourself out of it. do the study and see what the results are.

there are a couple of other considerations at this point - since can't really visualize exactly what you are doing - you may simply have a method that is not indicative of the seal strength. remember that the seal must held across the entire seal area and weak areas can be bolstered by stronger areas depending on the forces they are supposed to respond I always advise to test the thing in the way it will actually be used and could actually fail....if you are to test the whole thing then you want to select parts to pair made sequentially.

Do you always pass or do you sometimes fail?

 

[Bev D]

To expand on last comment I made: every test should actually test the function it is intended to assess. That doesn’t always happen. This step is a critical part of test method validation. It’s not the only one, but it is essential. A real life example that I’ve used here before: the inner diameter of a ‘tube’ was being measured by a 3 point ID tool. Even though every tube passed there were failures in the field. We performed a repeatability MSA suing 30 parts measured twice. It failed. We then used an instrument that utilized air pressure. It passed perfectly. We were suspicious. So we measured the parts using a CMM, measuring the diameter multiple times around the ID. The average diameter from the CMM also passed teh MSA as we suspected it would. BUT that isn’t why we used the CMM; we wanted to map the shape of the inner diameter. It showed Us that teh ID was tri-lobular. This is why the air gage passed aNPD the 3 point device didn’t. The measured ID from the 3 point device depended on the orientation of the 3 points in the ID. The machinists ‘knew‘ this by the way and would fiddle with the measurement device to get a passing answer. (I am NOT saying this was their fault at all by the way). The tri-lobular pattern was created because we were using a 3 jaw chuck to hold an machine the parts. Now this might have been OK depending on the function of the tube. IF it was to direct the flow of air or water the area of the ID then the air gage would have been appropriate. The difference between the smallest ID and Largest ID in the trilobular pattern may even have been small enough to allow for using the 3 point gage taking multiple measurements and using the average. However, the ‘tube’ was really an axle that had a press fit bearing installed in it. The minimum ID was the critical feature, not the area or average of the opening.

There is another aspect of your question that needs to be addressed: what was your study design? were your two set of data from two sets of DIFFERENT parts or the same parts? And what kind variation does this process experience? Depending on this answer there may be a different approach to validate the repeatability of the measurement process.

 

[Frypot]

Bev D, mainly pass but not always.
All tests were done on different samples (produced in one run on the same team). Testing used different sites & teams.
I think I may be confusing the situation: I have assumed reproducibility meant that the test method could be done somewhere else by someone else and still get a 'reliable result', and repeatability as meaning the same tester (and site) could reproduce the result of testing a batch.