SBS - The best value in QMS software

Difference between Test Method Validation and Gage R&R

Frypot

Starting to get Involved
#11
Hi Bev, if I repeat the tests and find no significant difference, between the two sets of data (performed by two sets of people on different equipment)?
 
Elsmar Forum Sponsor

Bev D

Heretical Statistician
Staff member
Super Moderator
#12
Frypot - the t-test is to see if there is a difference between two sets of independent data. of course the data are not independent as they are the same parts. but the bigger reason is that measurement error is confounded with part to part variation in a t-test analysis. since measurement error is typically random about the true value of the part, the means of the data sets will be the same. mathematically they have to be. unless of course there is systemic bias in the measurement process. Then if the bias is large enough you will see a statistically significant difference between the means. I have attached 3 examples. they are all 30 parts selected randomly from across the specification range and were measured randomly twice. they all 3 pass the t-test but have very different repeatability that you can only see when you do the appropriate analysis.

quality statistics are like your tool box. there are many tools all with different purposes. you should not use a nail file to cut down a tree and you shouldn't use a chain saw to trim your toe nails...
 

Attachments

Frypot

Starting to get Involved
#13
Thanks, Bev I appreciate your efforts but I'm certainly not comfortable with stats! (Shows doesn't it?)
I already know that the variation of the material is a big factor.
My big problem is the testing is destructive, therefore I cannot repeat a measurement. So all tests are on individual packs (made to the same specification).

Sorry to be a pest, but what tool would you suggest?
 

Bev D

Heretical Statistician
Staff member
Super Moderator
#14
Of course you can. you can cut the part in two, you can sample from the same well mixed batch. OR you can use a nested study that utilizes 'matched parts' that are made under the same conditions and get multiple parts from different conditions, lots or batches. Many of us do this all of the time. check out my MSA papers in the resources section here to see different examples. Also Miner discusses this in his blogs. If you tell us what you are measuring and the test method we can probably give direct advice...
 

Frypot

Starting to get Involved
#15
Thanks for your patience.
The product is a wound dressing pack.
The pack consists of an upper paper coated with a layer of LDPE (on the inside of the pack) and a lower layer of paper coated with a heat seal adhesive lacquer coated in a grid pattern.
The pack has a left, right and bottom seal. Top top has a chevron seal (pointing upwards) giving the shape of a child's drawing of a house (i.e. a pentagon) this gives a 'Top left' and 'Top right' seal.
This 'grid lacquer' paper is notorious for being variable in coating weight.
The seals of the pack are tested by cutting a 15mm strip (perpendicular to the seal) and pulling the two webs apart. This part of the seal cannot be tested again, a sample close to this area can easily have a seal strength value of 10% to even 100% more, due to the variability of the coating (?).

The pass strength is > 104gf/15mm strip.
 

William55401

Quite Involved in Discussions
#16
I have been asked by a customer to validate our test method for seal strength (as in ASTM F88)
If there is a published standard, I am not sure there is a need to validate. I do not have the authoritative source, but I have been trained that test method validation is not needed when a recognized standard is used. Disclaimer: I am not expert in TMV-- just sharing my quick take.
 

Bev D

Heretical Statistician
Staff member
Super Moderator
#17
percentages can be very misleading. it is always best to perform the MSA and see if the paired measurements -even tho destructive - actually are good enough. I've seen a lot of people get surprised! don't talk yourself out of it. do the study and see what the results are.

there are a couple of other considerations at this point - since can't really visualize exactly what you are doing - you may simply have a method that is not indicative of the seal strength. remember that the seal must held across the entire seal area and weak areas can be bolstered by stronger areas depending on the forces they are supposed to respond I always advise to test the thing in the way it will actually be used and could actually fail....if you are to test the whole thing then you want to select parts to pair made sequentially.

Do you always pass or do you sometimes fail?
 

Bev D

Heretical Statistician
Staff member
Super Moderator
#18
To expand on last comment I made: every test should actually test the function it is intended to assess. That doesn’t always happen. This step is a critical part of test method validation. It’s not the only one, but it is essential. A real life example that I’ve used here before: the inner diameter of a ‘tube’ was being measured by a 3 point ID tool. Even though every tube passed there were failures in the field. We performed a repeatability MSA suing 30 parts measured twice. It failed. We then used an instrument that utilized air pressure. It passed perfectly. We were suspicious. So we measured the parts using a CMM, measuring the diameter multiple times around the ID. The average diameter from the CMM also passed teh MSA as we suspected it would. BUT that isn’t why we used the CMM; we wanted to map the shape of the inner diameter. It showed Us that teh ID was tri-lobular. This is why the air gage passed aNPD the 3 point device didn’t. The measured ID from the 3 point device depended on the orientation of the 3 points in the ID. The machinists ‘knew‘ this by the way and would fiddle with the measurement device to get a passing answer. (I am NOT saying this was their fault at all by the way). The tri-lobular pattern was created because we were using a 3 jaw chuck to hold an machine the parts. Now this might have been OK depending on the function of the tube. IF it was to direct the flow of air or water the area of the ID then the air gage would have been appropriate. The difference between the smallest ID and Largest ID in the trilobular pattern may even have been small enough to allow for using the 3 point gage taking multiple measurements and using the average. However, the ‘tube’ was really an axle that had a press fit bearing installed in it. The minimum ID was the critical feature, not the area or average of the opening.

There is another aspect of your question that needs to be addressed: what was your study design? were your two set of data from two sets of DIFFERENT parts or the same parts? And what kind variation does this process experience? Depending on this answer there may be a different approach to validate the repeatability of the measurement process.
 

Frypot

Starting to get Involved
#19
Bev D, mainly pass but not always.
All tests were done on different samples (produced in one run on the same team). Testing used different sites & teams.
I think I may be confusing the situation: I have assumed reproducibility meant that the test method could be done somewhere else by someone else and still get a 'reliable result', and repeatability as meaning the same tester (and site) could reproduce the result of testing a batch.
 
Thread starter Similar threads Forum Replies Date
S What is the difference between Verification and Validation of Test Reports? AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 5
M Difference between a "check" and a "test" Manufacturing and Related Processes 7
L Difference between a Quality Plan and Inspection & Test Plan (ITP) ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 4
C Difference between "Pass/Fail Testing" "Test To Failure" "Degradation Test" FMEA and Control Plans 1
C What is difference between a t-test and Correlation Coefficient Statistical Analysis Tools, Techniques and SPC 2
S What is the difference between a Test Protocol and a Test Report? Records and Data - Quality, Legal and Other Evidence 9
C Difference between proficiency test and interlaboratory test? General Measurement Device and Calibration Topics 3
W What is the difference between TYPE B and TYPE BF? IEC 60601 - Medical Electrical Equipment Safety Standards Series 2
T The difference between ISO 14644-3:2005 and ISO 14644:2019 Other Medical Device Related Standards 2
Q Terminal Lugs sizes - Difference between 225/24 vs. 275/24 lugs Manufacturing and Related Processes 2
M Difference between "Production Trial Run" and "Run at Rate" IATF 16949 - Automotive Quality Systems Standard 8
A What is the difference between Design Process, Process Design and Design Control? 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 2
R What's the major difference between Green Belt and Black Belt in term of training and project Six Sigma 3
T Difference between a subcontractor and a supplier ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 21
H Difference between Stainless Steel 316 ASTM F899 and ASTM A276 Other Medical Device Related Standards 3
A Exact terms for a plating failure and difference between rejection rate and failure rate Manufacturing and Related Processes 9
M Difference between MSA and MSE? General Measurement Device and Calibration Topics 1
gramps What is the difference between discrete and continuous variables? Problem Solving, Root Cause Fault and Failure Analysis 3
JoCam Difference between Approval and Registration - ISO 13485:2016 ISO 13485:2016 - Medical Device Quality Management Systems 2
S Difference between EU-MDR Annex IX and the Annex-combo X&XI EU Medical Device Regulations 4
T ISO 17025:2017 Clause 4.2.2 - The difference between "be notified" and "be informed" ISO 17025 related Discussions 4
Jimmy123 What is the difference between Error Proofing and Controls? ISO/IATF 16949 - Control Plans FMEA and Control Plans 16
C IEC 60601-1-8, difference between table 4 and annex D IEC 60601 - Medical Electrical Equipment Safety Standards Series 2
A What is the difference between Basic UDI-DI and UDI-DI? EU Medical Device Regulations 6
Q What is the difference between AS9100D 9.3.2.f and 9.3.3.a AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 8
M Aluminum - What is the difference between 6061-T6 and 6061-T651 (both per ASTM B211)? Manufacturing and Related Processes 4
P What is the exact difference between Risk and Opportunity in context of ISO 27001? IEC 27001 - Information Security Management Systems (ISMS) 7
S Difference between Surface Finish (Ra) and Flatness (GD&T) Inspection, Prints (Drawings), Testing, Sampling and Related Topics 6
O Air Flow - Which is the operational difference between LAF (vertical and horizontal) and RLAF? Manufacturing and Related Processes 2
S What the difference is between Stub Acme & Acme thread? Oil and Gas Industry Standards and Regulations 1
S DO 178B - What is the difference between review and verification? Federal Aviation Administration (FAA) Standards and Requirements 1
T The difference between SOP and Kaizen Standardization Lean in Manufacturing and Service Industries 2
K Difference between intended purpose and intended use of the device EU Medical Device Regulations 9
Q What is the difference between normal and licensed internal auditor? VDA Standards - Germany's Automotive Standards 9
J What is the difference between Process Variation and Tolerance? Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 4
O Difference Between PFMEA & Control Plan FMEA and Control Plans 3
S Difference between an Advisory Notice (ISO 13485) and a Field Safety Notice? ISO 13485:2016 - Medical Device Quality Management Systems 3
T Difference between "data analysis" and "management review" ISO 13485:2016 - Medical Device Quality Management Systems 4
qualprod What is the difference between 7.4.1 (2008) and 8.4.1 (2015)? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 6
D Difference between uncertainty and expanded uncertainty of measurement General Measurement Device and Calibration Topics 1
S EASA Part 145 - The difference between non-certifying staff and Certifying staff Federal Aviation Administration (FAA) Standards and Requirements 2
S The difference between a Medical Device Accessory and Component Canada Medical Device Regulations 2
S What is the difference between a service request and a complaint? ISO 13485:2016 - Medical Device Quality Management Systems 2
I What is the difference between OOS/OOT Imported Legacy Blogs 3
A Definition Difference between Quality System Procedure and Standard Operating Procedure (SOP) Definitions, Acronyms, Abbreviations and Interpretations Listed Alphabetically 4
Q Difference between Monitor & Measurement ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 4
T QA vs. RA - The difference between QA and RA Coffee Break and Water Cooler Discussions 6
A Difference between Discrimination and Least Count IATF 16949 - Automotive Quality Systems Standard 2
K Difference between Medical Electrical Equipment and Medical Equipment System IEC 60601 - Medical Electrical Equipment Safety Standards Series 3
M Usability Standard - The difference between IEC 60601-1-6 to IEC 62336 Human Factors and Ergonomics in Engineering 1

Similar threads

Top Bottom