Calibration Intervals (Frequency) derived from Variables Data

R

rdragons

#1
I have been assigned the task of performing calibration interval analysis on two of our company’s products to see if the calibration interval can be extended. A visit to the Metrology Lab provided a copy of RP-1 from which I selected method S2 as being the most appropriate. Preliminary data analysis indicates the current calibration time is at the 95% reliability target, which negates any attempt at extending the calibration interval without further intervention. Rather than take my management bad news, I prefer to take them solutions. I want to lay out a scenario of what needs to change in order to extend the calibration time and let them make the business decision as to whether to implement or not. To accomplish this I decided I need to further analyze individual "variable" failures to get a better visual representation comparing trend, test tolerances, and confidence limits. I can fit a curve through data, calculate standard deviations, and confidence limits, but now I want confidence bands around a curve corresponding to 95% confidence and this confidence band is a function of time.

Enter “Calibration Intervals from Variables Data”

Calibration Intervals from Variables Data.pdf

Equation (14) is not working. The model is second order. (14) is returning a one row 2 column “variance” matrix that I can’t seem to pull the elements out of . And if I could get the elements out I have drawn a blank as to how to incorporate these elements into an equation to plot the confidence boundary vs. time.

I am looking for the solution from (14) to a confidence vs. time function OR an alternative set of equations “that work and are understandable” to calculate and plot confidence bands as a function of time.


Thanks
 

Attachments

Elsmar Forum Sponsor
R

rdragons

#2
Re: Calibration Intervals from Variables Data

Didn't mean to start my own blog here. After staring at "Uncertainty Growth Estimation in Uncertainty Analyzer" this morning. I got the brainstorm to compromise with MRbar/d2 scaled to enclose 95% of a normal distribution as a graphical solution that may work. I have enough data points. Follow up research this pm on Rbar/d2 revealed "Uncertinty Analysis and Parameter Tolerancing" "Figure 4" "(Deviations from Nominal)" confirms that this may be possible. Will give the math a try next week.

Thanks
 

Marc

Hunkered Down for the Duration with a Mask on...
Staff member
Admin
#3
Re: Calibration Intervals from Variables Data

Interesting topic. The post got in at the start of the weekend when it's pretty slow here. Hopefully someone will pick up on it by or on Monday.

Also, by all means - Let us know what you come up with as well. We appreciate the information!
 

Miner

Forum Moderator
Staff member
Admin
#4
Re: Calibration Intervals derived from Variables Data

Make sure that you test for significance on your equation. Your scatterplot looks an awful lot like a shotgun blast, which does not visually indicate much of a relationship. The equation that you developed may be heavily influenced by a few points.
 

BradM

Staff member
Admin
#5
Re: Calibration Intervals derived from Variables Data

Thank you for the paper. I do laud the effort to empirically develop methods to improve quality.

As your second post suggests, I assume you have found your answer to your question.

Are you at a point of validating the method? I was greatly intrigued to see the author suggest not using weights. Without sound theoretical justification for the variables chosen and no weights, I wonder if consistency problems will develop in replication.

I would be intrigued to learn from the author why he/ she chose not to use weights in the regression analysis. Plausibly, rational discussion would merit the use of the chosen parameters, but the data may suggest otherwise. Too, it might be interesting to explore some additional predictors of appropriate calibration interval.

Again, thank you for the article. I enjoyed reading it. Please repost with your results, and any correspondence with the author.
 

Marc

Hunkered Down for the Duration with a Mask on...
Staff member
Admin
#6
Re: Calibration Intervals derived from Variables Data

scatterplot looks an awful lot like a shotgun blast, which does not visually indicate much of a relationship.
That's what I was thinking, but not being an expert in the field I figure(d) I'd keep my mouth shut.

Yes - It's interesting. I've never seen anyone go this far (precise?) in determining calibration intervals.
 
R

rdragons

#7
Re: Calibration Intervals derived from Variables Data

Miner: The equation is heavily influenced by a few points. Least squares best fit does this. Its the ouliers that throw the prediction off. Which opens up a little of the next question. How do I deal with outliers. Every article I read has a different perspective. To predict take them out, but leave them in because they are the most intersting cases? Better yet analyze it both ways (which I don't really want to do).

BradM: I have not found the answer to my question, BUT I have an Rbar/d2 variation to try next week. Does least squares best fit return an average? Why not use least difference best fit?

Your observation about weighted vs nonweighted brought a flashback when I read it. I think the author doesn't use weights because he hasn't validated this method yet I also detected some hesitancy about validity of first versus second order fits. Notice my curve is second order, it beat out first order by .001.

Validation! I've never done a calibration interval analysis before so you get the opportunity to watch me struggle up the learning curve. I will do a lot of research and technique selection, crunch 100's of thousands of data points, (there are 313 variables/calibration and 270 As Found calibrations to analyze for the first product) make a lot decisions on what gives the most correct answer (make my best guess). Validation will come with time. As I gain experience the analysis method can and probably will be tweeked. Remember "my" analysis is a combination of RP-1 S2 and Intervals from Variables data. S2 is over glorified validated standard practice, the variable data intervals technique I suspect is not validated, but its theory sounds really good.

Marc: I don't see a shotgun blast, I see a normal distribution following the plotted curve of the mean. The shotgun blast x axis is "weeks", the y axis is dBm. The trend is -0.5 dBm drift downward in 100 weeks. The test tolerance is +-1dBm. +-1.96 standard deviations is equivalent to the Reliability Target of 95% (95% of As found product will be IN-TOLERANCE) and where it crosses the lower variable test tolerance is the estimate of calibration cycle time. I can eyeball this crossing at 80 weeks, the current calibration cycle is at 52 weeks.

I still want a mathematical technique to get the 1.96 standard deviation curve plotted around the mean, so I can get closer than eyeballing 80 weeks.

I get to do this with 313 variables/calibration. The Pass Fail of a S2 calibration is determined by the variable that goes OUT OF TOLERANCE first. The shot gun plot is one of the Variables that caused a S2 Fail.

I suspect I will find a 1:1 correlation between this variable and TCXO drift. ie the drift in the frequency standard is causing frequency shifts in all the mixers, filters, and IF bandwidths in our product and this is causing the output amplitude to roll off at -.5 dBm / 100 weeks. I will get around to checking for this correlation eventually.

So Marc, the data point in the lower right hand corner caused an entire S2 calibration to fail. Is it a valid data point or an outlier? Do I have a real S2 calibration Fail or can I change it to Pass?

My gut says its a valid data point and this calibration is a confirmed Fail.
 

BradM

Staff member
Admin
#8
Re: Calibration Intervals derived from Variables Data

Thank you!! What an excellent follow-up to our posts. This is such an interesting topic (at least to me). I'll "mull" over your post, and give you some more of my thoughts. Maybe since we're in the week now some others can chime in.

Please do keep us informed of your testing process.
 

Hershal

Metrologist-Auditor
Staff member
Super Moderator
#9
Re: Calibration Intervals derived from Variables Data

Yes - It's interesting. I've never seen anyone go this far (precise?) in determining calibration intervals.
Actually, the U.S. Navy at their Engineering Center in Corona has a department that has done only that type of work for 20+ years.....they have a staggering amount of data.....and I believe the Navy also has had a lot of input to RP-1.....

I couldn't download the information tonight, so I couldn't tell if it is taking more of a Monte Carlo approach (which is making a big come back in Metrology it appears), but is appears to have some similarities, based only on the write-up.

Hershal
 

Marc

Hunkered Down for the Duration with a Mask on...
Staff member
Admin
#10
Re: Calibration Intervals derived from Variables Data

Just out of curiosity, in what circumstances is such precision in determining calibration intervals necessary?
 
Thread starter Similar threads Forum Replies Date
T Basis for Calibration Method and Intervals (Frequency) - FAA approved repair station Federal Aviation Administration (FAA) Standards and Requirements 10
F Calibration Intervals (Frequency) for Stereoscopes, Microscopes, etc. Inspection, Prints (Drawings), Testing, Sampling and Related Topics 2
Jerry Eldred Adjustment of Calibration Intervals (Calibration Frequency) Calibration Frequency (Interval) 13
Marc Determination of Calibration Intervals and Frequency Analysis Calibration Frequency (Interval) 9
A Adjustment of Calibration Intervals (Shortening or Lengthen) Calibration Frequency (Interval) 6
F Gauge R&R & Calibration Intervals Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 14
A Adjusting Calibration Intervals for Welding related Gages AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 4
J Calibration intervals - Expiration dates on our certificates w/out customer approval Calibration Frequency (Interval) 3
D Factors for Determining Gage Calibration Intervals - Extended Intervals Calibration Frequency (Interval) 5
R Methods of Reviewing Calibration Intervals Calibration Frequency (Interval) 2
D Determining Calibration Intervals - Extend or change the length of Calibration Frequency (Interval) 11
lanley liao Question regarding the calibration of monitoring and measure equipment. Oil and Gas Industry Standards and Regulations 0
N IATF Calibration Lean in Manufacturing and Service Industries 3
Q Do these certificates of calibration meet ISO 9001 requirements for traceability to NIST? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 10
T Plug Gage Calibration Calibration and Metrology Software and Hardware 1
M Load Cell Calibration using a totalizer on a flow meter General Measurement Device and Calibration Topics 0
E Calibration Records needed ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 1
D Limited Range Calibration - 5000 lb Industrial floor scale General Measurement Device and Calibration Topics 3
D Calibration of Small Scales General Measurement Device and Calibration Topics 26
C How to Establish the Calibration & Measurement Capability (CMC)? ISO 17025 related Discussions 1
I IQOQ or just initial calibration required? General Measurement Device and Calibration Topics 3
B Calibration in real life ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 3
J Calibration cycle for monitoring & measuring tools used in medical device manufacturing General Measurement Device and Calibration Topics 5
A Is calibration of test weight required General Measurement Device and Calibration Topics 4
S Calibration Frequency for Slip Gauge Kit used for CMM Calibration? General Measurement Device and Calibration Topics 0
S Calibration/Verification of customer fixtures IATF 16949 - Automotive Quality Systems Standard 6
Ron Rompen Calibration by manufacturer ISO 17025 related Discussions 4
Q Calibration verification records 7.1.5.2.1 IATF 16949 - Automotive Quality Systems Standard 2
B AS9100D 7.1.5.2 Calibration or Verification Method using outside cal lab AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 1
W Next Calibration Due Date Calibration Frequency (Interval) 5
S Where do l get calibration standards to run a calibration lab? Other ISO and International Standards and European Regulations 2
A OEM On-Site Calibration issues during Covid19 ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 12
D Calibration tolerance question using Pipettes Medical Device and FDA Regulations and Standards News 1
M Calibration Certificate Result issued by an accredited external laboratory General Measurement Device and Calibration Topics 9
G Calibration of "Master Parts" Used as Gauges Calibration Frequency (Interval) 5
R Calibration lab environmental monitoring General Measurement Device and Calibration Topics 4
G Calibration of Rotronic probe but not digital readout? General Measurement Device and Calibration Topics 3
F Standard Calibration Procedures: Recommended Practice ISO 17025 related Discussions 0
T Temperature Requirements For In House Calibration - AS9100 AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 16
B Gage calibration frequency, ISO and IATF - What are the requirements Calibration Frequency (Interval) 3
Crimpshrine13 Laboratory Scope - Calibration vs. Test Methods - IATF 16949 IATF 16949 - Automotive Quality Systems Standard 3
Crimpshrine13 Calibration of pH Meter Probe Calibration and Metrology Software and Hardware 3
F ESD workbench "calibration" Manufacturing and Related Processes 2
C Correct Calibration Method for Dial Depth Gage General Measurement Device and Calibration Topics 6
F Nist traceable calibration certificates General Measurement Device and Calibration Topics 1
G Tool tracebility and First calibration requirements for aerospace (AS9100) organisation AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 5
D Calibration of Digital thermometer with surface probe General Measurement Device and Calibration Topics 1
T Calibration or Verification -> Cm and Cmk, etc. Definitions, Acronyms, Abbreviations and Interpretations Listed Alphabetically 3
G Is repeatability required for equipment calibration? General Measurement Device and Calibration Topics 10
was named killer AS 9100D - Calibration Instructions - Controlled documents? AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 6

Similar threads

Top Bottom