Pipette Calibration Failure

C

curryassassin

#1
Dear Learned Colleagues,
Please can you advise on whether the following approach is valid.
We perform specialist analytical tests for pharmaceutical companies. The tests often use volumes in the micro-litre range. For this purpose we use adjustable pipettes or pipettors which we calibrate for accuracy and precision on a monthly basis. Occasionally these pipettes fail the calibration, and we document this as a deviation or non-conformance. This is then followed by an investigation into the tests which may have used the failed pipette since the last succesful calibration. Usually when we do this, we find that all the tests have passed the test acceptance criteria, so we say that the pipette calibration failure did not have an effect on the test results. Then we re-adjust and re-caliobrate the failed pipette.
Is this a valid approach please?
 
Elsmar Forum Sponsor

Jerry Eldred

Forum Moderator
Super Moderator
#2
This is what I would refer to as Out-of-Tolerance Impact Analysis.

This is very particular to the requirements of your quality system. I'm not at all familiar with your specifics. But I would generally say that if you have a legitimate method to quantitatively validate tests between the last In-Tolerance calibration and the Out-of-Tolerance calibration, and prove that nothing erroneously failed or erroneously passed that they were used to test, you should be okay.

In my context, we record before and after data on everything; and so have data we can use to verify whether or not there was impact. We also have rules as to what constitutes significant or not significant out of tolerance conditions (which I'm not free to share here).

I would say your approach appears valid as long as you are able to quantify how much deviation occurred as a result of the out of tolerance, what the corrected values would be, and whether the corrected values pass or fail your criteria.
 
D

D.Scott

#3
This is what I would refer to as Out-of-Tolerance Impact Analysis.

This is very particular to the requirements of your quality system. I'm not at all familiar with your specifics. But I would generally say that if you have a legitimate method to quantitatively validate tests between the last In-Tolerance calibration and the Out-of-Tolerance calibration, and prove that nothing erroneously failed or erroneously passed that they were used to test, you should be okay.

In my context, we record before and after data on everything; and so have data we can use to verify whether or not there was impact. We also have rules as to what constitutes significant or not significant out of tolerance conditions (which I'm not free to share here).

I would say your approach appears valid as long as you are able to quantify how much deviation occurred as a result of the out of tolerance, what the corrected values would be, and whether the corrected values pass or fail your criteria.
Jerry - I am concerned that the OP hasn't mentioned that they correct the test values to allow for the deviation. I certainly agree with the answer you gave but only under the condition that recalculation of the results occurred. If the results were simply reviewed and the uncorrected results passed, I would not consider it to be acceptable.

I have no doubt curryassassin is doing it the way you describe but you never know how others might read this.

Dave
 

BradM

Staff member
Admin
#4
Dear Learned Colleagues,
Please can you advise on whether the following approach is valid.
We perform specialist analytical tests for pharmaceutical companies. The tests often use volumes in the micro-litre range. For this purpose we use adjustable pipettes or pipettors which we calibrate for accuracy and precision on a monthly basis. Occasionally these pipettes fail the calibration, and we document this as a deviation or non-conformance. This is then followed by an investigation into the tests which may have used the failed pipette since the last succesful calibration. Usually when we do this, we find that all the tests have passed the test acceptance criteria, so we say that the pipette calibration failure did not have an effect on the test results. Then we re-adjust and re-calibrate the failed pipette.
Is this a valid approach please?
For the most part, yes. Typically, the re-adjustment/ re-calibration happens right after the initial calibration. That is the correction activity. Then you report on the as-found values and follow the path you described.

Now, do you take as-found measurements prior to replacing gaskets, cleaning, etc.? Do you (or the entity you do work for) perform uncertainty analysis on your process? It's important if you know your uncertainties- where you can determine if you have a superior/inferior verification system as compared to the customer.

If you are having that many failures with no impact to the customer, maybe you can consider loosening your tolerances.

For reference, here is a pretty good paper I found on pipette uncertainty:


Calibration of micropipettes: Test methods and
uncertainty analysis

E. Batista a,*, L. Pinto a, E. Filipe a, A.M.H. van der Veen b


Measurement 40 (2007) 338–342
From the author, here are her variables:


M correspond to the standard deviation of the mean of the 10 mass measurements.
dmres is the uncertainty of the resolution of the balance
dmcal is the uncertainty of the balance
t is the uncertainty of the water temperature
dtcal is the uncertainty of the thermometer
ρw is the uncertainty of the density of the water
ρA is the uncertainty of the density of the air
ρB is the uncertainty of the density of the balance weights
γis the uncertainty of the expansion coefficient of the micropipette material (polypropylene).

If you need more detailed information please let me know.

Best Regards,
Elsa Batista


 

Hershal

Metrologist-Auditor
Staff member
Super Moderator
#5
BradM is correct.....there is the potential to lossen the tolerance, and you must know your uncertainty of the calibration. As painful as it seems to be, you need to calculate the uncertainty for the recalibration of that pipette as a part of the investigation.....

This is especially important if you are accredited to ISO/IEC 17025 by UKAS.

Hope this helps.
 
C

curryassassin

#6
For the most part, yes. Typically, the re-adjustment/ re-calibration happens right after the initial calibration. That is the correction activity. Then you report on the as-found values and follow the path you described.

Now, do you take as-found measurements prior to replacing gaskets, cleaning, etc.? Do you (or the entity you do work for) perform uncertainty analysis on your process? It's important if you know your uncertainties- where you can determine if you have a superior/inferior verification system as compared to the customer.

If you are having that many failures with no impact to the customer, maybe you can consider loosening your tolerances.

For reference, here is a pretty good paper I found on pipette uncertainty:

[/FONT][/SIZE]
Brad, thanks, I'll check it out.

We use a colourimetric method to calibrate our pipettes (the amount of colour produced directly relates to the volume added by the pipette). I think there may be reason to review our process uncertainties. We will shortly introduce trending of calibration data so that we can predict a failure before it happens, and then make adjustments. We will also increase the analyst training and competency assessments, which includes a demonstration of correct pipetting technique as well as performing succesful calibrations.

As mentioned in my original cry for help, the calibration failure does not necessarily affect the results obtained for the calibration standards and quality control samples which are included in each of our pharmaceutical tests, since these are biological tests. However, a known pipetting failure (under or over volume) by an analyst has been cited as the cause of a test failure.
 
K

KeithM

#7
I've been looking into the calibration of pipettes for some time now. We are an FDA regulated facility and require pipettes to be calibrated quarterly. For in house calibrations the Mfg Dept checks the pipettes at the high and low range taking 5 readings each and the average mean is what the pass or fail is based on. Upon further investigation I've found numerous erros in this approach by comparing our SOP to the manufacturers certificate. My findings are as follows:
The Manufacturer has a tightly controled environment with Temperature +/- 1 degree C and Humidity at 60% +/- 15%
Our Labs are +/- 7 degrees with 20-80% humidity.
The Manufacturer uses a 5 and 6 place balance and also I can only assume uses a humidity trap to stop evaporation and lock in the measurement. (Mettler makes a real nice Pipette calibration kit. That includes this humidity trap.)
We use a 4 place balance and a small dish with no humidity trap.
The Manufacturer and some contractors use Pipette tracker software that calculates the Z factor and converts the measurement from grams to microliters or ul if I didn't spell it right.
Critical error on our part we pass or fail based on avg mean in grams.
The Manufacturer passes or fails based on accuracy and precision which is not the same as avg mean.
So with that said I find that our inhouse equipment and procedure are highly inadequete.
Currently I'm building a new calibration table so we can calibrate pipettes by volume and take all of the model numbers off of the SOP. Also I find that pipettes really should be checked in the middle range as well which is were they are supposed to be used.Of course the R&D Dept has never bothered to follow this SOP and bought every kind of pipette that is not listed on our SOP to make matters worse. This new table will simply state the specs pipettes must meet no matter what brand they have. If they fail and cannot be repaired they will be disposed of. I'm also recomending that we buy the proper balance, Software and Humidity trap to start getting a more accurate calibration for our Pipettes. If anyone has an SOP or any ideas to help me along I would appreciate it.
 

BradM

Staff member
Admin
#8
Keith, there are some ASTM standards. E1154-89 is one. The below document is a pretty good one:

[FONT=TimesNewRoman,Bold]Measurement Good Practice Guide No. 69[/FONT]
[FONT=TimesNewRoman,Bold]The Calibration and Use of Piston Pipettes[/FONT]
John Blues
National Weights and Measures Laboratory
David Bayliss
National Physical Laboratory


I would post it, but it's got copyright stuff on it.:(

Keith, you sound like you have some pretty good ideas going about pipettes, and it would appear you have done some homework on this. If you are so inclined, it would be appreciated if you might share some of your findings, observations and stuff on pipettes. We can discuss them, and start a repository of ideas on pipettes.

As far as balances and stuff... well, the rule applies: I would get the best you can afford.
 
K

KeithM

#9
One Calibration company in particular has provided me with some great background info. I don't think it is improper to post a document he sent me as long as he gets his name in print. Please let me know if I am out of line by posting thisKenneth Bonnel has written a number or studies regarding Pipettes and I believe he played the lead role in writing and set up of Troemners Metrology SOP's. The table of data included with his report may not come out in this post very well but it's pretty easy to understand the final conclusions. While I understand some companies only calibrate because they are required to. I just don't feel satisfied with collecting numbers to have on file. Why not do it right and get real data?

Pipette Calibrations – The Correct Number of Test Volumes
Kenneth Bonnell
President
Alpha Omega Calibrations
kbonnell alphaomegacal
December 2002
Abstract

There has been much debate about the correct number of volumes that should be tested in order to accurately characterize the performance of any liquid handling device. Some manufacturers define the performance of these devices at only two volumes while others publish specifications that detail the performance at three or more points. In order to deduce a statistically justifiable answer, multiple pipettes must be tested at the two-end volumes (high and low) and then retested incorporating the center point. A gravimetric analysis will be performed using the procedures outlined in ISO 8655.
Introduction

There are numerous manufacturers of pipettes, which produce hundreds of different makes and models. The performance of these instruments is tested using different methodologies and employing different techniques, which makes a specification comparison between devices extremely difficult. Due to this variability, some manufacturers only define the performance at one or two points while others characterize the performance at three or more volumes. Obviously, this poses several problems for any metrologist interested in calibrating these devices and reporting meaningful data.

There are many calibration companies that simply follow the manufacturer’s recommendations, while others are uncomfortable reporting only two (2) volumes simply because the adjustment is based on only one other point. These devices are manufactured and reported to be linear, so it would seem to be essential that they are assessed at three points. This is due to the fact that the metrologist performing the calibration would have two (2) other points to reference while performing the calibration and, of course, two points determine a line.
Experimental Design

In order to validate this assertion that three volumes are a requirement in order to accurately characterize the performance of any liquid handling device, the following experimental design was employed.

- Gravimetric assessment according to ISO 8655
- Grade 3, bi-distilled, degassed water
- Standardize on only 1 technician that has been properly trained and proficiency tested
- All testing will be performed using the forward mode
- All pipettes will be assessed using a 10-10 measurement structure initially and then reassessed using a 10-10-10 measurement structure in order to incorporate the center volume
- All pipettes will be assessed 10 times by the same technician in order to collect a sampling of at least 100 aliquots per volume
- The Z-factor correction will be calculated using all of the requisite environmental inputs as well as the appropriate cubic expansion coefficient
- All instruments are new, manufacturer’s tolerances will be utilized
- The addition tare method will be employed
- The instruments will be allowed to thermally equilibrate to the laboratory for at least 4 hours before any data is ascertained.
- A wide variety of instruments were selected
o Different manufacturers
o Different models
o Different volumetric ranges
- The center volume was calculated using a linear extrapolation technique for any instrument defined at only two points.

All of the pipettes were initially tested at only the low and high volumes and adjusted if any of the instruments were out at either volume. We calibrated these instruments towards the center of the tolerance band, so that when the center volume was tested it should have passed. The pipettes were tested and adjusted multiple times using the aforementioned experimental design in order to provide at least 100 aliquots per instrument and volume, which, of course, offered enough data points in order to statistically justify the results.
Data

Mfg Model # of Times Tested Adjusted (Y/N) 10-10-10 Failures Failure %
Biohit Proline 0.1-2.5ul 10 Y 2 20%
Biohit Proline 2-20ul 10 Y 1 10%
Biohit Proline 100-1000ul 10 Y 2 20%
Eppendorf Reference, 2-20ul 10 Y 1 10%
Eppendorf 10-100ul 10 Y 1 10%
Eppendorf 100-1000ul 10 Y 2 20%
Gilson P2 P2 10 Y 4 40%
Gilson P20 P20 10 Y 1 10%
Gilson P100 P100 10 Y 2 20%
Gilson P200 P200 10 Y 3 30%
Gilson P1000 P1000 10 Y 2 20%
Nichiryo Nichipette 2-20ul 10 Y 3 30%
Nichiryo Nichipette 20-200ul 10 Y 5 50%
Nichiryo Nichipette 100-1000ul 10 Y 2 20%

Mean = 22.14%
Standard Deviation (based on experimental design, 100 aliquots per volume) = 1.27%
Uncertainty (k=2) = 2.54%
Analysis

The data mean in terms of failures was 22.14% with an uncertainty of + 2.54%, which, of course, gives us a 95% confidence interval in terms of the results. This means that a pipette that was adjusted at the low end and high end using only the other point as a reference will normally fail when assessed at the center volume between 19.60% and 24.98% of the time.
Conclusion

It is abundantly clear that all instruments should be assessed using at least three (3) points, since there is at least a 1 in 5 chance that the instrument will fail when calibrated using only two (2) volumes. This is simply due to the fact that these devices are designed to be linear and it takes at least two (2) reference points to adjust the instrument properly besides the one point that is being tested. The data clearly substantiates and supports this supposition.:applause:
 
C

curryassassin

#10
Keith,
Very interesting info. Do you think that a third calibration point on each pipette would ultimately result in fewer subsequent calibration failures?
When a pipette fails in your facility, how do you assess the impact of the failure please?
 
Thread starter Similar threads Forum Replies Date
R Assessing Pipette Calibration Failures General Measurement Device and Calibration Topics 5
M Pipette Calibration & ISO 8655 - No NIST Standard(s) Traceability ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 5
J Simplified Uncertainty Equation for Pipette Calibration Measurement Uncertainty (MU) 7
lanley liao Question regarding the calibration of monitoring and measure equipment. Oil and Gas Industry Standards and Regulations 0
N IATF Calibration Lean in Manufacturing and Service Industries 3
Q Do these certificates of calibration meet ISO 9001 requirements for traceability to NIST? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 10
T Plug Gage Calibration Calibration and Metrology Software and Hardware 1
M Load Cell Calibration using a totalizer on a flow meter General Measurement Device and Calibration Topics 0
E Calibration Records needed ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 1
D Limited Range Calibration - 5000 lb Industrial floor scale General Measurement Device and Calibration Topics 3
D Calibration of Small Scales General Measurement Device and Calibration Topics 26
C How to Establish the Calibration & Measurement Capability (CMC)? ISO 17025 related Discussions 1
I IQOQ or just initial calibration required? General Measurement Device and Calibration Topics 3
B Calibration in real life ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 3
J Calibration cycle for monitoring & measuring tools used in medical device manufacturing General Measurement Device and Calibration Topics 5
A Is calibration of test weight required General Measurement Device and Calibration Topics 4
S Calibration Frequency for Slip Gauge Kit used for CMM Calibration? General Measurement Device and Calibration Topics 0
S Calibration/Verification of customer fixtures IATF 16949 - Automotive Quality Systems Standard 6
Ron Rompen Calibration by manufacturer ISO 17025 related Discussions 4
Q Calibration verification records 7.1.5.2.1 IATF 16949 - Automotive Quality Systems Standard 2
B AS9100D 7.1.5.2 Calibration or Verification Method using outside cal lab AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 1
W Next Calibration Due Date Calibration Frequency (Interval) 5
S Where do l get calibration standards to run a calibration lab? Other ISO and International Standards and European Regulations 2
A OEM On-Site Calibration issues during Covid19 ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 12
D Calibration tolerance question using Pipettes Medical Device and FDA Regulations and Standards News 1
M Calibration Certificate Result issued by an accredited external laboratory General Measurement Device and Calibration Topics 9
G Calibration of "Master Parts" Used as Gauges Calibration Frequency (Interval) 5
R Calibration lab environmental monitoring General Measurement Device and Calibration Topics 4
G Calibration of Rotronic probe but not digital readout? General Measurement Device and Calibration Topics 3
F Standard Calibration Procedures: Recommended Practice ISO 17025 related Discussions 0
T Temperature Requirements For In House Calibration - AS9100 AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 16
B Gage calibration frequency, ISO and IATF - What are the requirements Calibration Frequency (Interval) 3
Crimpshrine13 Laboratory Scope - Calibration vs. Test Methods - IATF 16949 IATF 16949 - Automotive Quality Systems Standard 3
Crimpshrine13 Calibration of pH Meter Probe Calibration and Metrology Software and Hardware 3
F ESD workbench "calibration" Manufacturing and Related Processes 2
C Correct Calibration Method for Dial Depth Gage General Measurement Device and Calibration Topics 6
F Nist traceable calibration certificates General Measurement Device and Calibration Topics 1
G Tool tracebility and First calibration requirements for aerospace (AS9100) organisation AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 5
D Calibration of Digital thermometer with surface probe General Measurement Device and Calibration Topics 1
T Calibration or Verification -> Cm and Cmk, etc. Definitions, Acronyms, Abbreviations and Interpretations Listed Alphabetically 3
G Is repeatability required for equipment calibration? General Measurement Device and Calibration Topics 10
was named killer AS 9100D - Calibration Instructions - Controlled documents? AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 6
B Photovoltaic Tester Calibration General Measurement Device and Calibration Topics 0
B Temperature loop: partial calibration General Measurement Device and Calibration Topics 10
G Glass scale calibration at 74 degrees F? General Measurement Device and Calibration Topics 4
A Digital Timer Calibration Requirements General Measurement Device and Calibration Topics 3
Ron Rompen Barcode Verifier Calibration and Use Calibration and Metrology Software and Hardware 5
S Is Optical Parallel Set required calibration? ISO 17025 related Discussions 1
A Calibration of Power Supplies AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 2
D Calibration Process Flow Map Example Wanted General Measurement Device and Calibration Topics 3

Similar threads

Top Bottom