"Accurate to ±2%" means "64% of measurements should be within 2% of the actual value"?

#1
"The typical accuracy (reported as Accuracy Root Mean Square or Arms) of recently FDA-cleared pulse oximeters is within 2 to 3% of arterial blood gas values. This generally means that during testing, about 66% of SpO2 values were within 2 or 3% of blood gas values and about 95% of SpO2 values were within 4 to 6% of blood gas values, respectively." -- Pulse Oximeter Accuracy and Limitations: FDA Safety Communication

Is that normal? Technical specifications quote accuracy at only 2 standard deviations (69.1%) ?

I have always assumed that specifications are an outer bound of the accuracy, that I can be confident that my measurements are within the stated error, or at least perhaps 3 standard deviations (93.3%) of them, but only 64% of my measurements are expected to be within spec?!

-----------
Why I ask: I was looking for an accurate oximeter, so I read this 510k and the FDA statements such as the above.

A 510k for an oximeter (K191088) says that
Specification: SpO2 accuracy 70%~100%, ±2%;
Clinical studies: ARMS of 1.88
"the SpO2 accuracy performance results showed the Checkme Pro Health Monitor with external SpO2 sensor to have an ARMS of 1.88 during steady state conditions over the range of 70%-100%."

Is ARMS stated in standard deviations and perhaps anything less than 2 is a pass?
 
Elsmar Forum Sponsor
#2
Yes, for the Wellue Checkme O2 Max, the manual says: "about two-thirds of the measurements are expected to come within the specified accuracy range compared to CO-oximeter measurements."

Outstanding question: Is this the common definition for medical devices and other industry?

Back in university physics labs, we would track the error through all of out calculations but I don't recall ever realizing that the error we stated was only about 66% of the possible error.
 

Tidge

Trusted Information Resource
#4
I must admit: I was hoping more folks would chime in on this topic, particularly from anyone with experience involving specific devices where there is a consensus standard that explicitly makes statements about reporting "standard deviations" and "confidence intervals". In the absence of any such consensus standard, assuming frequentist statistical methods... when told that a measurement is accurate within ±1σ my mind immediate begins thinking along the lines of "this means that within the hypothetical ensemble of all possible measurements of that parameter (by this technique), 68% of the reported measurements would be that close to the measurement I recorded."
 

Steve Prevette

Deming Disciple
Leader
Super Moderator
#5
I will say in the nuclear world - as you are scanning something to see if it it is radioactively contaminated you hold the probe at a certain distance and move at a certain (slow) speed over the entire surface. The math has been done taking into the one standard deviation limits to say - we are quite certain you will detect something contaminated at this level if you follow that speed and distance from the object.
 
Thread starter Similar threads Forum Replies Date
I Can we use pin gauges to measure an accurate Cpk ? Capability, Accuracy and Stability - Processes, Machines, etc. 9
Ed Panek Ideas to keep quotes for consulting service accurate on not inflated Consultants and Consulting 11
N Accurate calculations of First Time Quality / First Time Yield Misc. Quality Assurance and Business Systems Related Topics 3
Q How can you justify using a more accurate Pin Gage class? General Measurement Device and Calibration Topics 16
Moumen H Controls to ensure that Quality Plans are accurate APQP and PPAP 2
R Accurate Quarterly Inventory Turn Numbers Quality Manager and Management Related Issues 3
C Accurate Measurement Method for Flash and Mismatch @0.002" Inspection, Prints (Drawings), Testing, Sampling and Related Topics 8
optomist1 Accurate Measure of Field Performance Capability, Accuracy and Stability - Processes, Machines, etc. 2
marmotte Definition Monitoring - Accurate definition of Monitoring - IEC 60601 Definitions, Acronyms, Abbreviations and Interpretations Listed Alphabetically 1
T Accurate Method of Internal Radius Check Inspection, Prints (Drawings), Testing, Sampling and Related Topics 18
J Printers and Plotters that do ACCURATE Mylar or Plastic Film Overlays Design and Development of Products and Processes 6
SteelMaiden What does 99.9% success look like if processes were 99.9% accurate? Training - Internal, External, Online and Distance Learning 19
Marc Keep Your Resume Accurate - Lying on Resume cost CEO his job and $25.1 million Career and Occupation Discussions 3
Marc Definition Accuracy (Accurate) - Definition Definitions, Acronyms, Abbreviations and Interpretations Listed Alphabetically 4
R 100% Inspection - 80% Accurate - Quote from Juran (?) Inspection, Prints (Drawings), Testing, Sampling and Related Topics 94
D Measuring ID's: Which is the most accurate? CMM / Sunnen Gage / Plug Gage? General Measurement Device and Calibration Topics 3
C Classification - "Metabolic Means" EU Medical Device Regulations 5
Q Does anyone know what this call out means? Manufacturing and Related Processes 3
E How to test 8.7.4.7 d) ? What "not protectively earthed accessible part" means? IEC 60601 - Medical Electrical Equipment Safety Standards Series 3
B IEC 60601 - Creepage Distance - Relay that acts as a means of physical mechanical protection Process Maps, Process Mapping and Turtle Diagrams 0
M Informational How to perform a clinical evaluation of medical devices – Part 2 – Level of clinical evidence and what sufficient clinical evidence means Medical Device and FDA Regulations and Standards News 9
M Applicability of Means of Protection, working voltage in an Automated External Defibrillator IEC 60601 - Medical Electrical Equipment Safety Standards Series 0
J President does not know what the Quality Policy means ISO 13485:2016 - Medical Device Quality Management Systems 10
G What degrees of freedom means in uncertainty budget and how do I set the value Measurement Uncertainty (MU) 2
P Uncertainty and Significantly Different Means - Buffer Solutions Measurement Uncertainty (MU) 4
G Definition Parallel Loop - Does anyone know what the term Parallel loop means? Definitions, Acronyms, Abbreviations and Interpretations Listed Alphabetically 2
N Explanation - Quality means doing it right when no one is looking ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 5
P Question on Table 6 "Test voltages for solid insulation forming a MEANS OF PROTECTION IEC 60601 - Medical Electrical Equipment Safety Standards Series 1
A Uncertainty Measurement - What this accuracy specification "1% + 10^5": means? General Measurement Device and Calibration Topics 2
F How to provide 2 MOPP (Means of Patient Protection) - Heating Rod IEC 60601 - Medical Electrical Equipment Safety Standards Series 3
F MOP (Means Of Protection) - Creepage and Clearance questions IEC 60601 - Medical Electrical Equipment Safety Standards Series 19
L What does "conformance matrix" means in terms of ISO 9001:2008? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 5
F Evaluation of MOP (Means of Protection) IEC 60601 - Medical Electrical Equipment Safety Standards Series 8
B Use of Y1 capacitors for MOPP (means of patient protection) IEC 60601 - Medical Electrical Equipment Safety Standards Series 1
E Correlated data or not? Determining the significance of differences in means Statistical Analysis Tools, Techniques and SPC 3
Q Definition C - What the "C" in C=0 Sample Plan means Definitions, Acronyms, Abbreviations and Interpretations Listed Alphabetically 10
J Minitab Sample Size Calculation: 1 Sample t vs Minimum Sample Size for Means Using Minitab Software 3
L What exactly does Configuration Management in ISO 9001, Clause 7.5.3 means? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 8
somashekar Friday the 13th - What it means to people around here. Coffee Break and Water Cooler Discussions 9
M Can I continue using IEC60601-1 Ed2 PSU for MOOP (Means Of Operator Protection) IEC 60601 - Medical Electrical Equipment Safety Standards Series 7
B Test of Means or Medians for Non Normal Populations? Using Minitab Software 6
F What Compliance to ISO 9001, 6.2.2 Competence, Awareness and Training Means ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 35
A Customer Demands (MSA=Bias + GRR/2). Can somebody explain what it means? Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 6
J Ford Acronym - What the acronym "ym" means in Ford lingo? Customer and Company Specific Requirements 1
S How to Design the Means of Protection for Home Use Unit IEC 60601 - Medical Electrical Equipment Safety Standards Series 12
M Definition Disposition Definition - Disposition means Prevention of Recurrence? Definitions, Acronyms, Abbreviations and Interpretations Listed Alphabetically 24
M Surface mount resistor as a MOPP (Means Of Patient Protection) IEC 60601 - Medical Electrical Equipment Safety Standards Series 10
Wes Bucey Keeping your cool means keeping your job Career and Occupation Discussions 12
D Means Of Protection of the Video OUT of an Internally powered device IEC 60601 - Medical Electrical Equipment Safety Standards Series 5
bobdoering From SME: What the Stimulus Plan Means to Manufacturers World News 1

Similar threads

Top Bottom