"Accurate to ±2%" means "64% of measurements should be within 2% of the actual value"?

[wwcanoer]

"The typical accuracy (reported as Accuracy Root Mean Square or Arms) of recently FDA-cleared pulse oximeters is within 2 to 3% of arterial blood gas values. This generally means that during testing, about 66% of SpO2 values were within 2 or 3% of blood gas values and about 95% of SpO2 values were within 4 to 6% of blood gas values, respectively." -- Pulse Oximeter Accuracy and Limitations: FDA Safety Communication

Is that normal? Technical specifications quote accuracy at only 2 standard deviations (69.1%) ?

I have always assumed that specifications are an outer bound of the accuracy, that I can be confident that my measurements are within the stated error, or at least perhaps 3 standard deviations (93.3%) of them, but only 64% of my measurements are expected to be within spec?!

-----------
Why I ask: I was looking for an accurate oximeter, so I read this 510k and the FDA statements such as the above.

A 510k for an oximeter (K191088) says that
Specification: SpO2 accuracy 70%~100%, ±2%;
Clinical studies: ARMS of 1.88
"the SpO2 accuracy performance results showed the Checkme Pro Health Monitor with external SpO2 sensor to have an ARMS of 1.88 during steady state conditions over the range of 70%-100%."

Is ARMS stated in standard deviations and perhaps anything less than 2 is a pass?

 

[wwcanoer]

Yes, for the Wellue Checkme O2 Max, the manual says: "about two-thirds of the measurements are expected to come within the specified accuracy range compared to CO-oximeter measurements."

Outstanding question: Is this the common definition for medical devices and other industry?

Back in university physics labs, we would track the error through all of out calculations but I don't recall ever realizing that the error we stated was only about 66% of the possible error.

 

[Steve Prevette]

I do know in the nuclear world, most geiger counters / radiation meters are stated at plus or minus one standard deviation.

 

[Tidge]

I must admit: I was hoping more folks would chime in on this topic, particularly from anyone with experience involving specific devices where there is a consensus standard that explicitly makes statements about reporting "standard deviations" and "confidence intervals". In the absence of any such consensus standard, assuming frequentist statistical methods... when told that a measurement is accurate within ±1σ my mind immediate begins thinking along the lines of "this means that within the hypothetical ensemble of all possible measurements of that parameter (by this technique), 68% of the reported measurements would be that close to the measurement I recorded."

 

[Steve Prevette]

I will say in the nuclear world - as you are scanning something to see if it it is radioactively contaminated you hold the probe at a certain distance and move at a certain (slow) speed over the entire surface. The math has been done taking into the one standard deviation limits to say - we are quite certain you will detect something contaminated at this level if you follow that speed and distance from the object.