IEC 60601 - Limits of agreement as Essential Performance

[MedicalResp]

Hi All,

Supposedly I have a medical device that measures and logs respiratory rate (it is not defined as an Alarm Monitor).

The efficacy of the device in clinical testing is determined with respect to a gold standard measurement (i.e by comparing the respiratory rate measured by the test device an a gold standard method). This is commonly done using a statistical method called BLAND ALTMAN which, in short, provides one with the "mean bias" between the two methods and the limits of agreement (i.e. the limits where 95% of the data will fall with 95% confidence)

For IEC 60601 one must define the Essential Performance which is the clinical function where loss or degradation beyond the limits results in an unacceptable risk.
During the safety and EMC testing we use a simulator that simulates respiration in a constant rate and we measure the respiration rate.

My question is if it is logical to use the same parameter used in the clinical testing - "limits of agreement = +/-X" as the essential performance, thus preforming a statistical post-analysis of the data collected during the safety&EMC tests making sure it does not exceed the pre-defined limits +/-X.

Thanks in advance.

 

[yodon]

Not to sidetrack the question but if the device did fail, what is the unacceptable risk that would be realized? (The main point being that some devices do not have Essential Performance.)

 

[MedicalResp]

That is actually a good question.
Since respiratory rate is a vital sign and the device is intended for hospitals , it must be defined as a medical device.
The risk here is that in a case of performance degradation, the medical staff would receive an unreliable clinical information. Although, in the current scenario it would be used in conjunction with other type of clinical information, it still might result in an unsuitable medical care.

The issue I have raised is due to the fact that we must define limits of "accuracy" for the EP of measuring respiratory rate.
Since for our device there is no guiding standard as opposed pulse-oximeter ( IEC 80601) which defines the accuracy, the question is what type of "accuracy" estimate would be most reasonable/accceptable to use.

If, for example, we say that the respiratory rate accuracy is +/- 10% then the immediate question would be how to calculate this, as we might get up to 60 respiratory rate value per minute. Does each value must be +/-10% with respect to the reference or would it be more logical to use a statistical analysis (like i suggested or other) over the duration of the test (e.g. EMC )

 

[Peter Selvey]

While there is a lot about "essential performance" which is nonsensical, there's actually a common sense bit at the end of the clause that says:

"Where this standard requires that ESSENTIAL PERFORMANCE is to be maintained following a particular test, compliance is checked by inspection and, if necessary, by functional test(s) that demonstrate the MANUFACTURER'S specified limits are maintained or the ME EQUIPMENT or ME SYSTEM transitions to a safe state as defined by the MANUFACTURER."

So, for example a water proof test shows that no water got anywhere near the electronics involved with measurement of respiration, then no need to test essential performance.

On the other hand another test might feasibly affect the electronics (say low or high temperatures?) you could devise a simulator test that gives reliable results that would allow you to detect if the electronics has been affected.

In other words, you can tailor the "functional tests" to suit. It does not need to be the full evaluation (e.g. multiple samples, statistical analysis) every time the standard calls out essential performance.

This does though highlight one huge oversight in the standard in that there is no baseline test to establish essential performance in a set of reference conditions. In practice, a manufacturer would do the more detailed baseline test first and then start to throw various "stress tests" such as EMC, water ingress, normal temperatures, storage, vibration, ageing, thermal shock, defib etc, and verify that nothing changed, usually by inspection or simplified tests based on the nature of "stress" being applied.

Also, in the case of a simplified test, it is normal to run the that test first to set a baseline, and possibly apply stricter limits than are defined in the essential performance, in order to detect whether the "stress" caused any variations. A fail result for this stricter limit should either trigger more detailed tests or investigate/eliminate the cause of the variation.

Anyway, without establishing a baseline the whole logic of tailoring the test to suit falls down. Even the above example of the waterproof test falls down logically if essential performance was never established in the first place.