5.5.3 - Software Unit Acceptance Criteria (Risk Control Measures)

[andypatkinson]

Hi,

When it comes to verifying our software units that contain a RCM, we use unit tests to evaluate the RCM implementation (is it present and working as expected) and where we cannot test for it (for one resaon or another) we verify by code inspection.

Does anyone see any probem with this?

I have heard from other medical device SW developers ('around town') that because the example in the standard says:

NOTE Examples of acceptance criteria are:
? does the software code implement requirements including RISK CONTROL measures?

That code inspection / review is the only acceptable verification method and that verifying RCMs with unit tests is not 'meeting the intent of the standard'. I have a hard itme accepting that - for me we're verifying that the RCM is present in the unit and I choose to do that by 1st, testing and 2nd, inspection (but only if I have to, i.e. I cannot test for it).

Am I on the money or really going against the 'intent of the standard'?

Thanks in advance......

Andy

 

[yodon]

In general, I would agree that some requirements can only be verified through unit test (weak) or code inspection (weakest).

For risk controls, though, the expectation is that you demonstrate the control is available and effective at the time of need. It's clear that in some cases, demonstration could only be shown via unit-level testing. I would have a hard time defending verification of a risk control via code inspection - I don't think you could defend the effectiveness of the control.

Can you share a scenario where ONLY code inspection could verify a risk control?

 

[andypatkinson]

Hi Yodon

Thanks for the reply.

The issue seems to be that because the example given in the standard talks about 'code', that the expected method of verifying this is via a code inspection, rather than testing. I agree that testing is a better method and is my preference.

Given that the standard talks about verifying risk control measures and traceability of requirements to tests, I don't see the point of section 5.5.3 and the focus on checking that a risk control measure had been implemented (if we're going to test for it before releasing the software, e.g. all verification will be completed before release).

Perhaps this is another example of a 62304 clause that if poorly written and simply causes confusion / misinterpretation.

Any further advice appreciated, Yodon

Regards, Andy

 

[yodon]

I can't argue for the merits of the verbiage in the standard. But maybe some discussion can clarify what I believe the intent is (at least I can express my opinion).

5.5.2 is where you define how you establish verification methods for the units. Note that it's limited to class B and C (potential for patient injury). Note also that they explicitly indicate that there are other methods besides test for unit verification. For some units; e.g., those that are on one side of an interface, maybe you do some testing with a debugger to confirm the unit properly handles all data passed - both valid and invalid (something that should not be possible in normal operations).

5.5.3 establishes unit acceptance criteria and again only required for class B and C. If you have units that implement risk controls, the expectation, as I read, is that you do a code inspection to verify the implementation of the control. This is NOT requirements verification; this is how you accept the UNIT. This is a very low-level acceptance gate. The examples are quite appropriate for such a low-level verification: did you write the code per your standards, did you conform to the interface defined, etc.

So for class B and C software, if you have risk controls implemented in software, you're effectively saying that you are ensuring patients aren't harmed (or worse) via these controls and so the expectation is that you take extraordinary measures to ensure it's done right.

If you consider the 'V' model, you would have a low-level confirmation at the implementation level to confirm (via code inspection) that the risk control was implemented properly. (There's merit to this given testing can't or won't necessarily test every possible scenario.) Then moving up the food chain, once you get to the requirements level, you would have some type of functional testing to formally verify the risk control is effective.

Keep separate unit acceptance from requirements verification. Hope that makes sense.