IEC 62304 compliance - Code reviews as part of verification strategy

KrishQA

Starting to get Involved
Hello All,

I have been introduced to a product under development. It is a Class C Medical device as per 62304.

The team has already performed some verification (System tests). Now, the verification strategy consists of a combination of the following testing levels:
  1. Code review
  2. Unit
  3. Integration
  4. Software system
What this resulted in, is that some software requirements are only tested by code review or unit testing and have no explicit system tests associated.

Is this considered okay, or does each and every software requirement need to be tested (not including code review and unit tests)

Looking forward to the answers. TIA
 

yodon

Leader
Super Moderator
Depends on the (software) requirement. Some likely cannot be verified at a system level (may need a debugger for error injection, for example). If you have a requirement that (and I'm carrying it to an extreme) a variable be named "Bob" then a code review is the only way to verify. Some may need a combination of the above to fully verify.
 

Tidge

Trusted Information Resource
First, the homily: System testing should be the LAST level of verification testing. It is not uncommon that developers do "system testing" and discover that one or more requirements was not tested (or unable to be tested); this is evidence of an incomplete development process. It appears that the ordinal numbering of the verification strategy in the original post recognizes this at some level.

Now to answer the question

...does each and every software requirement need to be tested (not including code review and unit tests)

Every requirement must be tested. Generally it is not a good idea to write (system) requirements that cannot be tested at the system level. IEC 62304 does not mandate that requirements be written at such a low level that code reviews and/or unit tests are the only mechanisms for testing requirements. It is encouraged that developers perform code reviews and unit tests to provide some evidence that compiled code will work prior to moving to integration testing. A simple example:

Code may be developed for a device that will display units of a patient medicine dose in units of 1 mg (REQ 1), but the device is to signal if the dose exceeds 10 mg (REQ 2). The code review (for the part of signal system monitoring the dose) could examine the dose variable in smaller units, such as 0.1 mg and the unit test could explore the behavior at the boundary 10 +- 0.1 mg, even though the system display might only be 9, 10, 11 mg. In this case the code review and the unit test is providing evidence that the signal system is behaving, but I would still expect system level testing for each requirement as written.

Some developers prefer to write requirements that prescribe design solutions, but this is a bad practice. In such cases: if the design solution changes, those developers will find themselves returning to a much earlier part of the design process which makes everything more difficult.
 

KrishQA

Starting to get Involved
Depends on the (software) requirement. Some likely cannot be verified at a system level (may need a debugger for error injection, for example). If you have a requirement that (and I'm carrying it to an extreme) a variable be named "Bob" then a code review is the only way to verify. Some may need a combination of the above to fully verify.

That does make sense. Also to add, I did some more digging and some of the software requirements will be tested as part of the whole system testing (mapping the system requirements to tests).

So what I read is that under specific conditions it might be okay to rely only on Code review/Unit testing as long as a justifiable explanation is provided.
 

KrishQA

Starting to get Involved
First, the homily: System testing should be the LAST level of verification testing. It is not uncommon that developers do "system testing" and discover that one or more requirements was not tested (or unable to be tested); this is evidence of an incomplete development process. It appears that the ordinal numbering of the verification strategy in the original post recognizes this at some level.

Yes, software system testing is indeed the last step of the software verification process. However, some software requirements are only verified by code review/unit testing. I see that it might be okay under certain conditions, however the software requirements themselves have to be written in a testable way (which is mentioned in 62304), which would mean it should be possible to perform system testing for all of them.
 

yodon

Leader
Super Moderator
Some developers prefer to write requirements that prescribe design solutions, but this is a bad practice. In such cases: if the design solution changes, those developers will find themselves returning to a much earlier part of the design process which makes everything more difficult.

Amen!
 
Top Bottom