IEC 62304 - Testing Independance

[KrishQA]

Hello All,

We have a Class C medical device. I was reviewing some of the testing documentation and find that some of the software system tests have been performed by the developers themselves.

The test cases were reviewed and approved by different people but some of them were still performed by the developers. Would this be allowed?

 

[yodon]

While it's probably not the best practice (due to a variety of reasons), there's nothing in the standards that would explicitly disallow it. Testing to requirements should generate objective evidence that the requirement was met. If, in fact, the testing did generate objective evidence of compliance, you should be in a pretty defensible position. As noted, there may be some landmines with this approach. If the requirement is open some to interpretation, the developers would, of course, be biased by how they implemented. Similarly, developers generally think "happy path" so the testing may not be as rigorous as you'd want, especially for a Class C device. So be careful.

 

[Tidge]

If I were trying to detonate some of the landmines suggested by @yodon I would ask:

• Were the test criteria established prior to the developers doing testing?
• Did the testing occur on a controlled release of the software system?

If either of these questions are answered "no", then it is very likely that the development model of 62304 was not followed.

 

[KrishQA]

@yodon
I have asked the team to retest those specific test cases again and get them approved.

There is evidence of performing the test, and the results have been captured, however there are no screenshots, logs or other evidence that can be looked at and verified.

@Tidge
The test criteria was established prior to testing and it has been done in a controlled release. For this retesting, we are creating a change request to make the changes to documentation as well, since the release notes are already signed.

 

[Tidge]

@KrishQA It sounds like you have the bases covered. Some companies may not have the resources to allow for a completely different group of people to do the system testing, even if it is a best practice.

If you have pre-approved protocols, with instructions and pre-determined expected results, you may not need screenshots. My own bias is to rely on screenshots only for those results where it would otherwise be too much trouble to record the information. Examples include: GUI testing, or if there are multiple items simultaneously displayed. Keep in mind that if screenshots and/or logs are used as evidence of system testing that those need to be preserved.

My experience has been that there is a certain generation of folks that are rather obsessive about screenshots. Too many screenshots can clutter up test results; screenshots are not impossible to manipulate so it isn't as if they are a 'better' form of proof.

 

[yodon]

There is evidence of performing the test, and the results have been captured, however there are no screenshots, logs or other evidence that can be looked at and verified.

My approach is that the tests should gather sufficient evidence that an independent reviewer (with sufficient knowledge) should be able to draw the same conclusion that the tester made.

Too many screenshots can clutter up test results;

AMEN! I once had a client that insisted on a screenshot for every screen transition. At the end, we had stacks of paper over a foot high and 5 feet long. (And, honestly, most of the screenshots did nothing to support the evidence that a requirement was met.) Be judicious!