ISO 13485:2015 Software Validation IQ/OQ/PQ

[YoungConner]

I need some support in running test scripts for my software validation. For the User Requirements, these are tested in the PQ. The CS functional requirements are tested in the OQ. Please confirm.

 

[yodon]

There are no specific "rules" for how to do software validation (I personally like the IQ/OQ approach but rarely use PQ for software). Nor are you required to have a requirements hierarchy. If your internal procedures require all that then that's what you need to do.

 

[LUFAN]

Combination of ISO 80002 and IQ/OQ/PQ for me. IQ generally not applicable for cloud applications, but overwise a hosted environment would be. OQ usually testing OOTB configurations, and PQ, if any configurations were performed to OOTB setup.

 

[Tidge]

In the context of NPS software validation, you have a variety of options. The actual choice you make ought to be spelled out in a validation plan document. Some organizations do proscribed activities by procedure, but it is pretty much a uniform expectation that for NPS validations there be some sort of Validation Report (typically concluding with a statement "this software system is validated for its intended use"), and I can't imagine generating a unique Report without a unique Plan. YMMV.

Some personal disclosures:

I have a personal distaste for adopting the tenets and terminology of Process validation (IQ/OQ/PQ) to NPS software validation. Many organizations do this; I think it demonstrates a lack of both experience and understanding.

I believe the "V-model" for validation leads to more confusion that clarification. The OP does not mention the V-model specifically, but the implied mapping between "high level requirements" (i.e. URS) and "PQ" (and mapping FRS to OQ) has the aroma of V-Model thinking.

Within the regulated medical device industry, the need for NPS validation (such as it is) is two-fold:

1. The possible effects of the NPS on patients and users is understood (via risk analysis) and the risk are suitably controlled,
2. The system meets the intended needs of the organization (for the entire life of the NPS system).

A relatively simple robust model of (initial) NPS validation to determine that the system meets its intended use (sidestepping planning, risk assessment) is:

1. Establish the requirements for the system (This is typically from the Intended Use, any necessary controls identified from Risk Analysis, and "user requirements")
2. If necessary, decompose the high-level user requirements into specific functional requirements and design choices which will satisfy the high-level requirements
3. analyze the design choices and challenge the specific functional requirements (most users of NPS won't set out to explicitly challenge the software, they just want it to work)
4. allow typical users to exercise the software in their typical ways to make sure that (as implemented) it is meeting their needs.

Traceability between these items should be established. Systems where you have no control over the detailed implementation may not even require steps 2 and 3 (if the risk to patients/users is low enough).

You can assign TLA (two/three-letter acronyms) to the various artifacts, but it isn't important what you call them... provided you aren't sowing confusion. Keep in mind that manufacturers of pharmaceuticals and manufacturers of medical devices don't agree (by consensus!) on the meanings of OQ and PQ, even in the context of Process Validation!

 

[LUFAN]

A relatively simple robust model of (initial) NPS validation to determine that the system meets its intended use (sidestepping planning, risk assessment) is:

1. Establish the requirements for the system (This is typically from the Intended Use, any necessary controls identified from Risk Analysis, and "user requirements")
2. If necessary, decompose the high-level user requirements into specific functional requirements and design choices which will satisfy the high-level requirements
3. analyze the design choices and challenge the specific functional requirements (most users of NPS won't set out to explicitly challenge the software, they just want it to work)
4. allow typical users to exercise the software in their typical ways to make sure that (as implemented) it is meeting their needs.

That's a good approach and mostly inline with what I expect FDA's Computer System Assurance guidance to look like. I think my challenge has always been coming up with an approach that's accepted sans any real universal harmonization. FDA's last guidance was issued almost 20 years ago, and a refresh is desperately needed to modernized the 'validate to death' approach that's become commonplace, and move to a model more like yours. ISO 80002 is helpful but not perfect as well.

 

[Tidge]

I am desperate for the CSA draft to be released because without it we (medical device manufacturers) can't make anything like an appeal to authority (when pushing back against third parties). I have encountered some very disingenuous players in the field of NPS validation for medical devices. A new guidance will (unfortunately) breathe new life into such folks, but they will still have to execute to a smarter way of doing things... which I'm not sure most of them will be able to do.

 

[yodon]

The local ASQ Medical Device Discussion group I participate in has started a 3-part (virtual) meeting series on (the expected) CSA approach. (And yes, it will be a tremendous improvement over the current validation approach.) I believe they said that the CSA approach is already in practice in a few places. Not sure I'm ready to jump in myself until the guidance is published. I can see if they would allow the slides and support material to be posted here if anyone is interested.

 

[YoungConner]

The local ASQ Medical Device Discussion group I participate in has started a 3-part (virtual) meeting series on (the expected) CSA approach. (And yes, it will be a tremendous improvement over the current validation approach.) I believe they said that the CSA approach is already in practice in a few places. Not sure I'm ready to jump in myself until the guidance is published. I can see if they would allow the slides and support material to be posted here if anyone is interested.

I am interested. I would like to see their approach to system validations and examples of how it will be rolled out. Hopefully, it is less confusing than the "V-model".

 

[LUFAN]

The local ASQ Medical Device Discussion group I participate in has started a 3-part (virtual) meeting series on (the expected) CSA approach. (And yes, it will be a tremendous improvement over the current validation approach.) I believe they said that the CSA approach is already in practice in a few places. Not sure I'm ready to jump in myself until the guidance is published. I can see if they would allow the slides and support material to be posted here if anyone is interested.

I listened in as well. I thought it was helpful, but I too am hesitant until I have a document I can leverage as a backbone of future CSV activities.

I am interested. I would like to see their approach to system validations and examples of how it will be rolled out. Hopefully, it is less confusing than the "V-model".

It was interesting. The presenter suggested assigning risks for your main validation topics, for example, 1 to 5, and if the scores come in at 1/2/3 you could perform unscripted testing where screenshots are not required, only doing so when a deviation is encountered, and for scores of 4/5 you still create your normal script and execute it with screenshots.

 

[Tidge]

Screenshots (as objective evidence of NPS assurance) is one of the items that the FDA working group felt was overblown, so I am surprised that that even came up.