Product Update and executing only affected System Tests, leaving out unaffected ones

Auxilium

Involved In Discussions
#1
Hi!
I am relatively new to Regulatory Affairs and I stand before the problem to make our Software Development Process leaner specifically regarding the Testing of our Medical Device Software (Standalone AI).
So far, our company has even with the slightest update of our product (e.g. new Version 4.1 to 4.2) repeated all of our defined System Tests.
Our thought is: can we make this Step leaner and only repeat the System Tests that the product update affects and not repeat all the other tests which are not affected?
I got the answer: yes, it is possible from an expert but the person was in a hurry so he couldn't tell me.

My question is: How would you specifically justify this from a regulatory perspective. specifically in the 62304 Norm or other applicable standards/guidelines.
Would you use the Software Architecture for that argumentation and simply write a statement somewhere for why we have left out the other tests?

Thank y'all!
 
Elsmar Forum Sponsor

yodon

Leader
Super Moderator
#2
The way we approach this is through regression test planning. We write a kind of mini V&V Plan. Tests selected are based on direct impact, but also we select those "around" the directly impacted area. We look at code, requirements, and design when justifying the test set. Oh, we generally always define a "safety set" that is executed each time to be sure we haven't broken any safety functionality.

62304 mostly talks about regression testing in general terms, leaving it up to you decide what gets (re)tested. Here's an excerpt:

Regression analysis and testing are employed to provide assurance that a change has not created problems elsewhere in the MEDICAL DEVICE SOFTWARE. Regression analysis is the determination of the impact of a change based on review of the relevant documentation (e.g., software requirements specification, software design specification, source code, test plans, test cases, test scripts, etc.) in order to identify the necessary regression tests to be run.

Basically, use the materials you have and document the justification for what you do / don't test.
 

Junn1992

Quite Involved in Discussions
#3
The way we approach this is through regression test planning. We write a kind of mini V&V Plan. Tests selected are based on direct impact, but also we select those "around" the directly impacted area. We look at code, requirements, and design when justifying the test set. Oh, we generally always define a "safety set" that is executed each time to be sure we haven't broken any safety functionality.
Hi yodon, can this regression testing be determined after a problem is discovered? Meaning that we discover the software has some minor bug, and we don't want to perform whole of system regression testing. I am assuming here that whole of system regression testing is the default during the planning phase post-release.

So during problem resolution process post-release of the software, we say 'oh here is a minor bug'. And then we do proper change control, configuration item control, and for the testing we say 'we are going to create a testing protocol B for software changes due to this minor bug, because of...'.

I guess the problem here is that, are we going to create a testing protocol everytime a new problem is discovered? I think auditors might not be pleased with this. Wonder if there was an easier way to do this.

Hope I was being clear and thanks for the help!
 

Tidge

Trusted Information Resource
#4
Hi yodon, can this regression testing be determined after a problem is discovered? Meaning that we discover the software has some minor bug, and we don't want to perform whole of system regression testing. I am assuming here that whole of system regression testing is the default during the planning phase post-release.

So during problem resolution process post-release of the software, we say 'oh here is a minor bug'. And then we do proper change control, configuration item control, and for the testing we say 'we are going to create a testing protocol B for software changes due to this minor bug, because of...'.
The SOFTWARE ARCHITECTURE and a traceability document (requirements <-> implementation <-> testing) is what lets you properly plan efforts for discovering the nature of, fixing and testing identified defects. Don't look for the root cause of the defect in areas not allocated to the impacted requirements, don't repeat testing in areas unaffected by the fix.

I guess the problem here is that, are we going to create a testing protocol everytime a new problem is discovered? I think auditors might not be pleased with this. Wonder if there was an easier way to do this.
I see two choices:
  1. You repeat (parts of) previously used test protocols.
  2. You create new protocols specific to the scope of the new project.
Many people prefer the first; I prefer the second. Here are some reasons why I prefer option 2: Option 1 has an aroma of "just do what was done before/trust the people that came before you"... but of course, the previous testing allowed a bug to escape! Directly recycling previous work absolves the current generation from applying critical thinking skills to the problem being worked on. I also happen to find multiple executions of the same protocol to be more problematic than it is worth... as soon as people start applying more thought on how to uniquely identify different executions than is being applied to the actual tests, I can tell that priorities are wrong.
 

yodon

Leader
Super Moderator
#5
  1. You repeat (parts of) previously used test protocols.
  2. You create new protocols specific to the scope of the new project.
We take a hybrid approach: update the protocols to cover the issue(s) found and then use parts of previous protocols to give assurance that the changes didn't break anything.

I think auditors might not be pleased with this.
Speculating on what you think an auditor might or might not be pleased with is the wrong approach. Comply with the standards / regulations. Get in a defensible position. Your questions should be "is this compliant?"
 
Thread starter Similar threads Forum Replies Date
E Gamma Sterilization Product Update Manufacturing and Related Processes 2
A Timeframe for Product Label Update after Company Name Change 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 3
S Product Registration Process - Update from Emergo Other Medical Device Regulations World-Wide 0
A Combination product Canada Medical Device Regulations 3
E UDI on product/packaging levels EU Medical Device Regulations 2
M Drug-Device Combination product CE Marking (Conformité Européene) / CB Scheme 1
H Product Safety & Conformity Representative (PSCR) Training Company IATF 16949 - Automotive Quality Systems Standard 0
X Design stage overview (Product specification) EU Medical Device Regulations 3
9 Responsibility for Product Paid for but not yet Shipped AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 4
M Product Acceptance Software (PAS) PROCEDURE (BOEING D6-51991) AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 3
S Combo Product Assembly Process OQ Qualification and Validation (including 21 CFR Part 11) 0
V ADDING NEW MEDICAL DEVICE / Product, WHEATHER THIS AFFECTS EXISTING ISO 13485 CERTIFICATION? ISO 13485:2016 - Medical Device Quality Management Systems 4
A Product Liability Insurance coverage for EU Importer by UK Manufacturer EU Medical Device Regulations 2
sonflowerinwales Outgassing - product inside a sealed housing Manufacturing and Related Processes 1
L Temporary Product Deviation ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 10
FRA 2 FDA Product NCRs- Quality Review Nonconformance and Corrective Action 6
armani 7.1.5 and design and development of product ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 10
M Aftermarket product PPAP approval IATF 16949 - Automotive Quality Systems Standard 1
G How to find applicable standards for a new product? EU Medical Device Regulations 6
T Link GMDN code and FDA product code US Medical Device Regulations 5
S Combination Product - Packaging US Food and Drug Administration (FDA) 6
S Can a product option change the product class? CE Marking (Conformité Européene) / CB Scheme 2
H Regarding EMC & Safety retesting requirements of the product CE Marking (Conformité Européene) / CB Scheme 2
Q Product audit assessment IATF 16949 - Automotive Quality Systems Standard 4
S Submit under a new product code in a 510k? US Medical Device Regulations 5
P Product Quality Review API-GMP Manufacturing and Related Processes 0
D Shelf life of non sterile class IIb product. ISO 13485:2016 - Medical Device Quality Management Systems 1
K Can I make an exclusion of Design and Development in ISO 13485:2016 if my product is not regulated ISO 13485:2016 - Medical Device Quality Management Systems 12
M How to show the effect of the failure mode on the manufacturing process as a customer of product design process? FMEA and Control Plans 3
J Enough for PSUR to cover product family? (Procedure packs, not the legal manufacturer) EU Medical Device Regulations 1
B Product registration CE Marking (Conformité Européene) / CB Scheme 2
V Product naming: Same name - different perfume & status CE Marking (Conformité Européene) / CB Scheme 0
E Marketing, product and system requirements IEC 62304 - Medical Device Software Life Cycle Processes 2
C PPAP requirements for consumer product APQP and PPAP 2
C How to place software version for SaMD product in HIBC secondary data structure (UDI-PI)? Other US Medical Device Regulations 4
T Does marketing company require CE mark if manufacturer has CE mark on product? EU Medical Device Regulations 5
S Classification of product (measuring or not?) CE Marking (Conformité Européene) / CB Scheme 5
Ron Rompen 4.4.1.2 Product Safety Compliance IATF 16949 - Automotive Quality Systems Standard 29
M Supplier Control for Unique Product ISO 13485:2016 - Medical Device Quality Management Systems 6
R Shelf life of product Other Medical Device Related Standards 4
D One Software as Medical Device product or two? EU Medical Device Regulations 4
J Aerospace Product Key Characteristics AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 2
F Product classification according to the 60601-2-57: Particular requirements [...] of non-laser light sources [...] IEC 60601 - Medical Electrical Equipment Safety Standards Series 0
M Reworking MDD product w MDR labeling CE Marking (Conformité Européene) / CB Scheme 5
D Regulations covering Sterile product shipments ISO 13485:2016 - Medical Device Quality Management Systems 1
M R&R Studies - how many? per product? Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 3
A Eudamed actor registrations if you change AR for MDR product EU Medical Device Regulations 0
S Reliability issue - Frequent component failure of the product Reliability Analysis - Predictions, Testing and Standards 2
R Statistical Methods for comparing test and reference product equivalence for quality attributes US Food and Drug Administration (FDA) 3
T Do I need to add non-product related service providers to my ASL? AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 12

Similar threads

Top Bottom