Computer system validation approach for Minitab Statistical software

#1
I have been tasked with the process of validating Minitab for our facility. The company that I work for is a medical device manufacturer (Class I and II products) that needs to meet USA FDA Quality system requirements and ISO 13485.

As per our risk assessment, our main risk with using minitab is that it could yield wrong results (the software would be used to assist on the analysis fo process/design validation reports, CAPA investigations, and Complaint trends). We do aknowledge that a) Minitab has widespread adoption in the industry and b) it is a off-the-shelf (GAMP Category 3) software that will need black-box testing in order to check functionality.

My question is: Is it acceptable, for software validation test scripts, to test a reduced number of functions (lets say a sample) of all the functions minitab has available?

The acceptance criteria four our validation protocol would be something like test 10 functions, if they pass our test script, then the whole system is validated.
 
Elsmar Forum Sponsor

ECHO

Involved In Discussions
#2
Generally yes, you can using automated test scripts to validate 3rd party tools.

That said, I would first elaborate on the risk assessment you did above. As you stated, Minitab would be used in assisting other processes that will need to be manually reviewed. I would most likely consider this a low risk tool. Depending on how your internal tool validation SOP is written, you might be able to get away with not testing the tool.
If you do end up testing your tool, I would clearly state what functions are being tested and why other functions are not being tested.
 

Steve Prevette

Deming Disciple
Leader
Super Moderator
#3
I suggest that a lot depends upon if the calculations you perform in Minitab are life critical or safety critical. If so, I would target specific examples related to the modules used in Minitab for those functions rather than a global review of Minitab. For example, if I am reading dosimetry and computing a dose, I may want to have a zero or low dose example, an average or normal example, and an overdose example all calculated by independent means This is also a good opportunity to test the procedure for doing the calculation as you run the three cases through the process.
 

Miner

Forum Moderator
Leader
Admin
#4
I have been tasked with the process of validating Minitab for our facility. The company that I work for is a medical device manufacturer (Class I and II products) that needs to meet USA FDA Quality system requirements and ISO 13485.

As per our risk assessment, our main risk with using minitab is that it could yield wrong results (the software would be used to assist on the analysis fo process/design validation reports, CAPA investigations, and Complaint trends). We do aknowledge that a) Minitab has widespread adoption in the industry and b) it is a off-the-shelf (GAMP Category 3) software that will need black-box testing in order to check functionality.

My question is: Is it acceptable, for software validation test scripts, to test a reduced number of functions (lets say a sample) of all the functions minitab has available?

The acceptance criteria four our validation protocol would be something like test 10 functions, if they pass our test script, then the whole system is validated.
Does this help? Minitab Products and Services and Compliance with CFR Title 21 – Part 11

A software validation kit is provided to tailor the validation to your specific needs.
 
#5
Thank you Miner, I've got Minitab's validation kit. My question is if it is common practice, in this type of off-the-shelf software, to do an "abbreviated" approach to testing.


I suggest that a lot depends upon if the calculations you perform in Minitab are life critical or safety critical. If so, I would target specific examples related to the modules used in Minitab for those functions rather than a global review of Minitab. For example, if I am reading dosimetry and computing a dose, I may want to have a zero or low dose example, an average or normal example, and an overdose example all calculated by independent means This is also a good opportunity to test the procedure for doing the calculation as you run the three cases through the process.
My idea is similar to what Steve proposed. Basically using the predefined test cases provided by Minitab's validation kit but focusing only key functions that we use often instead of testing 100% of functions. I'd consider the software as validated based on these results. What do you think?
 

Tidge

Trusted Information Resource
#6
I have been tasked with the process of validating Minitab for our facility. The company that I work for is a medical device manufacturer (Class I and II products) that needs to meet USA FDA Quality system requirements and ISO 13485.

As per our risk assessment, our main risk with using minitab is that it could yield wrong results (the software would be used to assist on the analysis fo process/design validation reports, CAPA investigations, and Complaint trends).
Unless Minitab is actually making decisions for your company, you may have overestimated the risk of the software. Usually it's a human making the call and they don't defer to a software package (if they are qualified to understand the tool). Write what we must about the "lowest common denominator" but the typical user of minitab is not a complete knucklehead.

We (medical device manufacturers) are actually in the process of updating the validation status of Minitab; we plan to leverage the validation package provided, and we will further enhance our validation by applying a number of typical, foundational techniques for which there already exists "expected results" as extra assurance that the latest version is still behaving as past (validated) versions. We aren't the quality control department for Minitab; we're just folks familiar with the variety of things we're going to use Minitab for... and know how to check our own work (and that of the software package).
 

Hi_Its_Matt

Involved In Discussions
#7
As others have mentioned, your company's approach to computer system validation should be risk-based.
What approaches you take, and what evidence you generate/require is more or less up to you, so long as they are based on those risks.

Having said that, the FDA is expected to release a draft guidance this year about Computer Software Assurance, their "new term" for Computer System Validation. (Really its not just a new term, but rather a new paradigm for managing risks associated with software used in production and quality system support).

Compliance Group Inc has a great collection of YouTube videos/discussions that discusses what CSA entails, and several real-life case studies from manufacturers that have changed their approach. Several of them even have the FDA CDRH's Case for Quality program manager (Francisco Vicenty) as a participant.

USDM has also written a whitepaper that talks about the upcoming draft guidance, which I have attached here for everyone's ease.

My impression, based on this material, is that a lot of companies seem to have gone completely overboard with their approach to computer system validation, and have focused more on compliance rather than true quality (can anyone say "non-value add busywork"?).
The FDA is now clarifying that it encourages manufacturers to take a more holistic approach to assuring that computer software is appropriate for its intended use, rather than attempting to independently verify that every function and feature of a particular software tool works.

So @C. Tejeda with that, my question back to you is, based on (1) how your company uses Minitab, (2) it's overall role and purpose within your QMS, (3) the specific risks that are involved with its use, (4) the non-software based controls you have in place external to Minitab to manage any risks should they materialize, and (5) Minitab's standing within the industry as one of, if not the leading statistical analysis tools, do you think that validation test scripts are even required, in order to have a high degree of confidence that the tool will operate as you need it to?

edit: I am not affiliated with either Compliance Group or USDM. I just found these resources particularly useful as I have been working to re-vamp my company's approach this topic.
 

Attachments

Tidge

Trusted Information Resource
#8
My impression, based on this material, is that a lot of companies seem to have gone completely overboard with their approach to computer system validation, and have focused more on compliance rather than true quality (can anyone say "non-value add busywork"?).
Your impression is NOT wrong (for the Medical Device industry, other industries may have different experiences). A foundational reason for where we find ourselves in the morass is that for NPS it is has been incredibly rare for folks to intelligently answer "Compliant to what?" This is, of course, something of a trick question.

I personally witness a host of diverse root causes for how the (US, likely elsewhere) medical device industry ended up with the NPS path that the FDA is desperate to course-correct. The most polite reason, assuming competence and good intentions, is that the "waterfall model", "inputs-trace-to-outputs", and "we work in an industry that saves lives" all got tangled up in such a way that it required real backbone to answer "no" to questions like "how about one more screenshot?"

If competence is not guaranteed, and intentions are less-than-pure, the situation only gets more problematic.
 
Thread starter Similar threads Forum Replies Date
A Applying agile model for Computer system Validation Medical Device and FDA Regulations and Standards News 3
R SAP B1 Computer System Validation Qualification and Validation (including 21 CFR Part 11) 0
S Computer System Validation of Bioinformatics Pipeline Qualification and Validation (including 21 CFR Part 11) 5
E Computer System Validation - Migrating to SAP Document Control Systems, Procedures, Forms and Templates 5
D Validation of Computer and Network Equipment Test System Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 4
T Medical Device Manufacturer Document Control Computer System Validation Qualification and Validation (including 21 CFR Part 11) 4
AnaMariaVR2 Computer System Validation Resources for the FDA Regulated Industry Qualification and Validation (including 21 CFR Part 11) 2
B Computer System Validation Records - Document Destroy Date - Animal pharma company Records and Data - Quality, Legal and Other Evidence 3
G Computer software validation system in medical device industry ISO 13485:2016 - Medical Device Quality Management Systems 4
O Any info on release date of FDA “Computer Software Assurance for Manufacturing and Quality System Software” document? 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 0
F EMC testing for a system that is provided with a computer IEC 60601 - Medical Electrical Equipment Safety Standards Series 8
N Computer System Access and Security Procedure example wanted 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 1
M Strategy to allow for OTS computer use as part of a Medical Equipment System EU Medical Device Regulations 14
M Computer System for Inspection Balloons in Technical Drawings Inspection, Prints (Drawings), Testing, Sampling and Related Topics 8
M Options for OTS Computer as part of Medical Equipment System IEC 60601 - Medical Electrical Equipment Safety Standards Series 6
somashekar Information sharing: Week 21, 2010 (COMPUTER TERMS, NUMBER SYSTEM AND DIGITAL) The Reading Room 0
J Developing Manufacturing Routings, etc - Computer based Work Order system training Manufacturing and Related Processes 2
S Kanban System Game - A Computer Simulation Quality Tools, Improvement and Analysis 1
F An Implementation Plan for a New Computer System Other ISO and International Standards and European Regulations 2
A Electronic Records for FDA - Canned computer program for our Corrective Action system Records and Data - Quality, Legal and Other Evidence 3
Punk Philosopher Insight? Sales Engineer at UL + Computer Science Faculty + Ph.D Student Career and Occupation Discussions 5
John C. Abnet ...validation of computer software ISO 13485:2016 - Medical Device Quality Management Systems 14
Jim Wynne Windows 11 is Coming--Is Your Computer Ready? Coffee Break and Water Cooler Discussions 16
D Computer access, password control ISO 13485:2016 - Medical Device Quality Management Systems 6
P Blood establishment computer software EU classification EU Medical Device Regulations 0
D FDA Guidance on Computer Software Assurance versus 21 CFR Part 11 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 2
M Informational USFDA Draft Guidance – Implanted Brain-Computer Interface (BCI) Devices for Patients with Paralysis or Amputation – Non-clinical Testing and Clinical Medical Device and FDA Regulations and Standards News 0
D Use of password managers on validated computer systems (21 CFR Part 11) Medical Information Technology, Medical Software and Health Informatics 2
L How to classify this computer-alike IT device EU Medical Device Regulations 2
U CE Marking of Customized Ruggedised Computer Systems Solutions CE Marking (Conformité Européene) / CB Scheme 5
Tagin Can/should SPC be applied to Computer Assembly and Software Imaging? Statistical Analysis Tools, Techniques and SPC 8
Q ISO 9001 Section 7.6 - Computer Software ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 2
L Computer stations unlocked - Office Stations vs Production Stations Supply Chain Security Management Systems 4
howste ASQ is Transitioning to Computer Based Exams ASQ, ANAB, UKAS, IAF, IRCA, Exemplar Global and Related Organizations 13
R Network/Computer Data Migration Sampling GMP Software Quality Assurance 1
D SDS (MSDS) for complex products such as a TV, computer, cars, etc Miscellaneous Environmental Standards and EMS Related Discussions 3
Q The ability of computer software to satisfy the intended application ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 8
Gman2 Control of Documents and (FORMS) on a Computer Network ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 16
H ISO 9001:2008 Clause 7.6 Control of Monitoring and Measurement (Computer Software) ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 6
T Class II Medical Device with Software - Change to Computer 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 4
M Is a computer used in hospitals considered a medical device? ISO 13485:2016 - Medical Device Quality Management Systems 17
Marc Shopping for Computer Spyware After Work and Weekend Discussion Topics 4
Q 21 CFR 820.30 - Automated with Computer Software - Applicable? US Food and Drug Administration (FDA) 5
A Is Computer Helpdesk a Special Process ? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 4
Marc Computer Viruses Are "Rampant" on Medical Devices in Hospitals World News 0
I Confirmation of Computer Software to satisfy Intended Application Misc. Quality Assurance and Business Systems Related Topics 2
N Complying with ISO 9001 7.6 - Customer Provided Computer and Software ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 6
E Gages on Computer Screens General Measurement Device and Calibration Topics 2
Marc Life after the Personal Computer After Work and Weekend Discussion Topics 14
A Suggest Computer Aided Quality Assurance (CAQ) software for Medical Devices Quality Assurance and Compliance Software Tools and Solutions 3

Similar threads

Top Bottom