Document Management Software : Validation and other requirements

R

redknight07

Hello fellow members of the cove,


In the past few weeks I have been evaluating several vendors for an electronic document management software (EMS) for specifically maintaining quality system in a medical device company making class II devices. Thanks to all the informative posts regarding this topic, I first created a list of what our requirements are which also included additional questions regarding user licenses, customization possibilities, additional costs for hardware/upgrades, validation, customer support hours + charge and so forth and what would be the implementation and training costs in first year of implementing the system and then for the subsequent years. I then asked all of those vendors to reply to the same questionnaire.


One question on which I got the most varied answers was regarding validation and I?m confused as to what is required and not to invest time in something that would be an overkill.


Since the EMS is an off the shelf software manufactured by a third party and it is specifically used for documentation purposes (at present we require one with capabilities for document/change management and the training associated with it) and it is not to be used for production or in any manufacturing capacity, I would assume that the risk associated with it would be low compared to a software that was actually part of the device or used in the manufacturing process?!


On validation, some of the vendors mention that they do the IQ/OQ themselves and provide scripts of PQ to the user which they fill out by themselves. Some have stated that they provide certificates that their software is compliant to the necessary standards required in medical device industry and that we don?t have any additional burden about it. Adding that their software has been used by medical device companies and there hasn?t been an issue with regard to this till date. Any changes/updates/bug fixes would be from the vendor?s side and I?d assume they will do the necessary testing themselves before implementing it and passing it on to their customers. Which is the best way forward?!


Needless to say, I am unsure as to what is the level of documentation that will suffice as far as Validation is concerned based on the responses that I?ve got. If any of you guys do manage your documentation electronically or have any expertise to share regarding this matter then I would greatly appreciate it.

Best Regards,
Aniket
 

Mark Meer

Trusted Information Resource
...provide certificates that their software is compliant to the necessary standards required in medical device industry...

I'd be a bit dubious of this claim. Exactly what "necessary standards" are they referring to?

More to the point, validation of the software is according to your requirements. So unless these "standards" (or some requirements contained therein) are part of requirements you've defined, they don't do much in the way of validating for your purposes.

The level of verification is really up to you. Bottom-line is: at the end of the day, have you gathered enough evidence to be confident that the software meets your requirements?

As you point-out, the risk to product is minimal, but a system failure could still have severe consequences for the company. How severe? You have to decide, and scale the validation efforts accordingly.

I know a "up to you" or "it depends" response is probably not what you're looking for...but that's really the case.
 

Marcelo

Inactive Registered Visitor
Nark already mad it clear that the "validation" that those firms offer means nothing in principle, because they would need to validate your use of the software. In fact, I think that those claims are totally misleading for the customers. What they should say is something like: "Our software can make it easier for you to validate your use of it because it was created following general principles expected by regulated and standards" - this would be true (if they did follow those principles :p)

it is not to be used for production or in any manufacturing capacity, I would assume that the risk associated with it would be low compared to a software that was actually part of the device or used in the manufacturing process?!

Even so, you need to validate it anyway. The FDA already requires this:

(i) Automated processes. When computers or automated data processing systems are used as part of production or the quality system, the manufacturer shall validate computer software for its intended use according to an established protocol. All software changes shall be validated before approval and issuance. These validation activities and results shall be documented.

And we are also changing ISO 13485 to require validation of all software used in the QMS.

The important thing, and this is related to the risk, and the "effort" of validation would be dependent on this risk.

You probably should read AAMI TIR36, Validation of software for regulated processes, which discuss those matters. We are creating an international version of this document (it will be ISO TR 80002-2
Medical device software - Part 2: Validation of software for regulated processes) but the international one will take some time to get finished.
 

Mark Meer

Trusted Information Resource
My suggestion: start with requirements for traceability (time-stamps, revision history), backup & export (either print or PDF) as a requirement, and ensure that they are met.

The reason I suggest this is that these are, in effect, risk control measures.

If you can demonstrate that the system provides reliable backups, and that edits to documents are traceable, this will go a long way in terms of justifying minimal risk of the system, and you can scale the other validation efforts accordingly...
 
K

Kevin_Emmrich

I don't have much to add -- I think everyone above has given you the right answers. You need to validate the software yourself in your environment!

We do doc control software (and corrective action, audit management, training records, ...) and while we can supply a validation guideline, the final validation should be done with your data and your expected way of using it. Now some companies have asked for copies of our validation and we have supplied it --and maybe it was good enough for them, but I think there is too much auditing risk there.
 
R

redknight07

Thank you all for your response on the matter. To further clairify (or rather confuse):

Mark, regarding the compliant certificates, it claimed that the software was developed and validated in compliance to the following standards and guidelines: IEC 62304:2006, IEC TR 80002, CFR Part 11 and FDA General Principles of Software Validation. Its validation was carried out in conformance with a Software Validation Master Plan which would also be made available to us. Most of them also had a matrix with requirements of 21 CFR 11 matched with how they were met by their product.

You suggested to start with a traceability document to ensure that requirements for time-stamps, revision history, backup etc. are met. These seem to be requirements under part 11. As I mentioned above, a few vendors made available a matrix of part 11 requirements and how they were met.


Marcelo made a good point that the vendor should rather assist the user in validating the software for its final use. So when a vendor mentions that they do the IQ/OQ themselves and provide scripts for PQ which needs to be filled out, I would assume that this PQ documentation would be based on how this requirement is met!?


820.70 (i) states that the manufacturer must validate computer software for its intended use according to an established protocol and that the changes shall be validated before approval and issuance. I’m thinking that for an Off The Shelf software these activities would be done by the software vendor instead of their end customers since it’s their product (such as creating a user requirements document and other required documentation as per IEC 62304 based on the level of concern of the software). Our job as an end user would be to make sure that these were indeed performed and documented.



From what I understand, it is an absolute requirement of us to create validation documentation to ensure that our specific needs are met which seems to be heavily centered to meeting part 11 requirements. We would also need a user requirements/intended use document? I don’t think we would have access to any of the codes and we would just be validating the final use of the software.


Phew! Seems to be such a hassle!
 

Marcelo

Inactive Registered Visitor
IEC 62304 is applicable to medical device software, not quality management software. I would be suspicious of anyone selling quality management software saying that they followed IEC 62304.
 

Mark Meer

Trusted Information Resource
Phew! Seems to be such a hassle!

Redknight07: while it's great you are being so diligent, in my opinion you are over-thinking this.

First:
Ensuring you are complying with 21 CFR regulations is a worthwhile effort, but I personally have never (nor have heard of) anyone having validation activities of their document management software scrutinized by the FDA, and this fact might guide you as to how much effort you want to put in.

Providing evidence that the software meets your requirements is good. Fretting about whether it's "good enough for the FDA" probably doesn't add any value. The real question should be "is it good enough for you?". If you are satisfied that you have enough evidence to show the software meets your requirements, stop there.

(any other covers out there who have had their document management software validations scrutinized by the FDA, please weigh in ....though I suspect there aren't many of you...)

And second:
How long is your list of requirements? I can't imagine it'd be so detailed that validation is difficult. It might be as simple as:

  • Meets 21 CFR Part 11 requirements (x, y, z...). - If your software provider already has a matrix, great! If not, it's not to hard to make one yourself.
  • Is platform-independent - Get specifications from the developer, and try it out on the platforms applicable to you.
  • Has a in-built system for automatic backups and recovery. - Check the backups, perform a recovery...done!
  • Has a system of timestamps and traceability that maintains a revision history of all edits, and traces all edits to user accounts and times. - ...does it? how does it work?
  • Meets internal documentation requirements (e.g. identification, legibility, etc.). - Does it?
  • Can export data to PDFs, or print records directly. - Can it?
  • Is simple enough to use and maintain that personnel can use with minimal training. - Do a trial period and document it. Did your staff have any problems?

Can't think of anything else at the moment, but you get the idea. This shouldn't have to be overly onerous. You could probably write up the validation report for the above in 2-3 pages max.
Good luck!
 
R

redknight07

Thank you Mark and Marcelo for your expertise in helping me understand the requirements related to implementing [FONT=&quot]an Electronic Document Management Software System. [/FONT] As suggested by most, I did start with a basic list of requirements and features in the form of a questionnaire for what we were looking for in the system and evaluated several vendors on it. Implementation has been postponed for some time but it was a great experience to get to know about this topic in some depth.

As part of the process, I did prepare a sort of requirements document with input from you guys and the from the industry and just wanted to share it here so that whoever has to go through the process in the future can have at least some idea of the activities required. (Also provided as attachment) I had sent over this to the vendors I evaluated and they were okay with the information it has.

Please feel free to correct if there's something you don't agree on.


[FONT=&quot]
"EQMS: Requirements for Implementation[/FONT]

[FONT=&quot]What the regulation says[/FONT][FONT=&quot]:[/FONT]
The Quality System regulation 21 CFR ?820.70(i) requires that ?when computers or automated data processing systems are used as part of production or the quality system, the [device] manufacturer shall validate computer software for its intended use according to an established protocol.?

In addition to the above validation requirement, computer systems that implement part of a device manufacturer?s production processes or quality system (or that are used to create and maintain records required by any other FDA regulation) are subject to 21 CFR Part 11 - Electronic Records; Electronic Signatures regulation. This regulation establishes additional security, data integrity, and validation requirements when records are created or maintained electronically.

These additional Part 11 requirements should be carefully considered and included in system requirements and software requirements for any automated record `keeping systems. Software validation should demonstrate that all Part 11 requirements have been met.

[FONT=&quot]What a software vendor must do[/FONT][FONT=&quot]:[/FONT]

We are responsible for ensuring that the [FONT=&quot]purchased off-the-shelf[/FONT] (OTS) software has been[FONT=&quot] developed by the software vendor using [/FONT]methodologies that are appropriate and sufficient for our intended use of that OTS software as required by regulatory agencies.

Documentation that the vendor should maintain: system requirements, user requirements specifications with traceability matrix, master validation plan, results of their validation, incident reports, validation certificate. The vendor?s life cycle documentation, such as testing protocols and results, source code, design specification, and requirements specification, can be useful in establishing that the software has been validated.

However, such documentation is frequently not available, or the vendor may refuse to share their proprietary information. Some vendors provide some of this information at an added charge. (Check Validation Requirements in the Vendor matrix). [FONT=&quot]Validation should be a key consideration in deciding from whom it will be purchased.[/FONT]

[FONT=&quot]What we must do[/FONT][FONT=&quot]:[/FONT]

[FONT=&quot]We must validate the software for its intended use prior to implementation. We have flexibility in defining how software validation will be accomplished. [/FONT]The extent of validation evidence depends on our documented intended use of that software and the risk posed by the automated operation. Only those functions of the software which will be used as part of the quality system needs to be validated.




DOCUMENTATION REQUIRED

PROCESS MAPPING:

Documentation of SOP?s for processes likely affected by the change, policies to be implemented regarding responsibilities and implications of misusing e-signatures. The software vendor will also be classified as one of our suppliers, and will need to be evaluated based on whether it is classified as a critical supplier or a non-critical one.

Key steps include:

? understanding our processes
? Identifying where and how the software touches these processes.


User Requirements Specification/URS:

A documented URS should define:
? Intended use of the software (extent to which we are dependent upon the software)

? [FONT=&quot]Meets 21 CFR Part 11 requirements[/FONT]

? Operating environment including any required hardware and software configurations or is it platform independent.

? document requirements for system performance, quality, error handling, startup, shutdown, security, etc. such as ;
? Has an in-built system for automatic backups and recovery.
? Has a system of timestamps and traceability that maintains a revision history of all edits, and traces all edits to user accounts and times.
? Meets internal documentation requirements (e.g. identification, legibility, etc.).
? Can export data to PDFs, or print records directly.
? Ease of use with minimal training

? identify any safety related functions or features, such as sensors, alarms, interlocks, logical processing steps, or command sequences; and

Risk Assessment:
[FONT=&quot]To analyze each of the software touch points to determine what risks each has and what mitigations we either already have in place, or which we need to implement, in order to ensure that the risk associated with each requirement is reduced to an acceptable level. The output of the Risk Assessment will be a detailed FMEA. [/FONT]Validation efforts will depend on the risk posed by the system.


VALIDATION:

Validation must be conducted in accordance with a documented protocol, and the validation results must also be documented. It does not need to be done on every computer accessing the system.

Installation, Operational and Performance Qualification Protocols (IOPQ): Since this is an Off-The-Shelf software, most vendors would have developed these scripts and offer them as part of the package or at an added premium. If we are using the system without making any modifications to the processes or workflows within the software then we can use them as it is. If any modifications are made, then the scripts need to be modified accordingly.


[FONT=&quot]IQ[/FONT][FONT=&quot] will focus on documenting the actual installation of the software, the capabilities of the hardware and the appropriateness of the supporting infrastructure and processes, such as networks and backups.[/FONT]
[FONT=&quot]
[/FONT]
[FONT=&quot]OQ[/FONT] will focus on documenting the tests and expected results that will be used to verify that the system is working as you intended it to work, and that any mitigations that were put in place to reduce an identified risk are also working and effective.



[FONT=&quot]PQ[/FONT] is typically performed over a period of time, once the software is being used in production and will focus on documenting, that your system performs consistently as intended over a period of time.
Test cases should be documented that will exercise the system to challenge its performance against the pre-determined acceptance criteria, especially for its most critical parameters which we will be depending on, ex. revision, approvals, electronic signatures, change management, requirements for traceability (time-stamps, revision history), backup & export (either print or PDF), startup, shutdown, system security and stress conditions applicable to the intended use of the equipment.

The test cases should be executed and the results should be recorded and evaluated to determine whether the results support a conclusion that the software is validated for its intended use.

When software is upgraded or any changes are made to the software, we should consider how those changes may impact the ?used portions? of the software, if any additional risks are introduced and must reconfirm the validation of those portions of the software that are used.

SUMMARY REPORT and TRACEABILITY MATRIX:

To summarize, we retain the ultimate responsibility for ensuring that the software is validated according to a written procedure for the particular intended use and that it performs as intended in the chosen application. If we can demonstrate that the software system meets our set of requirements, and if we have sufficient documentation in terms of justifying minimal risk of the system, then we can scale the validation efforts accordingly in order to implement the system.


Traceability Matrix may be created [FONT=&quot]to link all of the other documents together and will be the means by which we can quickly find all documentation related to each of the critical process that we have validated."[/FONT]


Hope this helps.


Best Regards,
Aniket
 

Attachments

  • EQMS Requirements for Implementation.docx
    18.8 KB · Views: 453
Top Bottom