FDA's expectation for validating OTS software updates

[Mark Meer]

Consider a system that includes a computer (running Windows OS) on which software is installed to run a (low-risk) medical device. As we all know, Microsoft issues Windows patch updates on a regular basis (daily/weekly), and major revisions or new versions approximately bi-annually. The ideal-world expectation that device software is revalidated following any changes to Windows OS quickly becomes unreasonable in most cases.

I would expect that validation efforts should be commensurate with the risk involved. But in a recent inspection, our claim that even in the worst-case of software failure there is no risk was insufficient justification for lack of documented revalidation of software following Windows updates.

So my questions are:

• Are there any FDA documents that explicitly tie scope of validation efforts to risk? I'd like some authoritative ammunition to argue our position.
• How do others handle Windows updates and revalidation of device software? The inspector said we should have a process for revalidating software before users may install Windows updates. How is this possible?

 

[yodon]

Re: FDA's expectation for validating OTS software updates?

• Are there any FDA documents that explicitly tie scope of validation efforts to risk? I'd like some authoritative ammunition to argue our position.

Certainly the guidance on general principles of sw validation talk about a risk-based approach: http://www.fda.gov/MedicalDevices/DeviceRegulationandGuidance/GuidanceDocuments/ucm085281.htm

The guidance, though, focuses on software changes and doesn't really address environmental changes.

• How do others handle Windows updates and revalidation of device software? The inspector said we should have a process for revalidating software before users may install Windows updates. How is this possible?

You could deliver a locked-down computer with the software where you have complete control over the environment. That's obviously impractical for most cases (although a case could be made for it if the software was life sustaining). If you just deliver software and user's load it on their PCs, it's not possible to re-validate before the changes are realized (there's also the problem of not completely knowing the environment - OS versions, platforms, browsers used / versions, etc.). You might consider monitoring when Windows updates are rolled out (which will depend on the OS versions!), do some nominal (automated?) testing to ensure no obvious impact, and hope nothing falls apart. Maybe come up with a risk-based rational to do this on a weekly basis (don't patches typically roll out on Tuesdays?). That would mean your customers could "live" with potentially faulty system for a period of time.

Definitely a tough nut to crack. It sounds like your inspector has a fairly dated mindset about software and environmental control. Overcoming that may be the biggest challenge.

 

[Mark Meer]

Re: FDA's expectation for validating OTS software updates?

You could deliver a locked-down computer with the software where you have complete control over the environment.

Yes, we've considered this. If we were to redo the whole project, I'd insist on a more "controllable" OS platform (like linux, unix...). Windows, as far as I know, has very limited options in terms of ensuring updates cannot happen. We disable automatic updates, but apparently this does not cover notification of major updates (e.g. 8.0->8.1 or likely the anticipated "upgrade to Windows 10 for free!").

That would mean your customers could "live" with potentially faulty system for a period of time.

This is my concern. The best we can do, as far as I can tell, is to get a copy of new OS releases as soon as possible and test. I don't think there's anyway to actually confirm there are no issues before a user might accept a Windows update.

 

[JJ_FDA]

Re: FDA's expectation for validating OTS software updates?

Don't changes to the OS, including installing Windows Update updates, typically require administrator permissions to effect?

It's also worth noting that certain Windows Update patches have been known to cause serious problems with the OS in the past. Are you willing to live with having to re-animate a non-functional system?

Depending on your auditor, switching to some flavour of Linux (and sometimes Unix) may cause you more validation problems as they may not consider the OS as COTS. It's happened to me before :/

 

[Mark Meer]

Re: FDA's expectation for validating OTS software updates?

Depending on your auditor, switching to some flavour of Linux (and sometimes Unix) may cause you more validation problems as they may not consider the OS as COTS.

Interesting. So your auditor did not consider Linux as a COTS? What was the basis? Because it was free? ...or did you highly customize it (e.g. modify the source code)?

 

[JJ_FDA]

Re: FDA's expectation for validating OTS software updates?

Because Linux wasn't well-known or have the credibility of Microsoft's or Apple's products, basically. Adequately validated, I don't see the problem.

 

[MC Eistee]

Even though this thread is more than a year old now, I would like to push it back up again.

Our company is currently thinking about upgrading to Windows 10.

For Windows 7 we found a way to handle patches and updates.

Patches are pushed to the computers of the responsible person for a computerized system two weeks earlier before everyone else gets those patches (as we expect that patches do not change the intended use of a function) and they do some UATs. For Updates (Service Packs) the software validation process is used.

Windows 10 is a bit more difficult:
Updates are applied every month or even more often (!) and it isn't differentiated between patches and functional updates.
Out of my current thinking functional changes to the OS would require a revalidation of the affected computerized systems.

As you cannot split between patches and functional updates you should better take the full update.

Our current solutions to handle Windows 10 are:
- Argument why a computerized system does not depend on the OS. For example web based applications
- Automated testing
- Virtualizing the application in for example Citrix under Windows 7
- Staying with a LTSB Version (Versions of Windows 10 that do not get frequent updates. Also no security patches.)
- Staying with Windows 7

Does anyone has some more ideas on how to handle Windows 10?

 

[yodon]

First off, what is the risk of having the software go south after a patch? If low risk then maybe you take the patches and do a periodic review.

If high risk then maybe the frequent automated testing is a better solution.

 

[MC Eistee]

Thank you for this answer.

For the majority of application the risk is very low. This is now documented. For some critical applications we aim for automated tests.