Search the Elsmar Cove!
**Search ALL of Elsmar.com** with DuckDuckGo including content not in the forum - Search results with No ads.

Software validation - Off The Shelf Software - Web hosted

#1
My company has several OTS Software we use in our QMS processes. We perform software validation on the majority of the Software systems based on risk and requirements. When validation is performed we document the software version number. When software versions are updated, we re-evaluate to determine if validation is required.

However, most of these OTS software are web-based, meaning they don't have a standard version release periodically with a change notification from the vendor. These web-based software providers update the software monthly with release notes only, no version number, or change notification.

How do others manage software validation for software providers that update software monthly without notifications?
 

yodon

Staff member
Super Moderator
#2
First off, what is the risk of those applications failing due to the "uncontrolled" upgrades? If not 'low' then maybe the application is not appropriate.

If the risk is tolerable, I would suggest you monitor (weekly?) for release notices and, when found, do the evaluation to determine if you're still in a validated state or if actions are required to bring you back into a validated state.

Of course, document everything.
 

MC Eistee

Starting to get Involved
#3
What I experienced that it is something hard to go through all the change notes and figure out whether it does have an impact or not.

If you are lucky the monthly changes are already clustered into different categories, e.g. bug fixes, security updates, functional updates. Bug fixes and security updates are usually not meant to change existing functions so the critical part is to look into the functional updates. Still also the functional updates can be to many to actually look through or to hard to understand.
Then it comes back to basic validation. During validation you should have identified the requirements that are higher than just low risk. So you would need to look into them right after the updates.
 
#4
We used some services like this, and came across similar difficulties. Changes were pushed to us with no ability to hold off or negotiate.

First part of the answer (for us) was to ensure contracts and agreements with those providers were adequate (i.e., see if you can get particular terms and conditions that protect you from the effects of changes, as well as sufficient advance warning of the releases). Depending on their policies, you might be able to make some risk-based ruling (e.g. minor software releases might only need low effort, while major version changes might require some re-testing)

A second part of the answer was to ask the provider to provide us copies of their internal validation documents when they make a new release - you still have to conduct activities to maintain the validated state, but you can lower the risk and effort required by accepting existing documentation where reasonable.

The third part was heavily risk-based activities - take a frank risk assessment of the application of the software, and determine what (if any) features could be modified with negligible risk, and allow those changes to not disrupt the validated state unless some particular problems are identified. We would further define which of the software's interfaces were critical. When changes came out, we would implement additional risk controls on those interfaces to keep outputs valid while our detailed validation documentation caught up, without having to stop using the software. (e.g., if the software automatically adds some marking to one of your documents, you might temporarily have quality staff manually verify that output on documentation until the re-validation is complete).

I hope that your usage is sufficiently low-risk; even with the above methods of keeping the work reasonable, it takes a surprising amount of resources to genuinely maintain validated states on continuously updated software. If the application is very low risk, you would want to justify a low level of effort unless some problem were observed - if the application is quite high risk, you may want consider a more stable service.
 
#6
Isn't the whole point in validating something making sure that stuff is actually what it's meant to?
Of course - I think MC wasn't implying that you wouldn't look at those updates, but that you would apply greater scrutiny and priority to more major functional changes (just as you would with any change impact assessment), since the risk of those affecting your application of the software is basically greater. Or at least that's what I imagine they meant!
 
Top Bottom