IS0 13485:2003 - Validation of the Application of Computer Software

G

Gert Sorensen

#51
Re: IS0 13485:2003, Validation of the application of computer software

Hi Wnd54 and welcome to the cove.

In my opinion there is not a lot of difference between a software validation and a process validation in this respect. In validations you encounter bugs, obstacles, unforeseen events etc. What matters is whether you document them properly (raise a deviation), handle them in structured manner , and evaluate them objectively and consider the risk for the patients. Depending on the type of bug you encounter, you may need to redo a lot of testing, but it may also be a minor issue that is easily resolved. A large part of that decision, when it comes to software, comes down to whether you have employed a robust development method with a good design of the software (good meaning an efficient documented process flow, separated modules, followed the Good Coding Standards etc.). If you have done that, then the decision on how to handle any deviations should be fairly easy.
 
Elsmar Forum Sponsor

sagai

Quite Involved in Discussions
#52
Re: IS0 13485:2003, Validation of the application of computer software

1) Does anyone with software experience agree that you can pass a verificaation or validation when one of the tests fail?
I think it is okay to pass a verification phase with software having bugs in it.
FDA has a recommendation to provide "Unresolved anomalies report" for the same purpose. Software release notes is typically the place to list all triaged defect relates to the particular release of the software.

2) When running a V&V for software, one is sure to uncover bugs. Based upon risk, you might choose to address a software bug in a later version. Our QA group captures these bugs on the V&V report. I look at this as passing a test step with an exception. How does one address software bugs when the test case actually passes but a minor bug is found?
For me, when there is a bug, than either it can clearly linked to the test step, or there is a description how to reproduce it.
It is more or less semantics, and we could run into endless debate how extensive the expected test result description could be, but I would not go that far.
However, clearly as part of the not yet released software V&V phase, we can have bugs as part of the test execution, or as part of sanity testing. Nevertheless not all bugs are bugs, could be due to lack of feature specification, not complete test cases, etc.
But I would not worry about it too much, as long as there is a registered way how to reproduce it, and we did triaged it at least.

Cheers!
 

pbojsen

Involved In Discussions
#53
For software validations we did find bugs, put them in the report, and also reported what we were going to do with them. Repair and retest? Workaround the bug? Leave it? There are many options depending upon the criticality and complexity of the software. I don't think there is bug-free software. If there was, companies like Microsoft would have a very hard time selling theirs.

Software validations and process/equipment VV are not the same, and shouldn't be handled the same either. Engineers, unless they are software engineers, like to think they know about software but rarely do, in my experience anyway. You need to get qualified software people in the discussion.
 
W

wnd54

#54
Re: IS0 13485:2003, Validation of the application of computer software

I think it is okay to pass a verification phase with software having bugs in it.
FDA has a recommendation to provide "Unresolved anomalies report" for the same purpose. Software release notes is typically the place to list all triaged defect relates to the particular release of the software.


For me, when there is a bug, than either it can clearly linked to the test step, or there is a description how to reproduce it.
It is more or less semantics, and we could run into endless debate how extensive the expected test result description could be, but I would not go that far.
However, clearly as part of the not yet released software V&V phase, we can have bugs as part of the test execution, or as part of sanity testing. Nevertheless not all bugs are bugs, could be due to lack of feature specification, not complete test cases, etc.
But I would not worry about it too much, as long as there is a registered way how to reproduce it, and we did triaged it at least.

Cheers!
Sagai,

Thank you. Our SW bugs are captured in a database when they are identified during test. I believe documenting bugs on the test is redundant. Maybe a separate bug report is more practical.

Would you pass the whole validation if one step was marked fail?
 
W

wnd54

#55
For software validations we did find bugs, put them in the report, and also reported what we were going to do with them. Repair and retest? Workaround the bug? Leave it? There are many options depending upon the criticality and complexity of the software. I don't think there is bug-free software. If there was, companies like Microsoft would have a very hard time selling theirs.

Software validations and process/equipment VV are not the same, and shouldn't be handled the same either. Engineers, unless they are software engineers, like to think they know about software but rarely do, in my experience anyway. You need to get qualified software people in the discussion.
I think your suggestion of a separate bug report with disposition and resolution is a great alternative to what my company is currently doing.
 

sagai

Quite Involved in Discussions
#56
Re: IS0 13485:2003, Validation of the application of computer software

Well, we really can administer the bugs on the way we found it is most suitable for the organization, however I think we need to be able to answer a simple question when the corresponding traceability for the actions carried out is asked to be identified next to the failed test case/step.
I also would advocate to call them prior triaging as an "incident" or whatever that is different from bug/defect due to the fact that "bug" and "defect" are already qualifying them whereas such qualification can not be encountered without triaging them. I know it is more or less a subtlety, I have found it worthwhile to mention here.

Would you pass the whole validation if one step was marked fail?
I think it is the nature of the software industry and the software itself, that software can go wrong in millions of ways and all we have prior releasing the software is a level of confidence (or a kind of religious belief of such confidence :rolleyes:) that stipulates us to say, well, probably, touch wood, it seems to us we happy with the level of confidence and we (or it would be better of not me, but you guys next to me :rolleyes:) can sign off to release this version.
I do not think there is a software that is defect free and I also very not believe in a software that truly can pass all well planned and executed test cases, because this is the nature of this business we are in.

So, sum it up, yes, whatever number of test steps were marked as failed, I would let it go out as long us these incidents were triaged, the necessary professional contributors including clinic fellows were involved and all of these professionals from engineering, quality control, management and clinic fellows are singing the same song, telling they are happy with the release to be released.
There could be extremities that may influence me otherwise, but normally this is the scenario I would mentione here.
Cheers!
 
#57
Re: IS0 13485:2003, Validation of the application of computer software

Hello to all here on The Cove,

Im a newbie who has made use of your collective wisdom often and I wish to thank you all profusely for your selfless sharings of what has certainly been at times a confusing journey of "Am I interpreting the QSR or ISO 13485:2003 [ and now ISO 13485:2016 ] correctly enough to prevent 483s or Majors in audits".

This particular thread on validating computer software has been enlightening, to put it mildly, and soon, I will be sharing as well about our small medical device manufacturer's journey to more complete compliance with the new ISO 13485 standard.

Again, to the many experts and trusted information sources, I humbly send you my gratitude and will be posting my offerings as well. Im most definitely a novice but feel my thoughts and perspectives may have value to the Forum.

Thank you, Jeff
 
B

BhupinderSinghPawa

#58
Re: IS0 13485:2003, Validation of the application of computer software

I am a quality manager for a class I device manufacturer. I'm having a debate with engineering about Pass/Fail criteria and final V&V reports for product transfer. In my past experience, if therre is one fail on a test report, the whole test fails and the information is fed back into the design process to be corrected. My engineering peers think that it is okay to show that a test case fails but still pass the verification or validation.

1) Does anyone with software experience agree that you can pass a verificaation or validation when one of the tests fail?

2) When running a V&V for software, one is sure to uncover bugs. Based upon risk, you might choose to address a software bug in a later version. Our QA group captures these bugs on the V&V report. I look at this as passing a test step with an exception. How does one address software bugs when the test case actually passes but a minor bug is found?

Any guidance or help would be greatly appreciated
With the documentation map as below -
V&V Test Plan -> V&V Test Protocol -> V&V Test Result / Execution Log -> V&V Test Report

The V&V Test Result / Execution Log - should state that the 'Test Case' has failed and provide a Defect Identification against the Test Case. The defect will have an associated severity level (impact) - catastrophic / major / minor / trivial.

The V&V Test Report - should evaluate if the 'Release Criteria' is met. The 'Release Criteria' could be acceptable level of defects by severity level and evaluation of residual defects to ensure that they do not contribute to an unacceptable risk. The release criteria can be captured on Test Plan.

The plan / protocol / results and reports are reviewed and approved by required stakeholders. This should provide the required traceability and control.

With this with residual 'minor or trivial' defects and residual risk evaluation, the software can be released.

The defect is then tracked with defect management process and consequent software releases.
 
Thread starter Similar threads Forum Replies Date
S Inspection Parts as saleable goods - IS0 13485 OEM medical device manufacturer ISO 13485:2016 - Medical Device Quality Management Systems 4
Q Gap Analysis - Bringing a smaller subsidiary company up to IS0 9001:2008 standard ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 3
Q Should distributors be also certified to IS0 9001? IATF 16949 - Automotive Quality Systems Standard 16
M IS0 9001 and Health & Safety plus Food Safety Intranet Forms and Guides ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 2
F Franchises and IS0 9001 ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 3
G ISO 17025 or IS0 9001 Training procedure - Needed ISO 17025 related Discussions 7
K External calibration laboratories - IS0 17025 Equivalents? ISO 17025 related Discussions 19
T Accreditation of In-House Labs to IS0-17025 ISO 17025 related Discussions 4
C Have QS & IS0:1994 need IS0:2000? QS-9000 - American Automotive Manufacturers Standard 1
Ed Panek ISO 13485:2016 Section 5.5.3 ISO 13485:2016 - Medical Device Quality Management Systems 3
ebrahim QMS as per ISO 13485, Clause 4.2 Requirements for regulatory purposes for Medical Devices Authorized Representatives. ISO 13485:2016 - Medical Device Quality Management Systems 3
D ISO 13485 scope (implantable) - Polymers for dental application EU Medical Device Regulations 9
N ISO 13485 7.3.9 Change control in medical device software ISO 13485:2016 - Medical Device Quality Management Systems 6
A ISO 13485 procedure change and reflect to legacy manufacture items ISO 13485:2016 - Medical Device Quality Management Systems 2
D ISO 13485 & CE Certification for Surgical Gloves CE Marking (Conformité Européene) / CB Scheme 0
S Inventory Listing and ISO 13485:2016 ISO 13485:2016 - Medical Device Quality Management Systems 3
M ISO 13485:2016 Certification Scope ISO 13485:2016 - Medical Device Quality Management Systems 2
D Reports under change management | ISO 13485:2016 & ISO 9001:2015 ISO 13485:2016 - Medical Device Quality Management Systems 3
0 To which part of 13485 does this refer? ISO 13485:2016 - Medical Device Quality Management Systems 3
M Scope for ISO 13485 Certification of a Translation Service Provider ISO 13485:2016 - Medical Device Quality Management Systems 17
Q ISO 13485 7.5.6 Validation - Off the shelf Software ISO 13485:2016 - Medical Device Quality Management Systems 3
A ISO 13485 Certification for Resin Manufacturer ISO 13485:2016 - Medical Device Quality Management Systems 4
A ISO 13485 Sterilization Clause Applicability ISO 13485:2016 - Medical Device Quality Management Systems 7
K ISO 13485 and compliance of electronic signature ISO 13485:2016 - Medical Device Quality Management Systems 5
T ISO 13485 - Assembly instructions written vs. online ISO 13485:2016 - Medical Device Quality Management Systems 5
M ISO 13485:2016 internal audit checklist Medical Device and FDA Regulations and Standards News 5
N 93/42/EEC certification without ISO 13485 EU Medical Device Regulations 3
M How Specific in an ISO 13485:2016 Scope for a Contract Manufacturer ISO 13485:2016 - Medical Device Quality Management Systems 9
A ISO 13485 for Class 1 Medical Device ISO 13485:2016 - Medical Device Quality Management Systems 7
0 ISO 13485:2016 Chapter 8 Integration of the subsections ISO 13485:2016 - Medical Device Quality Management Systems 3
M Change in Constitution / Ownership of firm -------ISO 13485 ISO 13485:2016 - Medical Device Quality Management Systems 1
J ODM not 13485-certified ISO 13485:2016 - Medical Device Quality Management Systems 2
Louddogsbark When your 13485:2016 certificate has been pulled ISO 13485:2016 - Medical Device Quality Management Systems 2
E ISO 13485 QMS certification as a Supplier ISO 13485:2016 - Medical Device Quality Management Systems 8
T ISO 13485:2016 Clauses related to process matrix ISO 13485:2016 - Medical Device Quality Management Systems 3
J Can signed agreements over-ride review of every "contract" under ISO 13485:2016? ISO 13485:2016 - Medical Device Quality Management Systems 2
J Implementing an ISO 13485 QMS Software ISO 13485:2016 - Medical Device Quality Management Systems 6
Q EN ISO 13485:2016/AC:2018 - AC:2018 being stated in the applicable harmonized standard listing Other ISO and International Standards and European Regulations 1
J Leveraging another company's ISO 13485:2016 ISO 13485:2016 - Medical Device Quality Management Systems 5
J New Job Position - Achieving ISO 13485 Certification ISO 13485:2016 - Medical Device Quality Management Systems 5
A Scope of ISO 13485 certificate ISO 13485:2016 - Medical Device Quality Management Systems 1
A ASL requirement when the supplier is certified for ISO 13485 ISO 13485:2016 - Medical Device Quality Management Systems 6
M ISO 13485-2016 online certification ISO 13485:2016 - Medical Device Quality Management Systems 3
S Thoughts on managing ISO 9001, 13485, IATF 16949 and 17025 ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 33
S Supplier Management ISO 13485: 2016- Which supplier needs to fill in a self assessment form? ISO 13485:2016 - Medical Device Quality Management Systems 6
J Possible to get ISO 13485 certified with only OEM Product? ISO 13485:2016 - Medical Device Quality Management Systems 4
D Definition of equipment for ISO 13485:2016 ISO 13485:2016 - Medical Device Quality Management Systems 1
M ISO 13485:2016 Complaint Definition Clarity Customer Complaints 2
D Rules for Paper Forms outside of an eQMS - 3 Questions (ISO 13485) Document Control Systems, Procedures, Forms and Templates 9
S Qualification question - ISO 13485 - Setting up a small lab Reliability Analysis - Predictions, Testing and Standards 2

Similar threads

Top Bottom