SBS - The best value in QMS software

How to separate Product Design Review, Verification, and Validation Activities

Helmut Jilling

Auditor / Consultant
#21
I agree this is possible in Hardware but what confuses me is the same processof defining Requirements, verifying them and validating them incase of Software. Would be great if how these can be conducted and recorded.
I think it is the same principle. You do not sit down and just start writing code. You have to plan the product objectives, what functions it should do (design inputs), outline the software (design plan), determine the outputs (design outputs), evaluate whether all the objectives are achievable (design verification), write the code, test the Alpha and Beta software (design validation), and continue to develop and improve (design changes).

If you need more detailed or specific help, you may send me a private email.
 
Elsmar Forum Sponsor
C

CATERAF

#22
That menas the test plans should be very undetailed as you ahve said does the SW do A, B and C and not as how do we get A, B and C.
I think it depends on what you're testing as to whether you need to say how it is tested. Essentially, you would be testing it by using it like a 'user' would use it. If there is a specific way of doing something in the software and that's what you're testing, then the 'how to do it' is important. In saying that though, your tester should be familiar with using the product and is likely to know the 'how' for a lot of the things being tested. It is my understanding is that the tester isn't there to test the way they feel.. they are to test what the developer has designed to ensure their design works.

However.. (see below)..

The problem we are facing is that several tetsters say that they have new ways of testing and that is not highlighted in the test plans.
To test something, it needs to be in the test plans. Bear with me, I'll try and explain below..

What they argue is to have a live test plan which could be changed as the testers change the way they test the product. They also argue that the testers become more experienced with the product as it eveolves. Any thoughts on this ? Would be a great help.
I agree that the testers will become more experienced as they use the product. In fact, I'd hope they do! The more you use it, the more you learn :) However, I don't think that means they can just test as they see fit.. their testing must still fit with the testing requirements stipulated earlier during the planning.

If, as you said in your example, you add more modules and fix more bugs, then you would definitely need to add test cases to test with because you've got more things to test. You want to make sure they're all working. The changes in what to test are not done as a reactive thing though - you don't change the tests when you're up to testing. The test plans are put in place early on to say what to test for and then your testing is simply carrying out those tests to check it works. If you want to add more modules and functions, then you write test cases for these and, when ready for testing, then you use those test cases to test.

I'm not 100%, but I think what you're missing is that you write the test cases earlier.. not during testing.
If there's no test case, you don't test for it... Perhaps your testers are trying to create your test cases during the testing phase? If they are, then you want to look at why they have a need to do that. If the predefined test cases are thorough enough, they shouldn't need to change anything during the testing; they just execute the tests. If things are being missed so they feel they need to change the test cases (plans), then it should be 'back to the drawing board' to write the test cases before you do another round of testing.

So, essentially, you change test plans BEFORE you test, not DURING the testing.

I may have gone far off track and if so, let me know and I'll try and answer your question again :)
 
C

CATERAF

#23
Just as a quick note, I don't mean to imply that the testers are simply testers and that their findings outside of tests cases are not important. The testers often find valuable information and they can then channel this in to create future test plans.. that way they aren't disempowered for their knowledge. However, note that the testing plans don't change during testing. The knowledge is fed back to the developer who then decides if that is something important to test for.. and, if necessary, test cases are then developed for later testing.

For example, I tested some of our software to do some marketing for it and found many bugs in the process. I let the developer know and he said they were important things to test for. The developer then asked me to write some test cases for it. He added them in with the other test cases and now they use it to test the software...
 
C

CBAL08

#24
Thank you for the clarification regarding the Test plans and changes that can be made to them.

Now that this has been cleared I would want to further clarify

If we have Product A version 1.0.0 released in October 2013 ( this has been through the whole product development lifecycle- as per IEC 62304)

The development phase has been ended and now release. After release it will have several maintenance phases

Say

Product A version 1.0.1 ( Bug fixes and released - Jan 2014)
Product A version 1.0.2 ( Bug fixes and released - Mar 2014)
Product A version 1.0.3 ( Bug fixes and released - June 2014)
Product A version 1.0.4 ( Bug fixes and released - August 2014)
Product A version 1.0.5 ( Bug fixes and released - October 2014)

The firm decides to now have a major change say Product A version 2.0.0 and release it in Dec 2014

The test plan that was written in version 1.0.0 were used in testing during all the maintenance phase . The argument is while testing in the maintenace phase the testers become better and learn a lot about the product. Could the test plans be updated while in maintenance phase so that the information can be captured all the way till the next major update?

Would appreciate your calrification in this regards.
 
B

BethP

#25
The test plan that was written in version 1.0.0 were used in testing during all the maintenance phase . The argument is while testing in the maintenace phase the testers become better and learn a lot about the product. Could the test plans be updated while in maintenance phase so that the information can be captured all the way till the next major update?
Yes, in my opinion, the test plans can be updated. However, this should not be done in isolation from the rest of the team. As an application moves into end user testing and use, the end user may and often does utilize the application in ways that the analyst, developer, and tester did not envision. This may result in uncovering bugs or ID'ing future opportunities. Having a method to capture this and then feeding back through to updated requirements, code fixes, and test plans allows the entire team to continually improve and refine the application and testing.

As an exception to this, a test plan update may occur without a requirement change as test processes are improved. The WHAT of the application remains the same, but the test team develops new methods on HOW to test that WHAT. For example, the test team may build an automated script to test a function vs. manually testing that function. As long as the test plan has full coverage of requirements and can demonstrate that the requirements were tested, I think HOW it is performed is the ownership of the test team, just as HOW the code is written is the ownership of the developer. Teams get into conflict when one group steps into another group's domain and start to tell HOW to do the job.
 

Helmut Jilling

Auditor / Consultant
#26
Thank you for the clarification regarding the Test plans and changes that can be made to them.

Now that this has been cleared I would want to further clarify

If we have Product A version 1.0.0 released in October 2013 ( this has been through the whole product development lifecycle- as per IEC 62304)

The development phase has been ended and now release. After release it will have several maintenance phases

Say

Product A version 1.0.1 ( Bug fixes and released - Jan 2014)
Product A version 1.0.2 ( Bug fixes and released - Mar 2014)
Product A version 1.0.3 ( Bug fixes and released - June 2014)
Product A version 1.0.4 ( Bug fixes and released - August 2014)
Product A version 1.0.5 ( Bug fixes and released - October 2014)

The firm decides to now have a major change say Product A version 2.0.0 and release it in Dec 2014

The test plan that was written in version 1.0.0 were used in testing during all the maintenance phase . The argument is while testing in the maintenace phase the testers become better and learn a lot about the product. Could the test plans be updated while in maintenance phase so that the information can be captured all the way till the next major update?

Would appreciate your calrification in this regards.
Test plans, as with any plans, can be and should be improved as more information becomes available. That section in the standard is titled :design and Development" to encourage an evolving and improving plan.
 
C

CATERAF

#27
Could the test plans be updated while in maintenance phase so that the information can be captured all the way till the next major update?

Would appreciate your calrification in this regards.
I hope Helmut Jilling's reply was useful because I would say essentially the same thing.

I think the same thing applies as before; if you want to update your test cases because you know more about the product, that's great! Just make sure you write it down before you actually do the testing :) That is, plan what to test for and then test for it. You can test for whatever you like (i.e., keep adding improvements, bug fixes etc) - as long as it's planned.

Hope that helps :)
 
Thread starter Similar threads Forum Replies Date
W Product Audit - It must be a separate and distinct activity (IATF 16949) IATF 16949 - Automotive Quality Systems Standard 4
Q Separate Quality System? Product line of a parent company spins off 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 1
V ISO 14971: 2007 Risk Management File - Separate file required for each product? ISO 14971 - Medical Device Risk Management 3
D Final Release Responsibilities - Separate responsibility for the release of a product ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 9
L Separate Corrective Action and Preventive Action Procedures ISO 13485:2016 - Medical Device Quality Management Systems 13
C Warehousing two separate components to a finished device while meeting regulations Other US Medical Device Regulations 7
D ISO 13485 - 7.3.6 Design and development verification - Do most folks create a separate SOP? ISO 13485:2016 - Medical Device Quality Management Systems 6
H ISO 13485 - Separate Microbiology Audits ISO 13485:2016 - Medical Device Quality Management Systems 8
M UDI Database and EUDAMED - Will there be two separate databases? EU Medical Device Regulations 4
I Can a document (form) approval record be separate? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 6
J Organization merger. Should we keep two separate ISO 13485 certificates? ISO 13485:2016 - Medical Device Quality Management Systems 6
CPhelan Internal audit - Combine similar nonconformities in one or keep separate? Internal Auditing 6
S "X-MR charts do not separate piece-to-piece repeatability of the process" Statistical Analysis Tools, Techniques and SPC 2
R How far apart can you schedule separate areas or departments in your internal audit? Internal Auditing 4
S Do we need separate stand alone procedure for Suspect Unapproved Parts? AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 0
J Facilities - Placing our assembly group into a separate building ISO 13485:2016 - Medical Device Quality Management Systems 8
C ISO 14001 vs Existing Quality System - How to keep them separate? ISO 14001:2015 Specific Discussions 3
T Should Testing be a separate operation in the Control Plan? FMEA and Control Plans 5
P Two separate 510k submissions for one device from the same company Other US Medical Device Regulations 10
B ISO 9001:2015 8.4 External Providers - If work is transferred to a separate facility ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 5
M Is a separate ISO/TS16949 certificate required for an extension site? IATF 16949 - Automotive Quality Systems Standard 4
T Two Separate Businesses - Only Part of Company Certified ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 6
D Separate Classifications for Medical Device Accessories EU Medical Device Regulations 2
R One-Time parts separate from QMS? Quality Manager and Management Related Issues 9
D Is a separate spreadsheet required for the ASL (Approved Supplier List)? AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 5
R Connection to a generic separate power supply IEC 60601 - Medical Electrical Equipment Safety Standards Series 5
S Regulations for Selling Convenience Kit vs. Selling Separate Parts Other US Medical Device Regulations 2
cscalise Separate Forms or Procedure Attachments - What's more common? Document Control Systems, Procedures, Forms and Templates 2
C Do separate sites need separate documents? Document Control Systems, Procedures, Forms and Templates 7
A Newly Incorporated Business - Separate Scope and/or Separate Cert? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 1
L Should Internal Audit CAPAs be kept separate from "normal" CAPAs? ISO 13485:2016 - Medical Device Quality Management Systems 6
P Separate procedures? ISO 14k System combined with OHSAS 18001 Occupational Health & Safety Management Standards 3
J Device Registration and Listing - Separate for Device and corresponding Instruments? 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 1
C How to handle Outsourcing with Same Site - Separate Cert ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 7
B Two Separate Legal Entities - Manufacturer Vs. Contract Manufacturer 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 20
C What can you do if Quality is treated as a separate function in an organization? Quality Manager and Management Related Issues 24
C Separate Certification - Can I combine documents for ISO14001, OHSAS18001 & AS4801? Miscellaneous Environmental Standards and EMS Related Discussions 3
S IEC 62366 - Embedded Software in separate Usability Engineering File? IEC 62366 - Medical Device Usability Engineering 1
K How do ISO 9001 Requirements apply to 2 sets of specifications to 2 separate vendors? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 3
D Include as an attachment or make a separate document? Advise on Document Numbering Document Control Systems, Procedures, Forms and Templates 1
F Multiple Business Lines (Separate Corporations) in one Facility AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 2
B Separate Clinical Trials in the US and the EU - When one fails - question US Food and Drug Administration (FDA) 3
M Can I be a legal manufacturer without being a separate legal entity? CE Marking (Conformité Européene) / CB Scheme 6
B Hospital Operating Room Regulations - Separate Entrances for Clean and Dirty Supplies Hospitals, Clinics & other Health Care Providers 4
S Separate Medical Device 12V Power Source - What inspection and testing is required? IEC 60601 - Medical Electrical Equipment Safety Standards Series 1
M Does ISO/IEC 17025:2005 require the Laboratory to have separate ISO 9001 Procedures ISO 17025 related Discussions 4
R HVAC Contaminant Test for Two Separate Areas Pharmaceuticals (21 CFR Part 210, 21 CFR Part 211 and related Regulations) 6
R CNC OTS Software Validation Separate from full IQ/OQ/PQ for 21CFR Part 820 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 9
S Accessing data from separate worksheets in Minitab Using Minitab Software 3
Q Corrective Action as a Separate Procedure ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 3

Similar threads

Top Bottom