Seeking reference guides/ documentation /tips on verification best practices

#1
Hi Everyone,

Having recently moved over to the design verification space, but in the same medical device company, I am interested in finding some reference documentation that I could read up on to help guide me through verification best practices. Things like test method development, planning, data analysis, reporting, change control/regression analysis, are what I would be looking for. Appreciate the help.
 
Elsmar Forum Sponsor

Tidge

Trusted Information Resource
#2
If you can look past the title and specific subject matter, William Pressman's Software Engineering: A Practitioner's Approach (many editions exist, I like the 5th) has many many chapters dedicated to diverse elements of planning, design, testing etc. that are applicable to all projects, not just software. There is a chapter on testing that has fundamentals that are applicable to more than just software, although many elements of what he describes do have a relatively unique application (at least of terminology) to software.

One of the things I like best about the Pressman book is that it is so complete, in so many different dimensions. For example, his chapters on projects should be required reading for anyone who has to evaluate, plan, or execute projects (not just software projects). A grasp of the material he presents will allow the student to not just recognize a project that is destined for trouble, but can be used to recover from disasters in the making.
 

yodon

Leader
Super Moderator
#3
One of the biggest issues I see is collection of objective evidence. I see a lot of "did the system do this" ... "yep" (with no evidence to support the conclusion). We develop protocols to drive collection of objective evidence. On the flip side of this is collecting too much data. I see this especially in software testing where a screenshot is taken of every screen transition.

Another issue I see is testing the implementation as opposed to the requirements. The implementation can change and still meet the requirement but there is now a discrepancy with the protocol.

If not obvious, you'll need to be able to show traceability between tests and requirements.

Data integrity is always necessary. Get to know ALCOA. This also addresses ensuring units are captured, precision is correct, etc.

Somewhat along those same lines is ensuring that equipment used is qualified, identified / identifiable, capable (precision), and in calibration if required. All this should be documented as part of the test data.

We incorporate a reviewer in our process that looks at the evidence to determine if they can come to the same conclusion made by the tester, that the results have data integrity, and all the i's are dotted and t's crossed.
 

Tidge

Trusted Information Resource
#4
We incorporate a reviewer in our process that looks at the evidence to determine if they can come to the same conclusion made by the tester, that the results have data integrity, and all the i's are dotted and t's crossed.
This is an item of MAJOR importance for me. I inherited a group (people, practices, procedures) that nominally had an "independent review" of a tester's assessment but practically offered no assessment of the actual results beyond "did they initial the test step correctly?" I'd like to report that the useless/incorrect attitude has be re-aligned, but I am still encountering this problem. For young employees, after the explanation from first principles I ask them if they want recognition and promotion for having recognizable levels of expertise or are the hoping to advance because they are good at "pencil-whipping". For older employees I am more blunt: You can choose to stick with the old ways, forgo advancement and get treated to the exact same monologue from me every time, or you can generate objective evidence that you really are smarter than the rest of us by doing the proper assessments.
 
#5
Thanks Tidge and Yodon for the helpful responses - Appreciate you taking the time to cover the key aspects of good verification practices. I can certainly appreciate the need to ensure data has been gathered through appropriate test method as well as objectively reviewed by independent reviewers. We have also set up this process and it seems to work well, but naturally not without challenges such as program pressures/expectations and product and test complexity leading to steep learning curve for the independent reviewers. This coupled with the "pencil Whipping" rightfully pointed out by Tidge makes for a strained process. I will take a look at "Software Engineering: A Practitioner's Approach" - if there are any other references I'd love to hear them.
 

Hi_Its_Matt

Involved In Discussions
#6
It may be more high level than you are looking for, but check out "Thinking Ahead to System Verification and System Validation" by Louis S. Wheatcraft of Requirements Experts. It provides a good overview of principles, and how to start thinking about verification/validation during the project planning and requirements definition stages.

I will say the terms and definitions Lou uses differ from those I see commonly used, but the guidance is still good, and you should be able to link the terms he uses to ones your organization uses.

(Ex: the companies I have worked for haven't used the terms "requirements verification" or "requirements validation" and instead just talk generally about "requirements review." And I think most med dev companies would use "Design Verification / Validation" where he uses "System Verification / Validation.")
 
Thread starter Similar threads Forum Replies Date
B Seeking training and reference material for ISO 9001 aimed at IT Service Organization Training - Internal, External, Online and Distance Learning 3
C Seeking ISO 9001 and ANSI Z10 crosswalk (cross reference matrix) ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 13
M TS vis-a-vis QS - Seeking TS-16949 to QS-9000 cross reference matrix IATF 16949 - Automotive Quality Systems Standard 6
D AIAG's PPAP Reference Material - Seeking an electronics copy such as a .pdf file APQP and PPAP 4
C AS9100 in Plain English vs. FAR145, ECAR145 - Seeking Cross Reference Matrix AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 20
G Seeking a good reference on gauge R&R studies Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 4
J New QMS Auditor - seeking opportunities to gain audit experience Career and Occupation Discussions 3
M Seeking mentor/consultant for orthopaedic soft goods regulations Job Openings, Consulting and Employment Opportunities 0
M Seeking Consultant for Orthopedic Start Up company Other Medical Device and Orthopedic Related Topics 1
H SOP Template for seeking regulatory clearance - MDSAP Document Control Systems, Procedures, Forms and Templates 6
W Seeking Guidance Verification Test Strategy for Class B Medical Devices IEC 62304 - Medical Device Software Life Cycle Processes 1
M Seeking advice regarding use of off-the-shelf (OTS) batteries Other Medical Device and Orthopedic Related Topics 4
A ISMS - Seeking VAPT Consultant Food Safety - ISO 22000, HACCP (21 CFR 120) 1
M Medical Device Directive - Seeking common nonconformance write up scenarios CE Marking (Conformité Européene) / CB Scheme 2
silentmonkey Seeking efficient method to manage install base data Manufacturing and Related Processes 0
K Seeking ISO 13485 Registrar Recommendations Registrars and Notified Bodies 15
R Tapping problems - 3/8" hot roll pickled & oiled steel - seeking expertise Manufacturing and Related Processes 15
S Responsibilities between government owner and private subcontractor of a power plant seeking ISO 9001 certification ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 9
B Seeking for an example or a template of a Business Plan Service Industry Specific Topics 3
B Health Canada Recall Definition - Seeking Clarification Canada Medical Device Regulations 5
W Seeking Returned Merchandise Procedure Customer Complaints 1
QMMike New Trend of not seeking Official ISO 9001 Registration ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 43
R Clinical Evaluation according to MEDDEV 2.7.1 (Rev 4) - seeking template ISO 13485:2016 - Medical Device Quality Management Systems 6
T ISO 9001:2015 Training for Management Team in NM - Seeking Suggestions and Providers ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 5
D Seeking Corrective Action Process Examples Nonconformance and Corrective Action 3
M AS9102 Training - Seeking third party agency that provides online AS9102 training AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 1
F Seeking Internship/Co-op Opportunities in Regulatory Affairs/QA/Clinical Domain (USA) Career and Occupation Discussions 4
L Seeking help in Basic Maintenance Knowledge Manufacturing and Related Processes 4
M Seeking opinion for ISO14001 team activity ISO 14001:2015 Specific Discussions 3
smryan Seeking good 2D barcode add-in for Excel Excel .xls Spreadsheet Templates and Tools 6
S Seeking Feedback on ASQ Guide to Failure Mode and Effect Analysis FMEA and Control Plans 2
S Seeking examples of Nonconforming Materials Nonconformance and Corrective Action 5
S Seeking Quality Training for Quality Technicians Training - Internal, External, Online and Distance Learning 1
P Seeking Information on ISO 15504 - Information Technology Process Assessment Other ISO and International Standards and European Regulations 3
K Seeking NDT Training and Certifications Inspection, Prints (Drawings), Testing, Sampling and Related Topics 3
Chennaiite Cause Analysis for Verification/Validation Failures - Seeking Opinions Problem Solving, Root Cause Fault and Failure Analysis 3
Q Seeking Examples of current ISO/TS 16949 Training Procedures IATF 16949 - Automotive Quality Systems Standard 2
Ron Rompen Seeking Test Lab which can do 1440 Hour Accelerated Weathering Characteristics Misc. Quality Assurance and Business Systems Related Topics 1
S Seeking Ballooned PFMEA and Control Plans for Training FMEA and Control Plans 4
B Seeking help in answering Australia Tax Withholding Declaration Coffee Break and Water Cooler Discussions 1
H ASQ CQE Exam Review Workshop - Seeking Volunteers! September 2013 ASQ, ANAB, UKAS, IAF, IRCA, Exemplar Global and Related Organizations 7
N Seeking information on use of Byotrol Cleaning Chemicals in Pharma Industry Manufacturing and Related Processes 1
M Seeking input for a process-based quality management system training course ISO 13485:2016 - Medical Device Quality Management Systems 1
B Seeking help in writing marketing message Career and Occupation Discussions 12
B Seeking help in writing an audit report that can stand on the court Internal Auditing 6
S AAMI EC57 for ECG Medical Device - Seeking 3rd Party Test Laboratory US Food and Drug Administration (FDA) 5
W Seeking a Freelance Mechanical Inspector - Source Inspection in Bangalore, India AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 1
V Seeking Internal Audit Plan form or template Internal Auditing 5
W X-ray based NDT Test Equip. Mfger. seeking CE Mark - Must X-Ray Source be included? CE Marking (Conformité Européene) / CB Scheme 5
F Quality Awareness Sessions - Seeking Ideas and Presentation Material Quality Manager and Management Related Issues 4

Similar threads

Top Bottom