Location of Verification Test Procedures

O

Oscar Wang

hi, guys. here is a discussion in our company about the location of verification procedure. Current situation is as below:
1. We have a system requirement specification (DHF) as Design Input
2. first version Test cases are compiled in verification procedures (DHF)
3. during the product verification, there are a lot of software defect.
4. defects are recorded and handled in defect management system (Bugzilla).
5. Some of these defects are not mapped to system requirement specification. As you known, it is impossible to enumerate all detail requirements in design input, we just choose critical ones. There are a lot of software bugs which don't map to any design input requirements.
6. In order to verify these bugs, some test case should be revised and others shall be newly created.
Here is the argument point: shall we add every newly created test case into our verification procedure? If the answer is no, what kind of test case and their test record can be written on the defect management system (Bugzilla). how to define these criteria?
:thanx:
 
P

pldey42

Re: help needed for location of verification test procedures

It's not only possible to define all requirements in detail, it's vital. (If the requirements are not specific, how do the engineers know what to make?)

Sometimes requirements specification is an iterative process of trial and error, with requirements specified in increasing detail over time as they are elicited, for example in discussions with the client , as a result of requirements analysis and draft architectural design, or as a result of lab experiments.

Poor requirements management is one common reason for software failure. In telecom, requirements specifications can include thousands of specific requirements.

And yes, all the test cases should accumulate into one or several test specs.

This isn't specific to the medical sector, but good software engineering practice as described, for example, in CMMI, Agile and TickIT.

Hope this helps
Pat
 
O

Oscar Wang

Re: help needed for location of verification test procedures

Thanks for pldey42's comment. I didn't say the requirement is not specific and how to define specific may be variance in different situation. for example, is the following requirement specific enough?
$ The system shall have a function that allow to create a new patient.
$ The patient table shall be able to move vertically from XX cm to XX cm.
In medical device development, we have requirements in system level and requirements in subsystem/component level. I am confusing about which level should be more specific. who can help me clarify this issue? when FDA audit the design control, which level do they care about?
 
Y

yodon

Leader
Super Moderator
Re: help needed for location of verification test procedures

Indeed, it would be impossible to specify every behavior required of software. So you do what you can... and it sounds like you're doing that.

Test failures, though are indicators of something gone wrong so you should take an introspective look and see if improvements can be made. This could be as wide-ranging as (yes) improving your requirements development process, improving design reviews, adding scenario reviews, improving (or implementing) coding standards, instituting peer reviews at the code level, peer programming, etc.

But to address your question, I generally take the tack of separating design change verification from system verification. At the design change verification level, you ensure that the changes really do fix the bug. You can do this through a variety of means; e.g., code inspections, unit testing, and up through (focused) system level tests. All this can be captured (with proper discipline) as verification evidence (possibly even as attachments in Bugzilla).

You can review the formal verification (confirmation that requirements are met) test procedures to see if they can be updated to test either specifically for the condition or more generically. It's not a bad idea, in my mind, to do both - have the formal test specifically exercise the condition that revealed the bug in the first place AND test generically 'around' the condition. As all studies indicate, changing code increases the probability of other failures. So if you have the specific condition tested, a change in the future that re-introduces the original error would be caught. It's a judgment call, though. A good test team should be able to strike the right balance.
 
O

Oscar Wang

Re: help needed for location of verification test procedures

Thanks yodon for your good suggestion. It is also the preferred solution in our internal discussion. But i still have a big concern on the bugzillar issue. because bugzillar is not customerized tool, it is difficult to ensure that the test case and its expected result is properly reviewed and confirmed before the verification starts and often test results are not eleborated properly. Some evidence comments are just stating that test passes. Currently there are a lot of procedure and tool flaws in our team.
 
You must log in or register to reply here.
Thread starter Similar threads Forum Replies Date
J Remote location IATF audit IATF 16949 - Automotive Quality Systems Standard 3
P Manufacturing facility moved to different location ISO 13485:2016 - Medical Device Quality Management Systems 10
B ISO 13485 Certificate Location / Scope ISO 13485:2016 - Medical Device Quality Management Systems 9
G IATF Remote Location audit timing IATF 16949 - Automotive Quality Systems Standard 3
J Not able to repeat measurement at the same location General Measurement Device and Calibration Topics 7
J 7010-M002 Label Location IEC 60601 - Medical Electrical Equipment Safety Standards Series 5
T API 4F & 7K license - Add Mexico location? Oil and Gas Industry Standards and Regulations 5
R GFE Audit - Violation? GFE Location Controls AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 1
D What evidence do I need to supply as a remote location in relation to manufacturing sites? IATF 16949 - Automotive Quality Systems Standard 14
K 10 parts, 4 operators, 3 replications and each part has 3 location to measure. Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 4
M IATF 16949 - Audit of Remote Location/Support Site and IT IATF 16949 - Automotive Quality Systems Standard 4
P Best european location to set up for a virtual medical device manufacturer? EU Medical Device Regulations 4
I Old Time Scatter diagrams for defect type and location- software Quality Tools, Improvement and Analysis 7
D Is there a specific location for PPE such as safety glass holders and glove dispensers should be mounted Occupational Health & Safety Management Standards 10
K Changes in process dispersion, and location hypotheses tests Using Minitab Software 3
J Remote location vs. site extension IATF 16949 - Automotive Quality Systems Standard 3
M Quality checklist for opening a second location Document Control Systems, Procedures, Forms and Templates 5
S Controlled Document Location Document Control Systems, Procedures, Forms and Templates 3
Ron Rompen Location of features identified on Print - Feature out of location Inspection, Prints (Drawings), Testing, Sampling and Related Topics 3
F Location requirements of the PRRC for the EU MDR Other Medical Device Regulations World-Wide 8
N Claiming exemptions (N/A) in AS9100D - what is the methodology and location? AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 9
S Is it okay to store the design files (device drawings) at contract manufacturer location? ISO 13485:2016 - Medical Device Quality Management Systems 6
J IATF 16949 - Company Sold - Remote Location Certification IATF 16949 - Automotive Quality Systems Standard 1
D Is ISO 13485 Location Specific? ISO 13485:2016 - Medical Device Quality Management Systems 1
J What to expect during a Change of Location Audit under AS9100D AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 3
S Process validation when changing location ISO 13485:2016 - Medical Device Quality Management Systems 10
S RSL (Remote Site Location) Laboratory - Does it need to be ISO 17025 IATF 16949 - Automotive Quality Systems Standard 5
G Change Of Location FAIR Requirements AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 8
E Change of Manufacturing Location EU Medical Device Regulations 6
D Certificate Scope Change - Remote Location(s) IATF 16949 - Automotive Quality Systems Standard 4
J Expanding our Company's AS9100D Registration Scope and Location AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 6
Sam Lazzara Preferred Location for Revision History Section in Documents Document Control Systems, Procedures, Forms and Templates 16
D No evidence of certain processes on our scope being audited at our remote location IATF 16949 - Automotive Quality Systems Standard 4
R Recommended Location for a 510(k) RTA Checklist with a 510(k) eCOPY 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 1
Q Labeling/Marking Location for BF Symbol IEC 60601 - Medical Electrical Equipment Safety Standards Series 3
L IATF 16949: Adding a Remote Location between certifications IATF 16949 - Automotive Quality Systems Standard 2
R Location Interpretation on Drawing - Hole to Hole? Inspection, Prints (Drawings), Testing, Sampling and Related Topics 7
D IATF16949 Second Location Certification IATF 16949 - Automotive Quality Systems Standard 2
Proud Liberal Cpk on Position w/ MMC on Location and Size Capability, Accuracy and Stability - Processes, Machines, etc. 3
N Is a new 510K required for change of manufacturing location for an existing device? US Food and Drug Administration (FDA) 1
K Equipment Mobility - Transfer of manufacturing lines to a different location Qualification and Validation (including 21 CFR Part 11) 5
C Gage R&R Measurement Location Methodology Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 1
A Remote Support Location Audit - Engineering prototyping and process evaluation Internal Auditing 2
T Moving a Calibration/Test Laboratory to a new Location General Measurement Device and Calibration Topics 5
L Responsibility for Labeling in Multi-Location Group Document Control Systems, Procedures, Forms and Templates 5
J Setting Up New QMS - New US Location ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 2
JoCam Control of Works Transfer to another location AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 6
B Class II and Class III Medical Device Amendment for Change in Location? Canada Medical Device Regulations 2
M Customer Service - Not at our location ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 4
M Change in Manufacturing, Assembly and Testing Location to a Sub-Contractor 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 6

Similar threads

Top Bottom