Location of Verification Test Procedures

#1
hi, guys. here is a discussion in our company about the location of verification procedure. Current situation is as below:
1. We have a system requirement specification (DHF) as Design Input
2. first version Test cases are compiled in verification procedures (DHF)
3. during the product verification, there are a lot of software defect.
4. defects are recorded and handled in defect management system (Bugzilla).
5. Some of these defects are not mapped to system requirement specification. As you known, it is impossible to enumerate all detail requirements in design input, we just choose critical ones. There are a lot of software bugs which don't map to any design input requirements.
6. In order to verify these bugs, some test case should be revised and others shall be newly created.
Here is the argument point: shall we add every newly created test case into our verification procedure? If the answer is no, what kind of test case and their test record can be written on the defect management system (Bugzilla). how to define these criteria?
:thanx:
 
Elsmar Forum Sponsor
P

pldey42

#2
Re: help needed for location of verification test procedures

It's not only possible to define all requirements in detail, it's vital. (If the requirements are not specific, how do the engineers know what to make?)

Sometimes requirements specification is an iterative process of trial and error, with requirements specified in increasing detail over time as they are elicited, for example in discussions with the client , as a result of requirements analysis and draft architectural design, or as a result of lab experiments.

Poor requirements management is one common reason for software failure. In telecom, requirements specifications can include thousands of specific requirements.

And yes, all the test cases should accumulate into one or several test specs.

This isn't specific to the medical sector, but good software engineering practice as described, for example, in CMMI, Agile and TickIT.

Hope this helps
Pat
 
#3
Re: help needed for location of verification test procedures

Thanks for pldey42's comment. I didn't say the requirement is not specific and how to define specific may be variance in different situation. for example, is the following requirement specific enough?
$ The system shall have a function that allow to create a new patient.
$ The patient table shall be able to move vertically from XX cm to XX cm.
In medical device development, we have requirements in system level and requirements in subsystem/component level. I am confusing about which level should be more specific. who can help me clarify this issue? when FDA audit the design control, which level do they care about?
 

yodon

Staff member
Super Moderator
#4
Re: help needed for location of verification test procedures

Indeed, it would be impossible to specify every behavior required of software. So you do what you can... and it sounds like you're doing that.

Test failures, though are indicators of something gone wrong so you should take an introspective look and see if improvements can be made. This could be as wide-ranging as (yes) improving your requirements development process, improving design reviews, adding scenario reviews, improving (or implementing) coding standards, instituting peer reviews at the code level, peer programming, etc.

But to address your question, I generally take the tack of separating design change verification from system verification. At the design change verification level, you ensure that the changes really do fix the bug. You can do this through a variety of means; e.g., code inspections, unit testing, and up through (focused) system level tests. All this can be captured (with proper discipline) as verification evidence (possibly even as attachments in Bugzilla).

You can review the formal verification (confirmation that requirements are met) test procedures to see if they can be updated to test either specifically for the condition or more generically. It's not a bad idea, in my mind, to do both - have the formal test specifically exercise the condition that revealed the bug in the first place AND test generically 'around' the condition. As all studies indicate, changing code increases the probability of other failures. So if you have the specific condition tested, a change in the future that re-introduces the original error would be caught. It's a judgment call, though. A good test team should be able to strike the right balance.
 
#5
Re: help needed for location of verification test procedures

Thanks yodon for your good suggestion. It is also the preferred solution in our internal discussion. But i still have a big concern on the bugzillar issue. because bugzillar is not customerized tool, it is difficult to ensure that the test case and its expected result is properly reviewed and confirmed before the verification starts and often test results are not eleborated properly. Some evidence comments are just stating that test passes. Currently there are a lot of procedure and tool flaws in our team.
 
Thread starter Similar threads Forum Replies Date
P Best european location to set up for a virtual medical device manufacturer? EU Medical Device Regulations 4
I Old Time Scatter diagrams for defect type and location- software Quality Tools, Improvement and Analysis 3
D Is there a specific location for PPE such as safety glass holders and glove dispensers should be mounted Occupational Health & Safety Management Standards 10
K Changes in process dispersion, and location hypotheses tests Using Minitab Software 3
J Remote location vs. site extension IATF 16949 - Automotive Quality Systems Standard 3
M Quality checklist for opening a second location Document Control Systems, Procedures, Forms and Templates 5
S Controlled Document Location Document Control Systems, Procedures, Forms and Templates 3
Ron Rompen Location of features identified on Print - Feature out of location Inspection, Prints (Drawings), Testing, Sampling and Related Topics 3
F Location requirements of the PRRC for the EU MDR Other Medical Device Regulations World-Wide 3
N Claiming exemptions (N/A) in AS9100D - what is the methodology and location? AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 9
S Is it okay to store the design files (device drawings) at contract manufacturer location? ISO 13485:2016 - Medical Device Quality Management Systems 6
J IATF 16949 - Company Sold - Remote Location Certification IATF 16949 - Automotive Quality Systems Standard 1
D Is ISO 13485 Location Specific? ISO 13485:2016 - Medical Device Quality Management Systems 1
J What to expect during a Change of Location Audit under AS9100D AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 3
S Process validation when changing location ISO 13485:2016 - Medical Device Quality Management Systems 10
S RSL (Remote Site Location) Laboratory - Does it need to be ISO 17025 IATF 16949 - Automotive Quality Systems Standard 5
G Change Of Location FAIR Requirements AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 8
E Change of Manufacturing Location EU Medical Device Regulations 6
D Certificate Scope Change - Remote Location(s) IATF 16949 - Automotive Quality Systems Standard 4
J Expanding our Company's AS9100D Registration Scope and Location AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 6
Sam Lazzara Preferred Location for Revision History Section in Documents Document Control Systems, Procedures, Forms and Templates 16
D No evidence of certain processes on our scope being audited at our remote location IATF 16949 - Automotive Quality Systems Standard 4
R Recommended Location for a 510(k) RTA Checklist with a 510(k) eCOPY 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 1
Q Labeling/Marking Location for BF Symbol IEC 60601 - Medical Electrical Equipment Safety Standards Series 2
L IATF 16949: Adding a Remote Location between certifications IATF 16949 - Automotive Quality Systems Standard 2
R Location Interpretation on Drawing - Hole to Hole? Inspection, Prints (Drawings), Testing, Sampling and Related Topics 7
D IATF16949 Second Location Certification IATF 16949 - Automotive Quality Systems Standard 2
Proud Liberal Cpk on Position w/ MMC on Location and Size Capability, Accuracy and Stability - Processes, Machines, etc. 3
N Is a new 510K required for change of manufacturing location for an existing device? US Food and Drug Administration (FDA) 1
K Equipment Mobility - Transfer of manufacturing lines to a different location Qualification and Validation (including 21 CFR Part 11) 5
C Gage R&R Measurement Location Methodology Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 1
A Remote Support Location Audit - Engineering prototyping and process evaluation Internal Auditing 2
T Moving a Calibration/Test Laboratory to a new Location General Measurement Device and Calibration Topics 5
L Responsibility for Labeling in Multi-Location Group Document Control Systems, Procedures, Forms and Templates 5
J Setting Up New QMS - New US Location ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 2
JoCam Control of Works Transfer to another location AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 6
B Class II and Class III Medical Device Amendment for Change in Location? Canada Medical Device Regulations 2
M Customer Service - Not at our location ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 4
M Change in Manufacturing, Assembly and Testing Location to a Sub-Contractor 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 6
S Customer or Support - Remote Location IATF 16949 - Automotive Quality Systems Standard 14
R Repeatability in MSA - Is repeatability a measure of Location Error Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 2
P Medical Device Label Location Requirements Other US Medical Device Regulations 6
G Manufacture of Drugs and Medical Devices in one location Manufacturing and Related Processes 7
P Measuring Feature Location from a Theoretical Centerline Inspection, Prints (Drawings), Testing, Sampling and Related Topics 4
B Need help in adding second location to FDA Registration Other US Medical Device Regulations 4
T Using a AS9100 QMS for a New ISO9001:2008 Location ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 4
0 Gage R&R Standard for Multi Location Fixtures Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 5
D AS9100 Shop Moving to a New Location AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 5
L Webcam with Macro Capability for Skype between our facility and a satellite location Inspection, Prints (Drawings), Testing, Sampling and Related Topics 2
R Is ISO 13485 required at each different location? ISO 13485:2016 - Medical Device Quality Management Systems 13

Similar threads

Top Bottom