What is the remedy if Unit level testing is not documented?

S

sl344

#1
Dear all,

My company has been developing piece of firmware that was initially classified as Class A and then upgraded to Class B based on risk come out later on. Since class A software does not require Unit level testing, the test results were not documented. The project has been going on for over 2 years and now is fully integrated. Per IEC62304 class B requirements, unit level testing has to be performed and documented. What are my options now? It is almost not possible to go back to all the units to perform testing...thanks!
 
Elsmar Forum Sponsor

glork98

Involved In Discussions
#2
You need to talk with your own regulatory staff to see what they'll support. If you can rationalize that your other quality practices are satisfactory then you've got a chance. I've seen exhaustive code reviews justified in place of thorough unit tests. Not in place of all unit test, however. There still need to be tests of critical functional operations.

Your best approach is to go ahead and re-do the unit tests. I'm surprised that you can't replicate there. Perhaps the test code was lost or invalidated by changes. Yes, lots of time and money but that's the terrain. I figure that writing the product code is near 10% of the work in developing a class B or C (II or III) medical device.
 

sagai

Quite Involved in Discussions
#3
I figure that writing the product code is near 10% of the work in developing a class B or C (II or III) medical device.
Actually writing product code is nothing else than writing specification for compiler or for the microprocessor and as such there is no distinction from writing user/system/software specification.

Regardless developers like to mystify coding. :cool:

br
Sz.
 

glork98

Involved In Discussions
#4
Actually writing product code is nothing else than writing specification for compiler or for the microprocessor and as such there is no distinction from writing user/system/software specification.
That's one way to look at it. Unfortunately, that's the 10% that makes the product actually work. Outside groups (E.g., upper management) wonder why it's taking so much time to deliver a commercial medical device when the prototype was done months ago.

I'm sure that's what the OP is facing.

And software is fairly mystical. You can't see it or touch it. You only see the results. Magical is some ways and critical in application. Consequently we need to apply a very large proportion of work to show that it was done intentionally and correctly.
 
S

sl344

#5
Thanks for all the replies!

Yes, I'm thinking about taking Glork98's suggestion. I will pick a subset of units which I think they are 'critical' functions and re-do the unit tests for these units only. Other units, I will provide evidence to show that code review covers.
 

Peter Selvey

Staff member
Super Moderator
#6
If the classification was changed from A to B then it sounds like a borderline situation. To formalize your approach above (i.e. test some units, not other), it might be possible to individually classify the software units as Class A or B depending on criticality (as allowed in the standard), and then test only the Class B units.

Also note the software classification system in IEC 62304 has a technical error, in that it does not account for probability factors external to the device. For example, most people expect a home use digital thermometer to be Class B. But it is possible that the failure in the thermometer's software can lead to death (e.g. due to delay in seeking profession help associated with a false low reading when the fever is high). Under IEC 62304 we should classify the thermometer as Class C. The reason why this is overkill is that even if the software fails there are external probability factors that make the probability of death very low. This compares to, say an infant incubator or a surgical laser system, where the failure of software has a high probability of a serious outcome.

I would recommend to use the EU MDD Classification as a guide:
Class I device = Class A software system
Class IIa device = Class B software system
Class IIb/III device = Class C software system
 
S

sl344

#7
Thanks Peter for the good explanation. The device(include firmware) is claimed to be an accessory for a classIII medical device, which is a gel that is implanted into human body. Since the Gel has gone through the PMA submission as class III medical device, we are planning to submit a PMA supplement for the device. Based on the software analysis, the highest level of severity is 'Minor injury', therefore the software is upgraded to Class B. It has been developing towards Class A until recently because the highest severity level was 'no injury' in prior hazard analysis.
 
S

sl344

#8
Hi Peter,

Is the below from a formal guidance? Thanks!

"I would recommend to use the EU MDD Classification as a guide:
Class I device = Class A software system
Class IIa device = Class B software system
Class IIb/III device = Class C software system
"
 

Peter Selvey

Staff member
Super Moderator
#9
Not "formal" in the sense that it has been known to be recognized by regulators, auditors and those reviewing technical files.

What I can be confident about is that very few would consider a home use thermometer as Class C software, even though strictly speaking it should. That example exposes the weakness in the classification system in IEC 62304.

It should also be noted that the FDA guide for pre-market submissions on software uses the term "direct" in determining both "major" and "moderate" levels of concern.

The word "direct" is not defined, but it seems clear the intention is to mean failure of the software has a high probability of causing harm.

The difficulty here for both the FDA and IEC is they want avoid fixing a specific probability of each type of harm. It is a common problem in standard which try to set different levels of controls depending on risk.

Ultimately it depends on your confidence in "black box testing" associated with Class A software, from which you can estimate the probability of "adverse failures" (i.e. not annoyance crashes etc), which in turn will feed into the overall probability of harm including other factors external to the software.

For example (note: just examples for illustration!!):

- in a simple system black box (Class A) testing can be highly effective, leading to an expected probability of <0.001 per use of a adverse failure. However, an adverse failure has a probablity of 0.5 of resulting in death. The overall probablity of death is 0.0005 per use, which is not acceptable according to the risk management plan. Class C controls are recommended, or Class B with an additional protection system.

- in a moderately complex system, black box (Class A) testing is assumed to have an adverse failure rate of 0.01 per use; there are external factors that mean the probability of a minor injury with an adverse failure is 0.05 (5%), leading to an overall probability of a minor injury of 0.0005. This is below the acceptable limit in the risk management plan. Therefore, Class A software is accepted.
 
Thread starter Similar threads Forum Replies Date
R HV Test of HF Surgical Unit IEC 60601 - Medical Electrical Equipment Safety Standards Series 1
F UDI - Unit of Use and Class I, single-use devices EU Medical Device Regulations 4
B Unit of Use DI (Device Identifier) - Products using the same device US Food and Drug Administration (FDA) 0
M What to do in SOP Phase of Head unit testing? IATF 16949 - Automotive Quality Systems Standard 0
D Torque fractional unit of measure; cN-m or N-cm? General Measurement Device and Calibration Topics 2
E Unit Dose Packaging Pharmaceuticals (21 CFR Part 210, 21 CFR Part 211 and related Regulations) 0
M Measurement Unit Rounding Nonconformity - Notified Body Audit Registrars and Notified Bodies 6
P HVAC System vs Air Handling Unit (AHU) - Differences Pharmaceuticals (21 CFR Part 210, 21 CFR Part 211 and related Regulations) 1
P Fan filter unit (un-ducted to air supply) vs fan filter unit (ducted to air supply) Pharmaceuticals (21 CFR Part 210, 21 CFR Part 211 and related Regulations) 1
K Production Unit Labeling - 21 CFR 820.184.e 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 5
P Qualification of a Temperature Controlled Unit have to be empty? Qualification and Validation (including 21 CFR Part 11) 5
M For a Type 1 Gage R&R I want to use 1 unit and measure it 30 times Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 5
C Software Unit Acceptance Criteria (5.5.4) IEC 62304 - Medical Device Software Life Cycle Processes 3
T ISO 14001:2015 cover more than 1 company or business unit? ISO 14001:2015 Specific Discussions 13
F Setting up a new sterilization unit in a new facility 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 6
R VSM - Value Stream Mapping (catheter manufacturing unit) Customer Complaints 10
A 5.5.3 - Software Unit Acceptance Criteria (Risk Control Measures) IEC 62304 - Medical Device Software Life Cycle Processes 3
A Unit of Expanded Uncertainty in Percentage General Measurement Device and Calibration Topics 4
S Is 0.1 micron monitoring required for ISO Class 5 Laminar Air Flow unit ? Other ISO and International Standards and European Regulations 4
O Can unit be categorized as EMI class A when it has EMI class B power supply in it? CE Marking (Conformité Européene) / CB Scheme 2
Z Final Validation Unit Requirements - Annex D of IEC 62366 IEC 62366 - Medical Device Usability Engineering 6
T Is IEC 61010 needed for a populated PCB within a unit? CE Marking (Conformité Européene) / CB Scheme 2
G Help with an old Spectrum 1000BX FTIR unit - User Manual wanted Manufacturing and Related Processes 2
D Calibration Cert - Measurement Unit for Nominal Value of a Calibration Resistor General Measurement Device and Calibration Topics 6
R Does dental unit need to meet the requirements of clause 14 (PEMS)? IEC 60601 - Medical Electrical Equipment Safety Standards Series 3
somashekar Use of Set or Pair as Unit of Measure in the Bill of Material (BOM) Manufacturing and Related Processes 5
B Blood Bag Classification - with Disposable Centrifugal Unit for processing blood EU Medical Device Regulations 4
R Does applied part of battery powered TENS unit need to be designed as type BF? IEC 60601 - Medical Electrical Equipment Safety Standards Series 9
F Does the SI acronym stand for Syst?me International d'Unit?s? General Measurement Device and Calibration Topics 1
W What is the applicable standard for Chair Mounted Dental Unit for CE IEC 60601 - Medical Electrical Equipment Safety Standards Series 2
S Calculating MTBF (Mean Time Between Failures) for the whole Production Unit Reliability Analysis - Predictions, Testing and Standards 2
P How to set up HSE Unit? What is the difference btw HSE Plan and HSE Manual? Occupational Health & Safety Management Standards 32
N Dental Treatment Unit (aka Dentist's Chair) requirements EU Medical Device Regulations 7
I IEC 62304:2006 Definitions - Software System, a Software Element and Software Unit IEC 62304 - Medical Device Software Life Cycle Processes 13
S How to Design the Means of Protection for Home Use Unit IEC 60601 - Medical Electrical Equipment Safety Standards Series 12
V SPC for Continuous Steam Sterilization Unit Statistical Analysis Tools, Techniques and SPC 2
M Decontamination Label on Unit Packages per UL (Underwriters Laboratories) Manufacturing and Related Processes 1
E Using Customer Unit as Reference Standard General Measurement Device and Calibration Topics 14
S About Ozone Concentration's Unit "pphm" and "ppcm" Reliability Analysis - Predictions, Testing and Standards 4
B GMP Compliant Flooring for Food Manufacturing unit Food Safety - ISO 22000, HACCP (21 CFR 120) 8
W EN 61000-4-4:2004 Compliance - Electrosurgical Unit Transient Voltage Pulse EU Medical Device Regulations 3
B Minitab Gage R&R using Defect Count per Unit Area Using Minitab Software 1
M Customer Complaint vs. Repair - Unit goes out of specification after the warrenty Customer Complaints 8
S Medical Gas Change Over Unit - Which Standard to use? Other Medical Device Related Standards 4
T Validation of water system in API (Active Pharmaceutical Ingredients) unit Qualification and Validation (including 21 CFR Part 11) 5
J FMEA for PMA Data Application Package - APU (Auxiliary Power Unit) parts Federal Aviation Administration (FAA) Standards and Requirements 3
R Filter cleaning procedure for Clean Room Air Handling Unit example wanted US Food and Drug Administration (FDA) 4
M V&V (Verification and Validation) Medical Device Unit Testing ISO 13485:2016 - Medical Device Quality Management Systems 4
kedarg6500 Unit Conversion Add-in for Excel Excel .xls Spreadsheet Templates and Tools 7
R TS16949 certificate for business unit (one half of the company) IATF 16949 - Automotive Quality Systems Standard 1

Similar threads

Top Bottom