The Cove Business Standards Discussion Forums
Firmware Verification Testing & Sample Size
Please read this thread...
Software update
Firmware Verification Testing & Sample Size
Go Back   The Elsmar Cove Business Systems and Standards Discussion Forums > >
Forum Username

Elsmar Cove Forum Visitor Notice(s)

Wooden Line

Firmware Verification Testing & Sample Size

Monitor the Elsmar Forum
Courtesy Quick Links


Links Elsmar Cove visitors will find useful in the quest for knowledge and support:

Jennifer Kirley's
Conway Business Services


Howard's
International Quality Services


Marcelo Antunes'
SQR Consulting, and
Medical Devices Expert Forum


Bob Doering
Bob Doering's Blogs and,
Correct SPC - Precision Machining


Ajit Basrur
Claritas Consulting, LLC



International Standards Bodies - World Wide Standards Bodies

AIAG - Automotive Industry Action Group

ASQ - American Society for Quality

International Organization for Standardization - ISO Standards and Information

NIST's Engineering Statistics Handbook

IRCA - International Register of Certified Auditors

SAE - Society of Automotive Engineers

Quality Digest

IEST - Institute of Environmental Sciences and Technology


Some Related Topic Tags
firmware, sample size, statistical analysis and studies, verification (general), verification testing
Reply
 
Thread Tools Search this Thread Rate Thread Content Display Modes
  Post Number #1  
Old 13th March 2018, 03:13 PM
v1cph1rth

 
 
Total Posts: 1
Please Help! Firmware Verification Testing & Sample Size

Hi All,

We are developing custom firmware for our medical device (class II), and it is only intended to run on two versions of hardware. We are ISO 13485 and there is some leeway with statistical sample sizes/justification. I am struggling trying to understand if statistical sample size justification is required or not. Or if we can provide a justification for much smaller testing regime.

The firmware developer is trying to claim that they perform exhaustive testing of all features on all hardwares (n=2) . Testing more than 1 sample per hardware version is silly, because you are testing that the "bits" are transferring properly and working. The firmware also verifies itself on startup.

I am used to looking at a statistical plan for testing (ex: identifying confidence level and probability of finding a bug). I was balked at when throwing out a 90% CI with 15% bug finding resulting in a n=15 sample size.

Now, I do Usability testing with larger sample size that validates a lot of these firmware functionality. But it isn't officially verifying the input/output at the firmware "language" level. Does anyone have some insight on proper sampling method for this situation?

Sponsored Links
  Post Number #2  
Old 13th March 2018, 06:48 PM
yodon

 
 
Total Posts: 1,156
Re: Firmware Verification Testing & Sample Size

The reason for a sample size is to account for variability. Usability testing has quite a bit of variability through the users.

Software, though, given the same inputs and conditions, is (supposed to be) deterministic and so doing more that one test given the same setup will give the same result. So, yes, a 'sample size' of 1 is appropriate for software tests. Given that you have, in your example, 2 hardware platforms, it is appropriate to test on both platforms.

The point about software verifying itself at startup is outside the scope of formal firmware verification (although, of course, the verification process itself would be verified).
Reply

Lower Navigation Bar
Go Back   The Elsmar Cove Business Systems and Standards Discussion Forums > >

Bookmarks



Visitors Currently Viewing this Thread: 1 (0 Registered Visitors (Members) and 1 Unregistered Guest Visitors)
 
Thread Tools Search this Thread
Search this Thread:

Advanced Forum Search
Display Modes Rate Thread Content
Rate Thread Content:

Forum Posting Settings
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Emoticons are On
[IMG] code is On
HTML code is Off


Similar Discussion Threads
Discussion Thread Title Thread Starter Forum Replies Last Post or Poll Vote
Determining Sample Size for FDA Verification and Validation Activities QualityNewbie Inspection, Prints (Drawings), Testing, Sampling and Related Topics 41 11th March 2015 05:34 AM
Design Verification Sample Size - What philosophies are out there MNPIKE Design and Development of Products and Processes 5 16th February 2013 12:12 PM
Sample Size for Design Verification and Validation gxzn008 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 6 9th November 2011 09:17 AM
Sample Size for Design Verification (Prototype Testing) for Medical Devices amadeuscurse Qualification and Validation (including 21 CFR Part 11) 12 27th July 2011 04:56 AM
Sample Size in Design Verification & Process Validation MasterBB Inspection, Prints (Drawings), Testing, Sampling and Related Topics 2 16th November 2009 09:00 PM



The time now is 02:50 AM. All times are GMT -4.
Your time zone can be changed in your UserCP --> Options.



Misc. Internal Links


NOTE: This forum uses "Cookies"