The Cove Business Standards Discussion Forums
Go Back   The Elsmar Cove Business Systems and Standards Discussion Forums > >
Forum Username

Elsmar Cove Forum Visitor Notice(s)

Wooden Line

Firmware Verification Testing & Sample Size

Monitor the Elsmar Forum
Courtesy Quick Links

Links Elsmar Cove visitors will find useful in the quest for knowledge and support:

Jennifer Kirley's
Conway Business Services

International Quality Services

Marcelo Antunes'
SQR Consulting, and
Medical Devices Expert Forum

Bob Doering
Bob Doering's Blogs and,
Correct SPC - Precision Machining

Ajit Basrur
Claritas Consulting, LLC

International Standards Bodies - World Wide Standards Bodies

AIAG - Automotive Industry Action Group

ASQ - American Society for Quality

International Organization for Standardization - ISO Standards and Information

NIST's Engineering Statistics Handbook

IRCA - International Register of Certified Auditors

SAE - Society of Automotive Engineers

Quality Digest

IEST - Institute of Environmental Sciences and Technology

Some Related Topic Tags
firmware, sample size, statistical analysis and studies, verification (general), verification testing
Thread Tools Search this Thread Rate Thread Content Display Modes
  Post Number #1  
Old 13th March 2018, 03:13 PM

Total Posts: 1
Please Help! Firmware Verification Testing & Sample Size

Hi All,

We are developing custom firmware for our medical device (class II), and it is only intended to run on two versions of hardware. We are ISO 13485 and there is some leeway with statistical sample sizes/justification. I am struggling trying to understand if statistical sample size justification is required or not. Or if we can provide a justification for much smaller testing regime.

The firmware developer is trying to claim that they perform exhaustive testing of all features on all hardwares (n=2) . Testing more than 1 sample per hardware version is silly, because you are testing that the "bits" are transferring properly and working. The firmware also verifies itself on startup.

I am used to looking at a statistical plan for testing (ex: identifying confidence level and probability of finding a bug). I was balked at when throwing out a 90% CI with 15% bug finding resulting in a n=15 sample size.

Now, I do Usability testing with larger sample size that validates a lot of these firmware functionality. But it isn't officially verifying the input/output at the firmware "language" level. Does anyone have some insight on proper sampling method for this situation?

Sponsored Links
  Post Number #2  
Old 13th March 2018, 06:48 PM

Total Posts: 1,169
Re: Firmware Verification Testing & Sample Size

The reason for a sample size is to account for variability. Usability testing has quite a bit of variability through the users.

Software, though, given the same inputs and conditions, is (supposed to be) deterministic and so doing more that one test given the same setup will give the same result. So, yes, a 'sample size' of 1 is appropriate for software tests. Given that you have, in your example, 2 hardware platforms, it is appropriate to test on both platforms.

The point about software verifying itself at startup is outside the scope of formal firmware verification (although, of course, the verification process itself would be verified).

Lower Navigation Bar
Go Back   The Elsmar Cove Business Systems and Standards Discussion Forums > >


Visitors Currently Viewing this Thread: 1 (0 Registered Visitors (Members) and 1 Unregistered Guest Visitors)
Thread Tools Search this Thread
Search this Thread:

Advanced Forum Search
Display Modes Rate Thread Content
Rate Thread Content:

Forum Posting Settings
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Emoticons are On
[IMG] code is On
HTML code is Off

Similar Discussion Threads
Discussion Thread Title Thread Starter Forum Replies Last Post or Poll Vote
Determining Sample Size for FDA Verification and Validation Activities QualityNewbie Inspection, Prints (Drawings), Testing, Sampling and Related Topics 41 11th March 2015 05:34 AM
Design Verification Sample Size - What philosophies are out there MNPIKE Design and Development of Products and Processes 5 16th February 2013 12:12 PM
Sample Size for Design Verification and Validation gxzn008 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 6 9th November 2011 09:17 AM
Sample Size for Design Verification (Prototype Testing) for Medical Devices amadeuscurse Qualification and Validation (including 21 CFR Part 11) 12 27th July 2011 04:56 AM
Sample Size in Design Verification & Process Validation MasterBB Inspection, Prints (Drawings), Testing, Sampling and Related Topics 2 16th November 2009 09:00 PM

The time now is 11:14 AM. All times are GMT -4.
Your time zone can be changed in your UserCP --> Options.

Misc. Internal Links

NOTE: This forum uses "Cookies"