SBS - The Best Value in QMS software

Software Quality Metrics

#1
I am looking for best in class metrics regarding software quality.

Some metrics I recall are n# defects per 1 kLoc of changed code and < 3 found by the customer.
I am interested in any such metrics I can find... and wonder if there are different metrics for Cloud Micro services.

Any pointers to good materials... or if you can provide any insights it would be most appreciated!
thanks in advance, Bruce
 
Elsmar Forum Sponsor

Steve Prevette

Deming Disciple
Staff member
Super Moderator
#3
Some Suggestions:
1. Cyber Security concerns and workability may be in scope.
2. More than just defects per line of code is - looking at your V&V / Operational Testing and saying how many times did the software package actually fail a verification test run?
3. Beta test feedback
4. Customer feedback - did they test the software and say - This is not what we asked for . . .
5. On budget on schedule?
6. Response time of the software reasonable? Is it a "memory hog?"
7. How susceptible is the software to the next change in Windows 10 causing it to crash and burn?
 

yodon

Staff member
Super Moderator
#4
To add on to what @Steve Prevette wrote, I like to assess each bug for when the issue should or could have been caught; e.g., "phase escapes." That helps identify in your processes where you can make substantial improvements (as everyone knows, finding issues early is much cheaper).

On the flip side, be careful with metrics. You can negatively impact your processes. Number of lines of code per day is a particularly bad one, IMO. And then there's always this:

Metrics-Dilbert.jpg
 
#5
Thank you for these suggestions. For these or other metrics, are there best in class goals?
One metric I am particularly trying to figure out is how many bugs is it "OK" for our customers to see?

Fundamentally, I'm trying to figure out if the value I've measured is good or not. Where am I lagging industry best in class goals?

Example:
if 1% escape from Unit Test - is that good?
If customers generate 5% of my tickets - is that good?


Any pointers would be greatly appreciated!
 

Steve Prevette

Deming Disciple
Staff member
Super Moderator
#6
It depends. As your mother may have said - if everyone else is jumping off the pier, do you have to also? "Benchmarking" can be very dangerous, leading the pack can lead to corporate success. It really comes down to is the corporation being successful as judged by its customers, the board of directors, stakeholders, and regulators.

What is telling also (and more important than the level of the metric) is when metrics are [significantly] changing in their levels / have trends. Why did the trends occur, are they positive or negative influences, and what can/should you do about the trends.
 

yodon

Staff member
Super Moderator
#7
Excellent post by @Steve Prevette - unless there's a really level playing field, comparisons are difficult. Use your metrics to improve your processes. Oh, and never, ever, use metrics in punitive measures. That leads to distrust, hiding data, etc.
 
#8
Thanks for everyone's responses.

Always the biggest hurdle when not comparing apples to apples... "it depends"...
And baselining is (of course) the basis for any corrective action strategy.

I'm hoping to find anything written on this topic that will help me move past my "gutt"

Any pointers to text books or case studies?
Anyone have personal advice or experience as it relates to Software Quality metric goals?
 

optomist1

A Sea of Statistics
Super Moderator
#10
here is to hoping there is a remnant of a sense of humor today....when I was first exposed the SW QA, Software Quality was considered a bit of an oxymoron, :cool:
 
Thread starter Similar threads Forum Replies Date
D Software Quality Metrics Software Quality Assurance 8
Ajit Basrur Dashboard Software for representing Quality Metrics like DPPM, On Time Delivery, etc. Quality Tools, Improvement and Analysis 4
O Any info on release date of FDA “Computer Software Assurance for Manufacturing and Quality System Software” document? 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 0
S SOP for ISO 13485:2016 Quality related Software validation ISO 13485:2016 - Medical Device Quality Management Systems 9
W Air Quality Measurement Hardware and Software General Measurement Device and Calibration Topics 11
K Templates for software development quality audit Document Control Systems, Procedures, Forms and Templates 1
R ISO 13485 Software validation procedure and Quality Objectives Monitoring wanted Document Control Systems, Procedures, Forms and Templates 1
T Software quality training course Training - Internal, External, Online and Distance Learning 2
R Quality Management Software Recommendations ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 5
S ERP Software and Quality Department Procedures Integration Misc. Quality Assurance and Business Systems Related Topics 4
P Need suggestion on Software or App for Tracking Quality Plan Status Software Quality Assurance 1
A Quality system vs SDLC (Software Development Life Cycle) (or ALM)? Other US Medical Device Regulations 2
G QMS Quality Plan Documents Software Document Control Systems, Procedures, Forms and Templates 4
P Why does a SQA (Software Quality Assurance) standard be implemented in our company Software Quality Assurance 4
A Software Tools for managing a QMS (Quality Management System) ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 2
S Comparing Quality System Software Features Quality Assurance and Compliance Software Tools and Solutions 18
A Suggest Computer Aided Quality Assurance (CAQ) software for Medical Devices Quality Assurance and Compliance Software Tools and Solutions 3
A Quality Software TQS-9000 Family of Products Quality Manager and Management Related Issues 3
R Moving to an Electronic System - Quality System Software Recommendations ISO 13485:2016 - Medical Device Quality Management Systems 5
M Has anyone used "Paradigm 3" software to Control their Quality or Management System? Quality Tools, Improvement and Analysis 2
M Software for our Quality System - IPI Solutions Experiences/Reviews wanted Quality Assurance and Compliance Software Tools and Solutions 3
T Quality Compliance Software to meet ISO 13485 QMS and 21 CFR 820 Requirements Quality Assurance and Compliance Software Tools and Solutions 3
M Auditing Test Management Software such as HP Quality Center (HPQC) Test Tool Internal Auditing 1
D Has anyone used Mango as their Quality Software? Quality Assurance and Compliance Software Tools and Solutions 9
P Software Quality Assurance - Projects developed without a Contract Software Quality Assurance 10
michellemmm Software Product Development Quality Assurance (PDQA) Audit Checklists Software Quality Assurance 12
D TQM (Total Quality Management) Diagram Software Quality Assurance and Compliance Software Tools and Solutions 4
T Software for Quality Management suggestions? Quality Assurance and Compliance Software Tools and Solutions 14
D Switching to a new Quality Management software - Seeking recommendations Quality Assurance and Compliance Software Tools and Solutions 4
Marc What software (if any) do you use to manage your quality system? Quality Assurance and Compliance Software Tools and Solutions 57
C Software Quality Assurance Plan Template Software Quality Assurance 7
R Supplier Quality Investigation - Software and Hardware Issues Supplier Quality Assurance and other Supplier Issues 8
M Cost of Software to Calculate the Cost of Quality Quality Assurance and Compliance Software Tools and Solutions 10
C Comprehensive Quality Management Software (database) Recommendations wanted Quality Tools, Improvement and Analysis 13
M Has anyone used Quality Software from ION called Simple Trak Quality Assurance and Compliance Software Tools and Solutions 7
A From Supplier Quality Assurance to Software Quality Assurance Software Quality Assurance 5
S Software Quality Audit - Internal Audit Checklists Software Quality Assurance 6
S A Sample Quality Manual for Software Industry Software Quality Assurance 2
E Quality Software - Who uses software to manage your Quality System? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 2
E Quality System Software with TS 16949, Does it good? IATF 16949 - Automotive Quality Systems Standard 1
Sidney Vianna New ISO/IEC standard: quality of IT system / software engineering life cycle process Other ISO and International Standards and European Regulations 1
B ETQ Quality & Compliance Software - Opinions or Reviews wanted Quality Assurance and Compliance Software Tools and Solutions 35
A CDRH Guidance Document - Software bugs submitted in 510(k): Quantity or Quality? 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 3
S Information regarding the Software quality systems Software Quality Assurance 3
G Report template for quality process audits of the software product teams Internal Auditing 1
F Seven Quality Tools diagrams and charts software - Excel alternative? Quality Tools, Improvement and Analysis 9
B Quality Training Software Recommendations Training - Internal, External, Online and Distance Learning 1
S Quality Defect Management Software - Discussion and Recommendations Misc. Quality Assurance and Business Systems Related Topics 3
S Internal Quality Audit Software - IQA Dash Board Internal Auditing 1
T QFD (Quality Function Deployment) Software recommendations Quality Assurance and Compliance Software Tools and Solutions 3

Similar threads

Top Bottom