Software Quality Metrics

[bruceian]

I am looking for best in class metrics regarding software quality.

Some metrics I recall are n# defects per 1 kLoc of changed code and < 3 found by the customer.
I am interested in any such metrics I can find... and wonder if there are different metrics for Cloud Micro services.

Any pointers to good materials... or if you can provide any insights it would be most appreciated!
thanks in advance, Bruce

 

[Jim Wynne]

No help from me here, but just a clarification for the unenlightened (like me) "KLOC" is short for 1000 Lines of Code.

 

[Steve Prevette]

Some Suggestions:
1. Cyber Security concerns and workability may be in scope.
2. More than just defects per line of code is - looking at your V&V / Operational Testing and saying how many times did the software package actually fail a verification test run?
3. Beta test feedback
4. Customer feedback - did they test the software and say - This is not what we asked for . . .
5. On budget on schedule?
6. Response time of the software reasonable? Is it a "memory hog?"
7. How susceptible is the software to the next change in Windows 10 causing it to crash and burn?

 

[yodon]

To add on to what @Steve Prevette wrote, I like to assess each bug for when the issue should or could have been caught; e.g., "phase escapes." That helps identify in your processes where you can make substantial improvements (as everyone knows, finding issues early is much cheaper).

On the flip side, be careful with metrics. You can negatively impact your processes. Number of lines of code per day is a particularly bad one, IMO. And then there's always this:

 

[bruceian]

Thank you for these suggestions. For these or other metrics, are there best in class goals?
One metric I am particularly trying to figure out is how many bugs is it "OK" for our customers to see?

Fundamentally, I'm trying to figure out if the value I've measured is good or not. Where am I lagging industry best in class goals?

Example:
if 1% escape from Unit Test - is that good?
If customers generate 5% of my tickets - is that good?


Any pointers would be greatly appreciated!

 

[Steve Prevette]

It depends. As your mother may have said - if everyone else is jumping off the pier, do you have to also? "Benchmarking" can be very dangerous, leading the pack can lead to corporate success. It really comes down to is the corporation being successful as judged by its customers, the board of directors, stakeholders, and regulators.

What is telling also (and more important than the level of the metric) is when metrics are [significantly] changing in their levels / have trends. Why did the trends occur, are they positive or negative influences, and what can/should you do about the trends.

 

[yodon]

Excellent post by @Steve Prevette - unless there's a really level playing field, comparisons are difficult. Use your metrics to improve your processes. Oh, and never, ever, use metrics in punitive measures. That leads to distrust, hiding data, etc.

 

[bruceian]

Thanks for everyone's responses.

Always the biggest hurdle when not comparing apples to apples... "it depends"...
And baselining is (of course) the basis for any corrective action strategy.

I'm hoping to find anything written on this topic that will help me move past my "gutt"

Any pointers to text books or case studies?
Anyone have personal advice or experience as it relates to Software Quality metric goals?

 

[optomist1]

have you visited the CMMi @Carnegie Mellon Univ.? It beginnings are rooted in SW development and testing, etc..

https://cmmiinstitute.com