Subject: Re: Software Release /Roque/Dey
Date: Tue, 8 Jun 1999 10:33:23 -0600
From: ISO Standards Discussion
From: Pat Dey
Subject: RE: Software Release /Roque/Dey
> From: maryanne_roque
> Subject:Q: Software Release /Roque
>
> Is there any one out there who could give me some input as to :
> a) when do you release a software product
When you are confident that you will earn more from revenues than you will lose in the cost of fixing problems you overlooked.
That sounds cycnical. But if you consider costs in their widest sense, it's in everyone's interests.
If you're selling low price high volume software, you might not want to wait until it's 100% bug free: Microsoft and many others have shown that first to market wins -- and the market will buy software that's "good enough" provided it's cheap enough. (Sure, there are complaints and you have to factor in the cost of fixing complaints and consequent loss of business.)
For low volume, high price custom software the cost of problems will usually include higher support costs, reduced customer referenceability. Ultimately, poor quality can cost current and future contracts and consequently you will probably want to be pretty confident it's good before release.
> b) how do you classify test failures (minor, major, catasthropic?)
In terms of the cost to the customer, eg,
Severity 1 - castrophic, no workaround (eg engine will not start)
Severity 2 - major functional failure, workaround exists (engine starts, must keep revs up)
Severity 3 - minor functional failure (turn signals don't work)
Severity 4 - cosmetic (don't like digital dash, prefer analog instruments)
> c) when there are bugs found during system (integration) test, do you
> still release the product?
If custom software, no severity 1 or 2 bugs unless customer agrees to take delivery, eg, to start their integration test.
Most important thing here is openness with the customer.
> d) any reasons of not releasing a product to the customer?
Existence of severity 1 or 2 bugs that will cause customer serious expense. Worst is to deliver knowing such problems exist and wasting customer's time (= money) finding them and reporting them back to you (not to mention their loss of confidence in you).
Better to be late and right than on time and wrong, in my view.
> e) how do you track test failures?
>
There are several bug tracking tools out there which enable you to keep a list of open issues, prioritize them, assign them to people and track them to closure. Try the Configuration Management Yellow Pages at *** DEAD LINK REMOVED *** for an extensieve review of the subject. You can spend as many or as few bucks as you like. Important thing is to define a tracking discipline and get buy-in from the engineers involved.
> Appreciate any input that I will receive from you fellows.
Good luck,
Pat
Date: Tue, 8 Jun 1999 10:33:23 -0600
From: ISO Standards Discussion
From: Pat Dey
Subject: RE: Software Release /Roque/Dey
> From: maryanne_roque
> Subject:Q: Software Release /Roque
>
> Is there any one out there who could give me some input as to :
> a) when do you release a software product
When you are confident that you will earn more from revenues than you will lose in the cost of fixing problems you overlooked.
That sounds cycnical. But if you consider costs in their widest sense, it's in everyone's interests.
If you're selling low price high volume software, you might not want to wait until it's 100% bug free: Microsoft and many others have shown that first to market wins -- and the market will buy software that's "good enough" provided it's cheap enough. (Sure, there are complaints and you have to factor in the cost of fixing complaints and consequent loss of business.)
For low volume, high price custom software the cost of problems will usually include higher support costs, reduced customer referenceability. Ultimately, poor quality can cost current and future contracts and consequently you will probably want to be pretty confident it's good before release.
> b) how do you classify test failures (minor, major, catasthropic?)
In terms of the cost to the customer, eg,
Severity 1 - castrophic, no workaround (eg engine will not start)
Severity 2 - major functional failure, workaround exists (engine starts, must keep revs up)
Severity 3 - minor functional failure (turn signals don't work)
Severity 4 - cosmetic (don't like digital dash, prefer analog instruments)
> c) when there are bugs found during system (integration) test, do you
> still release the product?
If custom software, no severity 1 or 2 bugs unless customer agrees to take delivery, eg, to start their integration test.
Most important thing here is openness with the customer.
> d) any reasons of not releasing a product to the customer?
Existence of severity 1 or 2 bugs that will cause customer serious expense. Worst is to deliver knowing such problems exist and wasting customer's time (= money) finding them and reporting them back to you (not to mention their loss of confidence in you).
Better to be late and right than on time and wrong, in my view.
> e) how do you track test failures?
>
There are several bug tracking tools out there which enable you to keep a list of open issues, prioritize them, assign them to people and track them to closure. Try the Configuration Management Yellow Pages at *** DEAD LINK REMOVED *** for an extensieve review of the subject. You can spend as many or as few bucks as you like. Important thing is to define a tracking discipline and get buy-in from the engineers involved.
> Appreciate any input that I will receive from you fellows.
Good luck,
Pat