Software Releases - When do you release a software product?

[Marc]

Subject: Re: Software Release /Roque/Dey
Date: Tue, 8 Jun 1999 10:33:23 -0600
From: ISO Standards Discussion

From: Pat Dey
Subject: RE: Software Release /Roque/Dey

> From: maryanne_roque
> Subject:Q: Software Release /Roque
>
> Is there any one out there who could give me some input as to :
> a) when do you release a software product

When you are confident that you will earn more from revenues than you will lose in the cost of fixing problems you overlooked.

That sounds cycnical. But if you consider costs in their widest sense, it's in everyone's interests.

If you're selling low price high volume software, you might not want to wait until it's 100% bug free: Microsoft and many others have shown that first to market wins -- and the market will buy software that's "good enough" provided it's cheap enough. (Sure, there are complaints and you have to factor in the cost of fixing complaints and consequent loss of business.)

For low volume, high price custom software the cost of problems will usually include higher support costs, reduced customer referenceability. Ultimately, poor quality can cost current and future contracts and consequently you will probably want to be pretty confident it's good before release.

> b) how do you classify test failures (minor, major, catasthropic?)

In terms of the cost to the customer, eg,

Severity 1 - castrophic, no workaround (eg engine will not start)
Severity 2 - major functional failure, workaround exists (engine starts, must keep revs up)
Severity 3 - minor functional failure (turn signals don't work)
Severity 4 - cosmetic (don't like digital dash, prefer analog instruments)

> c) when there are bugs found during system (integration) test, do you
> still release the product?

If custom software, no severity 1 or 2 bugs unless customer agrees to take delivery, eg, to start their integration test.

Most important thing here is openness with the customer.

> d) any reasons of not releasing a product to the customer?

Existence of severity 1 or 2 bugs that will cause customer serious expense. Worst is to deliver knowing such problems exist and wasting customer's time (= money) finding them and reporting them back to you (not to mention their loss of confidence in you).

Better to be late and right than on time and wrong, in my view.

> e) how do you track test failures?
>
There are several bug tracking tools out there which enable you to keep a list of open issues, prioritize them, assign them to people and track them to closure. Try the Configuration Management Yellow Pages at *** DEAD LINK REMOVED *** for an extensieve review of the subject. You can spend as many or as few bucks as you like. Important thing is to define a tracking discipline and get buy-in from the engineers involved.

> Appreciate any input that I will receive from you fellows.

Good luck,
Pat

[Marc]

Subject: Re: Q: Software Release /Roque/Borges
Date: Tue, 8 Jun 1999 10:56:58 -0600
From: ISO Standards Discussion

From: Eliana Borges
Subject: RE: Q: Software Release /Roque/Borges

I will give it a shot, hoping this will help:

> a) when do you release a software product

When it meets the criteria for developing it. A criteria, I hope, was developed with the customer's success in mind. The software will be as good as the spec that was approved for its development. Hopefully the customer does have a representative speaking for them. Unfortunatelly, for all of us, some software is developed without much thought as to our success. And many of us may spend many hours dealing with it. Without mentioning something specific, think of all the products out in the market that are very quick to go to sale and then deal with the problems later, through customer support lines and "patch" or new releases of the product. There is a balance here - time to market versus risking the products name, versus customer support. It is really a team decision based on a study of facts. There should be someone running the project who is used to making such decisions and drawing the line. Someone has to make a decision as to when to "stop developing it" and to get it out. It is not an easy one, especially if the top spec and lower level specs were not well prepared. But the answer is still the same: stop when it meets the spec. Leave improvements for the future.

> b) how do you classify test failures (minor, major, catasthropic?)

You could use some self developed guidelines or look into industry standard guidelines. The good old military specs used to give us classification of defects and configuration control rules defining things as minor, major and critical. Minor and major are easier to understand. Critical was left for things that affected the loss of human life or property. Define the failure in respect to meeting the spec and meeting the user's needs - then classify it accordingly. What exactly do you mean with catastrophic? If you are dealing with regulated industries the classifications would be spelled out for you in the regulations or contract.

> c) when there are bugs found during system (integration) test, do you
> still release the product?

Does the product still meet the spec? What are the implications of the bug - how does it change what the software need to do? During integration we would hope you will run many test cases to ensure performance. If a bug is introduced the whole software configuration is now compromised because you do not know what other things may go wrong because of the bug. The least you can do is fix the bug and retest from the start.

> d) any reasons of not releasing a product to the customer?

All of the above plus the fact that it is unethical, and ruins careers and companies.

> e) how do you track test failures?

By developing a good failure reporting system which involves multi disciplinary functions and implements effective preventive and corrective action. Better than tracking test failures is preventing failures by breaking the test into small pieces and avoiding the enourmous task of trying to test it all at once. Integration should be the final rehersal of the orchestra, after all the instruments were already playing well together, all of them tuned up separately but working well with the others.

I hope I have given you some food for thought. And I hope the software you are talking about is not critical to the human race. And if it is, take good care of it.

Regards,

Ellie Borges

[Marc]

Subject: Re: Q: Software Release /Roque/Sanghavi
Date: Mon, 14 Jun 1999 08:33:04 -0600
From: ISO Standards Discussion

From: Jatin Sanghavi
Subject: Re: Q: Software Release /Roque/Sanghavi

I work for a company which has been ISO certified so I hope your questions will be answered well.

a) when do you release a software product
b) how do you classify test failures (minor, major, catasthropic?)

The Software product is released when there are no bugs but most of the software in the world have some minor bugs cause to err is human. Most of the software in the world have some minor bugs and to remove these bugs, patches are being released after the software product is lauched. These patches will have to be given to the customer free of cost, most of the companies put these patches on the website so that the customer can download it.

c) when there are bugs found during system (integration) test, do you still release the product?

As I have written above If the bugs are minor and will not put the customer and the company into any problems then there is no problem to launch it but in future this bugs have to be fixed for customer satisfication.

d) any reasons of not releasing a product to the customer?

If the software have major bugs and the customer is not satisfied with its performance then it is not advicable to release the product. The software can be tested by the software testing and quality department before release.

e) how do you track test failures?

The test failures must be recorded each time the person performs the test as per the stratergies laid down by the software testing and Quality department. These records can be on the computer or hand written by the testor and should be verified signed by his project lead/project manager after the bugs or error have been fixed by the programmer.

If you need any more clarification, please fell free to write to me.

Warm Regards

Jatin