Screening DOE (Design of Experiments) with qualitative response

[tahirawan11]

Hi all,

I am planning to perform a screening DOE using Plackett-Burman design with 12 runs. There are 7 seven factors at 2 levels. The response variable is the 'Product Quality' and is determined by a quality inspector as 'Good', 'Medium' and 'Bad'. I would like to know how i can analyse the experiment results using Minitab 15 and if there is a particular method which is suitable to analyse qualitative response (NPP, ANOVA, Response surfact etc)

Many thanks in advance

 

[Miner]

Re: DOE with qualitative response

Is there any possible way that you can get continuous data from this experiment? You will make life much easier on yourself.

Running an experiment such as you describe severely limits the tools available for analysis. For example: Regression requires both X and Y variables to be continuous. ANOVA/RSM require the Y to be continuous while the X can be continuous or attribute provided it is treated as attribute (i.e., discrete levels).

Your situation is an attribute X and attribute (ordinal) Y. This pretty well limits you to a tool like the Chi-Square test or Fisher's exact test. Minitab 15's Chi Square limits you to a maximum of 2 Xs and 1 Y. You can analyze the results repeatedly with different pairs of Xs, but this will increase the experiment-wise error rate significantly.

Do not. I repeat, do not make the mistake of treating ordinal data as pseudo-continuous and analyzing it using ANOVA. While the directionality of your data does indicate a direction for improvement, the distances between categories will not have the same meaning as in continuous data.

One last option that you may have (outside of Minitab) is a method promoted by Taguchi called Accumulation Analysis that he developed for ordinal data. I will warn you in advance that most statisticians do not accept it as a valid test. Assumptions made for continuous data such as the sparcity of effects (of high order interactions) are not valid with ordinal data. This can result in spurious results and reversal of the importance of factors.

 

[tahirawan11]

Re: DOE with qualitative response

Hi Miner,

Thanks a lot for your informative reply. The product in question is a 'Coupling Disc' made of Composite Material and the response variable is the 'Amount of voids' formed at the end of the manufacturing process. The amount and size of voids varies and it depends on many factors. The size of a void can vary from 1 mm to 10 mm and the total nr of voids in one disc can be 100+. Therefore it is not possible to get a continuous data for 'Amount of voids', and a quality inspector make a visual inspection of the disc and rate the disc as Good, Medium or Bad quality.

As you mentioned that the only option i have is Chi square or Fisher's exact test. Is there any method or software which can allow me to analyse 7 factors and 1 response for Chi square outside Minitab. so i dont increase the experiment wise error.

 

[Miner]

Re: DOE with qualitative response

I think that you do have a few options here. Two that I would try are:

1. Create a grid or circle of a known size and in a repeatable location on each part. Count the number of voids within that area. Minitab supports the Freeman-Tukey transform for count data (Calc > Calculator > Function > Transform Count).
2. Same grid or circle. Measure average size of void.

I would do both of these and analyze them separately.

 

[Bev D]

Re: DOE with qualitative response

I would also advise a few other things:

first ensure that that portion of the disk that you count is either representative of the whole disk in terms of voids OR that you select the worst area in terms o fvoids. Of course if teh voiding is not homogenously distributed that is a big clue that may enable you to narrow your field of supect variables.

You really should perform a repeatability and reproducibility study on whoever is 'inspecting' the disks. This is absolutely critical with human visual judgment and ordinal scales...Does each inspector bucket the same disks in the same bucket each tiem they look at it? Does inspector B bucket the same disk in the same bucket as inspector A did? You will need a very high level of agreement here >95% as a rough estimate on a sunday morning.

What is your sample size of disks for each treatment? how did you derive that? The sample size will also be crucial to ensuring reliable results and conclusions

*I* would place the experimental disks together in a matrix representing the treatments and *I* would personally look at each disk set along with the inspectors. If you have the primary factor(s) it should be patently obvious without statistics. If you need statistics to verify a small difference that isn't obvoius to everyone - then you probably dont' have teh primary X in your experiment OR you didn't set the levels far enough apart (but nor beyond what they would normally be in production!)

I would also strongly suggest a confirmatory experiment - a 1 - 4 factor full factorial experiemtn to confirm. although PBs are highly effective screening tests they are screening tests. There is confounding that you cannot resolve until you run the full test.

 

[tahirawan11]

Hi Bev D,

A Gauge R&R is already carried out and the level of agreement is 85% but the treatment runs will be inspected by only one inspector as we only need to count the voids and not declare them as Bad or Good.

The sample size is 1 disc per treatment for cost saving purpose. The idea is to first identify the Main factors and once the main factors have been identified then a Full-factorial experiment with 1-4 factors with two 'discs' per trial (2 replicates) will be performed for confirmation purpose.

I have one more question, does it make any difference if i do a 7 factor; Fractional factorial design of 8 runs having resolution III instead of PB with 12 runs, as by doing so i have less treatments to perform and still i get the same knowledge from the experiments.

 

[Miner]

Sorry for the delay in response.

I recommend running the fractional factorial. Both experiments are resolution III, so the amount of confounding/aliasing is the same for both. The fractional factorial has fewer runs, and several hidden advantages.

Advantage 1: Each main effect that you remove from the model adds a hidden replicate and has the effect of reducing the fractionation and increasing the resolution. Removing 1 main effect changes your experiment from a 1/16th fraction to a 1/8th fraction. Removing a second main effect changes it to a 1/4th fraction. This can rapidly increase the resolution of the design.

Advantage 2: If the first advantage does not work, you can run a foldover design (a mirror image) that will reduce the confounding. This would result in an additional 8 runs totaling 16 (only 4 additional runs than the PB design).

The PB design is only advantageous when you saturate the design with the maximum allowable number of factors.

 

[tahirawan11]

Hi Miner,

I agreed with you and i think its a good idea to perform Factional factorial design, i have one question when i do the foldover design, do i need to fold on all factors or some of the factors. i mean if i say on all factors i mean all 'High level' will be switch to 'Low level' and vise versa.

I have attached my design, can you give any comments on that. I got this design using Minitab 15 and folded on all factors and i managed to reduced the number of control factors to 5, so it makes more sense to use fractional factorial design instead of PB

thanks in advance

/tahir

 

[Miner]

Your experiment is now a resolution IV design, so there should be no issues with confounding/aliasing.

Regarding foldover designs. The typical progression is to run the first fraction (non foldover design). If there is high risk of confounding, such as in a resolution III design, you then create the foldover design (lows are now highs and highs are lows) and run it. In you design it is a half fraction, so the foldover is the other half.

 

[tahirawan11]

So do i need to perform all the 16 runs and then analyse the results or first i can run 8 runs and then analyze them and later perform the remaining 8 runs and analyse them and since the design is folded; the 'active factors' from both first 8 run and last 8 runs should be the same and if they are not then i can say there are other factors also which are effecting the response.