Hi all,

I am planning to perform a screening DOE using Plackett-Burman design with 12 runs. There are 7 seven factors at 2 levels. The response variable is the 'Product Quality' and is determined by a quality inspector as 'Good', 'Medium' and 'Bad'. I would like to know how i can analyse the experiment results using Minitab 15 and if there is a particular method which is suitable to analyse qualitative response (NPP, ANOVA, Response surfact etc)

Many thanks in advance

Is there any possible way that you can get continuous data from this experiment? You will make life much easier on yourself.

Running an experiment such as you describe severely limits the tools available for analysis. For example: Regression requires both X and Y variables to be continuous. ANOVA/RSM require the Y to be continuous while the X can be continuous or attribute provided it is treated as attribute (i.e., discrete levels).

Your situation is an attribute X and attribute (ordinal) Y. This pretty well limits you to a tool like the Chi-Square test or Fisher's exact test. Minitab 15's Chi Square limits you to a maximum of 2 Xs and 1 Y. You can analyze the results repeatedly with different pairs of Xs, but this will increase the experiment-wise error rate significantly.

Do not. I repeat, do not make the mistake of treating ordinal data as pseudo-continuous and analyzing it using ANOVA. While the directionality of your data does indicate a direction for improvement, the distances between categories will not have the same meaning as in continuous data.

One last option that you may have (outside of Minitab) is a method promoted by Taguchi called Accumulation Analysis (http://math.usm.my/research/OnlineProc/ST57.pdf)that he developed for ordinal data. I will warn you in advance that most statisticians do not accept it as a valid test. Assumptions made for continuous data such as the sparcity of effects (of high order interactions) are not valid with ordinal data. This can result in spurious results and reversal of the importance of factors.

Hi Miner,

Thanks a lot for your informative reply. The product in question is a 'Coupling Disc' made of Composite Material and the response variable is the 'Amount of voids' formed at the end of the manufacturing process. The amount and size of voids varies and it depends on many factors. The size of a void can vary from 1 mm to 10 mm and the total nr of voids in one disc can be 100+. Therefore it is not possible to get a continuous data for 'Amount of voids', and a quality inspector make a visual inspection of the disc and rate the disc as Good, Medium or Bad quality.

As you mentioned that the only option i have is Chi square or Fisher's exact test. Is there any method or software which can allow me to analyse 7 factors and 1 response for Chi square outside Minitab. so i dont increase the experiment wise error.

I think that you do have a few options here. Two that I would try are:



Create a grid or circle of a known size and in a repeatable location on each part. Count the number of voids within that area. Minitab supports the Freeman-Tukey transform for count data (Calc > Calculator > Function > Transform Count).
Same grid or circle. Measure average size of void.


I would do both of these and analyze them separately.

I would also advise a few other things:

first ensure that that portion of the disk that you count is either representative of the whole disk in terms of voids OR that you select the worst area in terms o fvoids. Of course if teh voiding is not homogenously distributed that is a big clue that may enable you to narrow your field of supect variables.

You really should perform a repeatability and reproducibility study on whoever is 'inspecting' the disks. This is absolutely critical with human visual judgment and ordinal scales...Does each inspector bucket the same disks in the same bucket each tiem they look at it? Does inspector B bucket the same disk in the same bucket as inspector A did? You will need a very high level of agreement here >95% as a rough estimate on a sunday morning.

What is your sample size of disks for each treatment? how did you derive that? The sample size will also be crucial to ensuring reliable results and conclusions

*I* would place the experimental disks together in a matrix representing the treatments and *I* would personally look at each disk set along with the inspectors. If you have the primary factor(s) it should be patently obvious without statistics. If you need statistics to verify a small difference that isn't obvoius to everyone - then you probably dont' have teh primary X in your experiment OR you didn't set the levels far enough apart (but nor beyond what they would normally be in production!)

I would also strongly suggest a confirmatory experiment - a 1 - 4 factor full factorial experiemtn to confirm. although PBs are highly effective screening tests they are screening tests. There is confounding that you cannot resolve until you run the full test.

Hi Bev D,

A Gauge R&R is already carried out and the level of agreement is 85% but the treatment runs will be inspected by only one inspector as we only need to count the voids and not declare them as Bad or Good.

The sample size is 1 disc per treatment for cost saving purpose. The idea is to first identify the Main factors and once the main factors have been identified then a Full-factorial experiment with 1-4 factors with two 'discs' per trial (2 replicates) will be performed for confirmation purpose.

I have one more question, does it make any difference if i do a 7 factor; Fractional factorial design of 8 runs having resolution III instead of PB with 12 runs, as by doing so i have less treatments to perform and still i get the same knowledge from the experiments.

Sorry for the delay in response.

I recommend running the fractional factorial. Both experiments are resolution III, so the amount of confounding/aliasing is the same for both. The fractional factorial has fewer runs, and several hidden advantages.

Advantage 1: Each main effect that you remove from the model adds a hidden replicate and has the effect of reducing the fractionation and increasing the resolution. Removing 1 main effect changes your experiment from a 1/16th fraction to a 1/8th fraction. Removing a second main effect changes it to a 1/4th fraction. This can rapidly increase the resolution of the design.

Advantage 2: If the first advantage does not work, you can run a foldover design (a mirror image) that will reduce the confounding. This would result in an additional 8 runs totaling 16 (only 4 additional runs than the PB design).

The PB design is only advantageous when you saturate the design with the maximum allowable number of factors.

Hi Miner,

I agreed with you and i think its a good idea to perform Factional factorial design, i have one question when i do the foldover design, do i need to fold on all factors or some of the factors. i mean if i say on all factors i mean all 'High level' will be switch to 'Low level' and vise versa.

I have attached my design, can you give any comments on that. I got this design using Minitab 15 and folded on all factors and i managed to reduced the number of control factors to 5, so it makes more sense to use fractional factorial design instead of PB :)

thanks in advance

/tahir

Your experiment is now a resolution IV design, so there should be no issues with confounding/aliasing.

Regarding foldover designs. The typical progression is to run the first fraction (non foldover design). If there is high risk of confounding, such as in a resolution III design, you then create the foldover design (lows are now highs and highs are lows) and run it. In you design it is a half fraction, so the foldover is the other half.

So do i need to perform all the 16 runs and then analyse the results or first i can run 8 runs and then analyze them and later perform the remaining 8 runs and analyse them and since the design is folded; the 'active factors' from both first 8 run and last 8 runs should be the same and if they are not then i can say there are other factors also which are effecting the response.

No. Run the 8 runs first and analyze them. Reduce the model until you only include significant terms. Then examine the Alias Structure in Minitab's Session Window. If there is aliasing (confounding) that you believe to be a credible risk, then you may want to run the foldover experiments.

The operative word is credible risk. Your design is resolution IV, so main effects are aliased with 3-way interactions, and 2-way interactions are aliased with other 2-way interactions.

3-way interactions are relatively rare in electromechanical processes, but are more common in chemical or nuclear processes. 2-way interactions are relatively common (Time-Temp, Pressure-Temp., etc.).

If you only have main effects in your model and it is not a chemical process, you probably do not need to run the foldover. If it is a 2-way interaction and the aliased interaction is not technically feasible, you do not need to run the foldover. If it is a 2-way interaction and the aliased interaction is possible, you should run the foldover to isolate the real interaction.

Finally, fractional factorials are used for screening out unimportant factors/interactions from important ones. You should then follow-up with a modeling experiment, typically a full factorial, on the important factors and interactions. If you ran centerpoints in your screening experiment to detect curvature, you will know whether to use 2 or 3 levels in your full factorial. You could also add axial points in a third experiment to develop a Response Surface Design to optimize your control factors.

I am thinking of running the foldover design even if i dont suspect any two or three factors interaction to be active. The reason is that i think there might be some factors which effect the response but are not included as a control factor. so if i run the foldover over design and i dont get the same results as the first runs then i can go back and include more control factors. But if i get the same results then i can proceed with a modelling experiment. But what decides to either run a 'Response surface design' or a 'Full factorial design' for the third wave of experiments.

You are starting with a 1/2 fraction experiment. After running the foldover experiment and combining the two, you have a full factorial run in two blocks. If you add centerpoints, you can detect if curvature exists.

If no curvature exists, the full factorial is your model and there is no need for a response surface design.

If curvature does exist, you can add axial points as a third block and analyze the combined three experiments as a response surface.

Please correct me if I am wrong, I am starting with 5 factors in 8 runs (marked as yellow) so it means i have 1/4 fraction and after running the fold over design and combining the two results (yellow + green) i get a 1/2 fraction, so it means in the end i have 1/2 fraction and not a full factorial run (still some 2 factor interaction confounded with each other).

I have discussed with the process engineer and there is no need to run central points as we don't suspect any curvature.

You are correct, but this experiment is different from the previous attachment.

Thanks Miner, yes the DOE design is continuously evolving :), can you please comment on my analysis technique

Step 1: Conduct the first 8 runs and analyse results on NPP

Step 2: Conduct the second 8 runs (fold over) and analyse the results on NPP. (the results should be the same, and if they are not then it means i am missing some factors in my design which are effecting the response.

Step 3: Combine the results of step 1 and step 2 and analyse the results to build the model

I am not sure what you mean by NPP, but the rest of your plan appears sound.

You may have to add runs if you have significant results in these interactions, but you should be okay otherwise.
Slide 5 .O {color:black; font-size:149%;} a:link {color:#CCCCFF !important;} a:active {color:#3333CC !important;} a:visited {color:#B2B2B2 !important;} •BC + DE
•BD + CE
•BE + CD

By NPP i mean i will plot the contrasts on Normal Probability Paper to find which factors are active at an alpha risk of 0.5; or do you suggest any other method such as ANOVA. But i guess when using ANOVA to analyse the first 8 runs; all the Degrees of freedom (7) are utilized to estimate the 5 main factors + 2 two factor interactions and i cant calculate the 'Residual error' and i dont know if this will effect my analysis.

If i found BC interaction to be active then in order to resolve the confounding between BC and DE, how many additional runs do i need to run. Does it mean i need to fold on BC and DE both or just fold on one factor?

I would run Minitab's Analyze Factorial Design routine and select the Pareto graph in addition to the Normal Probability Plot. The Pareto plot uses a factor denoted PSE to estimate significance, until you reduce the model enough to estimate Residual Error.

Another way around this issue is to run 2-3 center points. The variation between center points is an estimate of residual error.

Your next foldover would complete the full factorial 1/4 to 1/2 to full. You could fold the original on a different factor in order to separate the BC interaction as you suggest. You will need to examine the alias structure and possibly try different foldovers to achieve what you want.

Another option would be to design a new experiment with only the confirmed and suspected significant factors/interactions. This may or may not result in fewer runs.

Hi Miner,

I hope you are doing well, now i have ran all the 16 experiments (8+8). I have analysed the results of the first 8 runs and factors B,D and E appear to be most significant and PSE value is 13 which i dont know what it means and i also don't suspect any 2-factor interaction to be active.

Now i would like to analyse the remaining 8 runs to see if i get the same results from them as it is a reflected design, but i don't know how i can analyse these remaining 8 runs. can i just replace the 'response variable' in the first 8 runs with the last 8 runs and analyse them (as the factors and confounding is the same) or do i need to analyse them with their original coefficient. And afterwards combine the results of first 8 runs and second 8 runs and analyse them. I have attached the results (Y1 is the results of the first 8 run and Y2 is the result of last 8 runs).

Can you please help me with the analysis.

Many thanks

Sorry for not replying sooner. I started analyzing your data, but got tied up and did not get a chance to finish. I'll get back on it Monday and respond then.

Hi Miner,

Thanks for your reply, i have also done some analysis and below are my findings

Step 1. (Before folding analysis)
I have analysed the first 8 runs (before folding) and i found that main factors 'B', 'D' and 'E' appear to be significant and none of the 2-factor interactions are significant. In my design all main factors are confounded with 3-factor interactions and i dont suspect 3-factor interactions to be significant. I have also plotted 'Main effects plot' and 'Interaction plot'. The main effect plot
shows that the high level of 'B', 'D' and 'E' leads to lower response values (which is desirable). The interaction plot does shows active interactions between different factors but i dont suspect any of these interaction to be significant; as it was found earlier from 'Pareto analysis'

Step 2 (After folding analysis)
I analysed the second 8 runs (after folding) and factors 'BC+DE', 'B' and 'C' appears to be significant. Since its a fold over design the results from the first 8 runs should match the second 8 runs. In my case they did not. Only main factor 'B' appears to be significant in both the runs and main factor 'A' does not seems to be significant in both the runs. By analysing the 'Main effects plot' i can see that factor 'B' has still the similar effect on the response, but the main factors 'D' and 'E' has very little effect on the response. And interestingly factor 'C' has a opposite effect on the response now. The two factor 'BC' is not not possible so i consider 'DE' to be active
although both 'D' and 'E' does not seem to be active.

After analysing the results from both the trials i conclude that, Factor 'A' is not effecting the response and factor 'B' is effecting the response but factors 'C','D' and 'E' are suspicious. For my future runs i am considering to run a 3 factors (C, D and E) full-factorial experiment with folding on all factors so i can make a model. In these runs i will keep factor 'B' at high level and factor 'A' will be at its economical level. And once i analysed these results i can finalize my model.

Please comment.

Sorry for the delay. I did get a chance to review your analysis today.

1) Your response variable is a percentage. To properly analyze a percentage variable, you should use the Freeman-Tukey transform. In Minitab, this is Calc > Calculator > Arithmatic > Transform Proportion. I could not do this for you because the transform requires the raw numerators and denominators that make up the proportion.

2) You need to reduce your models to determine the significant factors. Start with only the terms that show up in the pareto graph. Start eliminating one-at-a-time the smallest terms that lie below the red line on the pareto. Read up on Minitab's heirachy principle (you cannot remove a main effect if an interaction using it is still in the model). Some of the terms that you mentioned were not significant. This is part of the reason you got inconsistant results.

3) Since you have already run the foldover, focus on analyzing the combined data set. The first design is Block 1, the foldover is Block 2. Check for significance of the block, then remove it from the model if not significant.

4) When you have the model reduced to your satisfaction, check the residual plots and the R^2 Pred. If the R^2 Pred is low, you still have sources of variation that are not in the model. This could be factors not in the experiment, noise or measurement error.

Hi Miner,

It is not possible to transform the percentages using Freeman-Tukey transform. The reason is that 'Void image' is fed into a computer program which gives us the 'void percentage' value and it is not possible to get the required raw numerators and denominators from the program that makes up the proportion.

I have analysed the complete data set and reduce the model until I get a maximum R^2(pred) value but in that model there are terms which i don't find significant in Block 1 or Block 2 and also a 3 factor interaction appear to be significant but if i remove these terms then my R^2(pred) value drops significantly. Do you think its a good idea to keep these terms or i should remove these terms from my model? I have also checked the residuals and the residuals appear to be normally distributed and there is no pattern visible.