Best quadratic subset for analyzing a Response Surface Design with Minitab 15

W

weisse

Hello,
I use Minitab 15 and am new to this forum. I would like to automatically find the best subset of full quadratic regression terms when analyzing a response surface design. By "best," I mean having the highest predicted R^2. Does such a macro exist?
 
B

Barbara B

Re: Minitab 15 - Best quadratic subset for analyzing a response surface design

Hello weisse,

welcome to the Cove :)

There are many selection criterions or criterions for model comparison. R^2 is a very popular one, but it has three disadvantages:
  1. The more complex a models is, the higher R^2 gets, independent of the quality of the model. You can increase R^2 by adding terms, even if they are not significant and don't have any impact on the response.
  2. R^2-based model selection comes with a high risk of overfitting (adding terms without improving the amount of information from a model).
  3. Models with different numbers of terms shouldn't be compared using R^2, because R^2 doesn't reflect the information quality of a model but the fit quality.

One method to find the best model (=explains most based on the smallest number of terms) is a Wikipedia reference-linkstepwise_regression. To check whether a model is better compared to another model, an information criterion can be used (e.g. AIC: Wikipedia reference-linkAkaike_Information_Criterion).

Information criterions do reflect the information quality of a model, but the disadvantage is that they aren't limited to a specific range of values. R^2 could fall between 0 and 100%, whereas the information criterions are only numbers and could be (the smaller the number, the higher the information quality).

Minitab provides a stepwise regression only for linear effects (main effects in a DoE): Stat > Regression > Stepwise
This is done via the F-test with specified alpha values to add or remove a term.

AFAIK there exist no Minitab macro which supports model selection for a response surface model (linear, interaction, quadratic terms). Hopefully it will be included in the next release (R17), but this is just one out of several points on my wish list and I don't know if Minitab found that relevant, too.

Other software packages do provide these methods, e.g. R (function step), DesignExpert and JMP.

Hope this helps :bigwave:

Barbara
 
W

weisse

Re: Minitab 15 - Best quadratic subset for analyzing a response surface design

Barbara, thank you very much for the welcome and excellent information!

I should have been clearer about predicted R^2. I was referring to R-Sq(pred), not R-Sq, in Minitab.

Is there a software package that can automatically select the best response surface model based on R-Sq(pred) or, less preferably, R-Sq(adj)?
 
B

Barbara B

Re: Minitab 15 - Best quadratic subset for analyzing a response surface design

Sure, but I'm not quite sure if you really want to use SAS (see SAS GLMSELECT Procedure for details). You can choose PRESS as selection criterion there and R^2(pred) is derived from PRESS (see Minitab knowledgebase ID 922: What is the formula for Predicted R-Squared (R-Sq(pred))?).

Maybe JMP has a R^2(pred)/PRESS-selection option additionally to the F-test, since JMP is some kind of sub-product of SAS. You can ask e.g. Bev D who works with JMP for further details.

Just being curious: Why do you want to use a criterion which is differemt from the usual selection criterions? JMP and DesignExpert use F-tests and R an information criterion (AIC, BIC, DIC, etc.)
 
W

weisse

Re: Minitab 15 - Best quadratic subset for analyzing a response surface design

Thanks again, Barbara. You are a great resource. I'm not sure how I'd contact Bev D directly.

From ID 922 it looks like lowest PRESS would work the same as highest R^2(pred) for a selection criterion. Whether the criterion is R^2(pred), PRESS, or some other statistic, I'd like to use a software package that can automatically apply the criterion to all subsets of a full quadratic (linear + interactions + squares) response surface design. My eventual goal is to predict the optimal response, so I've been assuming R^2(pred) is best (see ID 922's link to ID 983). I'm an engineer with some knowledge of statistics but not an expert, so I welcome any suggestions for software and any advice about how to decide which statistic is the best criterion. I've heard good things about R, which I believe is free.
 
B

Barbara B

Re: Minitab 15 - Best quadratic subset for analyzing a response surface design

You're welcome :)

You can contact Bev D (and other users) by clicking on "Send Message to..." in the user profile and choosing either private message or visitor message (see attached screenshot). A specific user profile can be found by clicking on the username in a posting or with the search function.

R is OpenSource software and therefore free to use, but to work with R you have to write object oriented code. There are some packages for DoE which are currently available as beta version, but it will take some more time to finish them.

To get an automatic variable selection you have to take three steps:
  1. Recode your settings into coded numbers (-1: lower corner point, +1: upper corner point, 0: center point, alpha / appropriate value for star points)
  2. Build a full model including all main effects, interactions and quadratic terms (function lm)
  3. Use a stepwise selection (function step)
But you have to know how datasets are imported, how a formula for a model has to be written and which selection criterion should be used as well as how the results are interpreted. AFAIK there doesn't exist a response optimizer in R, so you have to write your own functions to get the best setting for your response(s).

A more convenient way is to use the trial versions of e.g. DesignExpert or JMP and analyze your design there using the default settings for automatic model selection.
 

Attachments

  • SendMessage.png
    SendMessage.png
    13.5 KB · Views: 218
W

weisse

Re: Minitab 15 - Best quadratic subset for analyzing a response surface design

I had downloaded the DX8 trial shortly before your more recent post. After using it some, I really like it! Thanks again.
 
Top Bottom