For screening a design, can we live with the low/poor predicted R-Sq.?

[v9991]

during analysis of the results, Design Expert provides three Regression coefficient for a model,
R-Squared
Adj R-Squared
Pred R-Squared

A. the query is, can we ignore/live-with poor or low Pred R-Sq.
1) during screening designs(Res-II or Placket Burman Design),
2) there is an analysis of DOE through option for historical data ,

in both cases, we are essentially looking for not a model, but JUST co-efficient and their magnitude (apart from the main effects)

B. for exploratory work, can we do work with p-value on marginally higher side (approx 0.06-0.1 vs 0.05 )

 

[Miner]

A screening design is just that, a means of filtering a few significant factors from the many non-significant factors. It is not used for building a mathematical model. At such low resolution, it would miss all interactions anyway.

Since it is used for filtering only, I often recommend using an alpha of 0.1. The assumption is that once you have narrowed the field to the few factors of interest, you will follow up with a series of modeling or optimization experiments at a lower alpha that would confirm or dispel these borderline factors.

 

[v9991]

A screening design is just that, a means of filtering a few significant factors from the many non-significant factors. It is not used for building a mathematical model. At such low resolution, it would miss all interactions anyway.

Since it is used for filtering only, I often recommend using an alpha of 0.1. The assumption is that once you have narrowed the filed to the few factors of interest, you will follow up with a series of modeling or optimization experiments at a lower alpha that would confirm or dispel these borderline factors.

yes, that is the intent,
in the resulting model, we are trying to draw some inference from the co-efficients of the factors.
+ can you also pl comment on the predicted R-Sq.!

 

[Miner]

I would not use the results of a screening design for anything more than determining the significance of a factor and directionality (i.e., +/-). To interpret coefficients, you must have a mathematical model. To have a mathematical model, you must include any significant interactions (which you cannot test with a low resolution screening design). Omitting these interactions will change the coefficients of the factors.

BTW. Please be precise on your meaning behind coefficients. The most common meaning is Bx where Y = B0 + B1X1 + B2X2... Bx is a coefficient. R^2 is also less commonly called the coefficient of determination. I have been responding based on the assumption that you meant Bx when you asked about coefficients, but am no longer certain.

Regarding R^2:

• R^2 explains the percentage of variation explained by the mathematical model, but has one serious drawback. If you continue to add terms to the model, even if they are non-significant, R^2 will continue to increase.
• R^2 adjusted compensates for the flaws of R^2 by penalizing you for every term in the model. If the term adds value to the model, R^2 adjusted will increase, but if the term adds no value, R^2 adjusted will decrease. The ideal model will have similar R^2 and R^2 adjusted values.
• R^2 predicted goes even further by sequentially removing one observation from the data set, calculating the regression coefficients and predicting the missing datum. If you have over fit the model, R^2 predicted will drop. If you try to fit a linear model to a nonlinear data set, or are missing interaction, R^2 predicted will also drop.