Every problem in world has linear regression equation ?!!??

[cyberspider]

This refers to MiniTab's regression output, (Stat > Regression > Regression). With all due respect to MiniTab guys, can anyone tell why all the variables (Xs) are correlated in linear manner and not a single quadratic term ?? This also leads to next question, if Std. Dev in regression output (written in MiniTab output as s) can not be fully explained by linear terms, how do we find out, which X is to be made quadratic ?

Thanks in advance.

 

[Jim Wynne]

This refers to MiniTab's regression output, (Stat > Regression > Regression). With all due respect to MiniTab guys, can anyone tell why all the variables (Xs) are correlated in linear manner and not a single quadratic term ?? This also leads to next question, if Std. Dev in regression output (written in MiniTab output as s) can not be fully explained by linear terms, how do we find out, which X is to be made quadratic ?

Thanks in advance.

How do the folks at MiniTab explain it?

 

[Darius]

I don't know Minitab but I figure out that maybe you are looking somewhere else, it should be somewhere, I used NCSS sometime ago and it has regresion-non lineal and you can do multiple non lineal regresion.

 

[Statistical Steven]

This refers to MiniTab's regression output, (Stat > Regression > Regression). With all due respect to MiniTab guys, can anyone tell why all the variables (Xs) are correlated in linear manner and not a single quadratic term ?? This also leads to next question, if Std. Dev in regression output (written in MiniTab output as s) can not be fully explained by linear terms, how do we find out, which X is to be made quadratic ?

Thanks in advance.

You are asking a very interesting modeling question. Do you have any reason to suspect that any of your predictors should be quadratic? If so, create a quadratic term in your model and test if it is significant.

Most people use linear models because they are easy to explain. When you get into quadratics, Logarithmic or any power function, unless you have an explanation, it is hard to convince people that you are not over-fitting the model.

 

[Barbara B]

If you choose the model in the manner your described, you'll get a linear regression model (linear in parameters, not in the X's). So if you want to have non-linear (e. g. quadratic) X's you have to calculate a new variable with X²-values and make another regression with X² instead of X.

Developing a linear regression model includes a few steps:

1. Make a scatterplot matrix of all X vs. Y to detect the kind of dependencies. If you see a non-linear connection, transform X in a manner that the connection is afterwards (nearly) linear. (Nice candidates to start are log(x), 1/x, x², x³,..)

2. Let the software build your model and take a deep look at the residual plots! Minitab gives you four: qq-plot, residuals vs. fitted, histogram and a run chart. The values should all be near the line in the qq-plot. There shouldn't be a pattern at the residuals vs. fitted plot, the histogram should be bell-shaped and the run chart shouldn't have patterns either.

3. If patterns exist: Go back to the process (or the source of data) and look for systematic influences, perhaps there are factors which were forgotten to track. If there are time-dependencies, a time-series model could be more appropriate than a simple linear regression model. If you have a heteroscedastic residuals vs. fitted plot (looks like a trumpet), the variance of the response Y is not constant. You have to find a transformation for Y there.

4. If patterns are absent: R² should be "good enough" (for production higher than .80). If the linear model is appropriate and despite this R² is poor, the variation among the X's is too high.

To answer your first question: The default is that all X's are modeled linear. A search for better modelling as a standard procedure will perhaps give you the best fitted model from a mathematical viewpoint, but it will be more difficult to interpret it and it could be an over-fitted model. E. g. for 8 X's and the number of possible transformations restricted to 6 the software has to build 6^8=1'679'616 different models and select the best one.

Regards,

Barbara

 

[cyberspider]

...so I need to roll sleeves for making Xs quadratic

....I understand that I need to calculate Xs quadratic (hit and miss !! or trial and error things ). Thanks, Madam Barbara for the caution of overfit...

But does any software answer such thing automatically ?

 

[Statistical Steven]

....I understand that I need to calculate Xs quadratic (hit and miss !! or trial and error things ). Thanks, Madam Barbara for the caution of overfit...

But does any software answer such thing automatically ?

There is a software package called Table Curve that can fit over 3000 different curves to your data and rank them. You can select a subset of families of curves to fit to reduce the chance of overfit.

 

[Randy]

Duh! Now I know why I don't visit this thread

 

[Statistical Steven]

Duh! Now I know why I don't visit this thread

Randy, why?

 

[ralphsulser]

Duh! Now I know why I don't visit this thread

Statistics can be interesting. Back when Bo Dereck was in the movie "10",
we used to say she was +3 sigma above the mean