Correlation using t-test and correlation coefficient - What is the correct result?

[Miner]

Bev is right. It is not that we do not want to help you, but you are providing very minimal information. We would need specifics about the way the data was collected before we can provide meaningful advice.

The proper analysis could depend on very exact information on how the experiment was performed. For example, the exact same experiment may be run as a fully randomized experiment or as a split-plot experiment. The analysis technique is different in each case.

[amanbhai]

In Reply to Parent Post by Geoff Withnell

Without seeing the data, just the summary data, I suspect that the problem is that while the means and variances MAY be different, they are not ENOUGH different for a sample of 20 to reliably distinguish between them. I will bet if you plot both samples as histograms, and lay one over the other, you will see that they are very similar. When dealing with two populations such as we have here, with means separated by only a fraction of a standard deviation, a fairly large sample is usually required to show the separation, unless the SDs are very different. Not mathematically rigorous, but it often helps to look at the data visually to diagnose a problem.

Geoff Withnell

Offcourse the sample size in many cases maybe important.
but Do we determine ourself the sample size? or the population size?

[Bev D]

In Reply to Parent Post by amanbhai

Offcourse the sample size in many cases maybe important.
but Do we determine ourself the sample size? or the population size?

we never determine the appropriate sample size, but we should calculate it (in this case, the definition of the words matter): the system from which we are sampling determines the sample size. The process's standard deviation and homogeneneity of data determines what sample size and sampling scheme is necessarry to make accurate estimates. We can choose how accurate we wish to be and how much difference we wish to detect. This will enable us to calculate what sample size is necessarry to meet our wishes.

Population size is mostly an irrelevent question in the practical world (unless very small)

[jas4710]

In Reply to Parent Post by Bev D

Again: we need to know EXACTLY what was done. what factors were tested, what levels for each factor? what was the sample size for each level? Was the 'experiment' passive (just took X and Y data as it naturally occured) or was it invasive? (deliberately set levels of the factor(s) and then collected data on the resulting output Y.) was it a series of one factor at a time experiments or were there full factorial or fractional factorials?

I will again caution that you that very often when 'experiemnts' are not properly structured, no amount of analysis will help; you may need to design a bettter experiment.

You can use 'nicknames' for the factors but we need more than the general info you've provided.

While in a.xls, better correlation in r1-r2 (col L) was identified, while less good result was observed in b.xls. In fact, there are far more factors but my boss said just focus on two (b.xls missing one). There are other results files like the attached two. I hope this answer some of the questions. The experiment was invasive for the 2 factors (but should be accurate) and carried out with a set of factors that can be totally different from one to another. The experiment per se is not to test the effects of those factors but I am here to question whether the factors will affect the results significantly. My boss believes normalization should help but he also agrees that how to normalize well is difficult. I'm unable to repeat the experiments with different factors.

Thank you once again for Bev and Miner's response.