Return to Answer

By contrast, if wee take ranks of the combined data, the ranks will run from 1 to 40, so outliers are not likely. Yet, the relative standing of the values in the two samples is preserved. Consequently, the pooled 2-sample t test (correctly) does not reject.

By contrast, if we take ranks of the combined data, the ranks will run from 1 to 40, so outliers are not likely. Yet, the relative standing of the values in the two samples is preserved. Consequently, the pooled 2-sample t test (correctly) does not reject.

For simplicity, I will illustrate with pooled 2-sample t tests--instead of ANOVAs. Consider following fictitious data comparing two samples of size 20 from the same exponential distribution. There is no difference between the two populations, so a t test should not reject the null hypothesis.

However, the pooled 2-sample t test is (narrowly) significant at the 5% level.

For simplicity, I will illustrate with pooled 2-sample t tests--instead of ANOVAs. Consider the following fictitious data comparing two samples of size 20 from the same exponential distribution. There is no difference between the two populations, so a no test should reject the null hypothesis. We look first at a pooled 2-sample t test.

Nevertheless, the pooled 2-sample t test is (narrowly) significant at the 5% level.

One difficulty (out of several) with far outliers in what ought to be normal data is that the null hypothesis (no differences) will be rejected too often, leading to false discovery..

pv.r = replicate(105, t.test(rank(c(rexp(20,1),rexp(20,1)))~g, var.eq=T)$p.val)
mean(pv.r <= .05)
[1] 0.01904762

In this case, the true rejection rate when there is no difference between the two populations are the same is about 2%. Granted, about 5% would be better, but doing the pooled 2-sample t test on ranked data is better than ignoring the skewness of exponential data and resuting outliers.

One difficulty (out of several) with far outliers in what ought to be normal data is that the null hypothesis (no differences) may be rejected too often, leading to false discovery.

pv.r = replicate(105, t.test(rank(c(rexp(20,1),rexp(20,1)))~g, 
                 var.eq=T)$p.val)
mean(pv.r <= .05)
[1] 0.01904762

In this case, the true rejection rate when there is no difference between the two populations is about 2%. Granted, about 5% would be better, but doing the pooled 2-sample t test on ranked data is better than ignoring the skewness of exponential data and resuting outliers.