Hacking statistics with R

>"Load this dataset and use the t.test() function to compute the confidence interval of the mean for both variables with a level of 95%. Does those random variables seems to follow distributions who have the same means?

We see that the confidence intervals doesn’t overlap. This is an indication that the real value of the mean of the first variable is not in the same interval as the distribution mean of the second variable. As a consequence, we can safely suppose than both mean are different and that they don’t have the same probability distribution."

The width of a confidence interval is an arbitrary choice, just because two fail to overlap doesn't tell you anything. You can see this very easily using R:

  x      = rnorm(1000)
  levels = seq(.5, .99, by = .01)
  ci     = matrix(nrow = length(levels), ncol = 2)
  
  for(i in 1:length(levels)){
	  ci[i,] = t.test(x, conf.level = levels[i])$conf.int
  }
  
  plot(levels, apply(ci, 1, diff), xlab = "Conf Level", ylab = "Interval width")