Never forget that even small effects can be statistically significant if the samples are large. To illustrate this fact, consider a sample of 145 small businesses. During a three-year period, 14 of the 99 headed by men and 8 of the 46 headed by women failed. (a) Find the proportions of failures for businesses headed by women and businesses headed by men. These sample proportions are quite close to each other. Give the P-value for the test of the hypothesis that the same proportion of women's and men's businesses fail. (Use the two-sided alternative). What can we conclude (Use a = 0.05)? The P-value was so we conclude that The test showed no significant difference.
(b) Now suppose that the same sample proportion came from a sample 30 times as large. That is, 240 out of 1380 businesses headed by women and 420 out of 2970 businesses headed by men fail. Verify that the proportions of failures are exactly the same as in part (a). Repeat the test for the new data. What can we conclude? The P-value was so we conclude that The test showed strong evidence of a significant difference.
(c) It is wise to use a confidence interval to estimate the size of an effect rather than just giving a P-value. Give 95% confidence intervals for the difference between proportions of men's and women's businesses (men minus women) that fail for the settings of both (a) and (b). (Be sure to check that the conditions are met. If the conditions aren't met for one of the intervals, use the same type of interval for both) Interval for smaller samples: to Interval for larger samples: to What is the effect of larger samples on the confidence interval? The confidence interval's margin of error is reduced.