The Limit of Finite Sample Size and a Problem with Subsampling
Joint with: Donald Andrews
This paper considers tests and confidence intervals based on a test statistic that has a limit distribution that is discontinuous in a nuisance parameter or the parameter of interest. The paper shows that standard fixed critical value (FCV) tests and subsample tests often have asymptotic size—defined as the limit of the finite sample size—that is greater than the nominal level of the test. We determine precisely the asymptotic size of such tests under a general set of high-level conditions that are relatively easy to verify. Often the asymptotic size is determined by a sequence of parameter values that approach the point of discontinuity of the asymptotic distribution. The problem is not a small sample problem. For every sample size, there can be parameter values for which the test over-rejects the null hypothesis. Analogous results hold for confidence intervals. We introduce a hybrid subsample/FCV test that alleviates the problem of overrejection asymptotically and in some cases eliminates it. In addition, we introduce size-corrections to the FCV, subsample, and hybrid tests that eliminate over-rejection asymptotically. In some examples, these size corrections are computationally challenging or intractable. In other examples, they are feasible.
For more information, contact Vee Roberson.