<b Multiple looks:

In a classic statistical experiment, treatment(s) and placebo are applied to randomly assigned subjects, and, at the end of the experiment, outcomes are compared. With multiple looks, the investigator does not wait until the end of the experiment — outcomes are compared at earlier stages. The more often you look, the more likely it is that chance variability might produce an outcome of apparent interest. Therefore, the benchmark requirement for statistical significance must be adjusted accordingly to preserve Type I error rates. You cannot simply use the traditional p = 0.05 test each time you look at the data.

Browse Other Glossary Entries