In a test of significance (also called a hypothesis test), Type I error is the error of rejecting the null hypothesis when it is true — of saying an effect or event is statistically significant when it is not. The projected probability of committing type I error is called the level of significance and designated as alpha. For example, say you are testing whether the means of two samples differ. A 5% level of significance (alpha = .05) means that when the null hypothesis is true (i.e. the two samples are part of the same population and share the same mean), you believe that your test will mistakenly conclude that the sample means differ significantly 5% of the time.
