Not significant: what now? (Q1929699)

Summary: In a classic significance test, based on a random sample with size \(n\), a \(P\) value will be calculated at size \(n\) aiming to reject the null hypothesis. The sample size \(n\), however, can retrospectively be divided into partial samples \((1, 2, 3, \dots, n - 2, n - 1, n)\) and a test of significance can be calculated for each partial sample. As a result, several partial samples will provide significant \(P\) values whereas others will not show significant \(P\) values. In this paper, we propose a significance test that takes into account the additional information from the \(P\) values of the \(n - 1\) partial samples of a random sample. We show that the \(n - 1\) \(P\) values can greatly modify the results of a classic significance test.