Invalid permutation tests
type: Item
instance of: scholarly article
title: Invalid permutation tests
zbMATH Open document ID: 1189.62079
DOI: 10.1155/2010/769780
published in: International Journal of Mathematics and Mathematical Sciences
publication date: 2010-06-25T00:00:00Z
full work available at URL: https://eudml.org/doc/232818
review text: Summary: Permutation tests are often presented in a rather casual manner, in both introductory and advanced statistics textbooks. The appeal of the cleverness of the procedure seems to replace the need for a rigorous argument that it produces valid hypothesis tests. The consequence of this educational failing has been a widespread belief in a ``permutation principle'', which is supposed invariably to give tests that are valid by construction, under an absolute minimum of statistical assumptions. Several lines of arguments are presented here to show that the permutation principle itself can be invalid, concentrating on the Fisher-Pitman permutation test for two means. A simple counterfactual example illustrates the general problem, and a slightly more elaborate counterfactual argument is used to explain why the main mathematical proof of the validity of permutation tests is mistaken. Two modifications of the permutation test are suggested to be valid in a very modest simulation. In instances where simulation software is readily available, investigating the validity of a specific permutation test can be done easily, requiring only a minimum understanding of statistical technicalities.
Mathematics Subject Classification ID: 62G10
Mathematics Subject Classification ID: 62A01
Mathematics Subject Classification ID: 65C60
zbMATH DE Number: 5726446
Wikidata QID: Q58651427
author: Aickin, Mikel
MaRDI profile type: Publication
OpenAlex ID: W2076379480
cites work: Q3262608
cites work: Concepts of nonparametric theory
cites work: On the Behavior of Randomization Tests Without a Group Invariance Assumption
cites work: PERMUTATION TESTS AND OTHER TEST STATISTICS FOR ILL-BEHAVED DATA: EXPERIENCE OF THE NINDS t-PA STROKE TRIAL
description: scientific article; zbMATH DE number 5726446