Statistical Analysis Stack Exchange
Statistical Analysis Stack Exchange

Questions tagged [permutation-test]

Statistical tests based on rearrangements of data that are consistent with the null hypothesis.

436 questions
89
votes
2 answers

Resampling / simulation methods: monte carlo, bootstrapping, jackknifing, cross-validation, randomization tests, and permutation tests

I am trying to understand difference between different resampling methods (Monte Carlo simulation, parametric bootstrapping, non-parametric bootstrapping, jackknifing, cross-validation, randomization tests, and permutation tests) and their…
Ram Sharma
  • 2,226
  • 3
  • 20
  • 24
68
votes
4 answers

Look and you shall find (a correlation)

I have several hundred measurements. Now, I am considering utilizing some kind of software to correlate every measure with every measure. This means that there are thousands of correlations. Among these there should (statistically) be a high…
David
  • 855
  • 1
  • 8
  • 7
61
votes
6 answers

Which permutation test implementation in R to use instead of t-tests (paired and non-paired)?

I have data from an experiment that I analyzed using t-tests. The dependent variable is interval scaled and the data are either unpaired (i.e., 2 groups) or paired (i.e., within-subjects). E.g. (within subjects): x1 <- c(99, 99.5, 65, 100, 99,…
Henrik
  • 13,314
  • 9
  • 63
  • 123
48
votes
3 answers

Bootstrap vs. permutation hypothesis testing

There are several popular resampling techniques, which are often used in practice, such as bootstrapping, permutation test, jackknife, etc. There are numerous articles & books discuss these techniques, for example Philip I Good (2010) Permutation,…
Tu.2
  • 2,627
  • 6
  • 26
  • 26
36
votes
3 answers

Computing p-value using bootstrap with R

I use "boot" package to compute an approximated 2-sided bootstrapped p-value but the result is too far away from p-value of using t.test. I can't figure out what I did wrong in my R code. Can someone please give me a hint for this time =…
Tu.2
  • 2,627
  • 6
  • 26
  • 26
26
votes
1 answer

Have the reports of the death of the t-test been greatly exaggerated?

Reading through CV all-time classics I came across a statement that I would like to clarify. This is the post and my question refers to the closing remarks: "I have to note that all of the knowledge I just imparted is somewhat obsolete; now that we…
Antoni Parellada
  • 23,430
  • 15
  • 100
  • 197
26
votes
1 answer

What are the assumptions of the permutation test?

It's often stated that permutation tests have no assumptions, however this is certainly not true. For example if my samples are somehow correlated, I can imagine that permuting their labels would not be the correct thing to do. Only think I found…
rep_ho
  • 6,036
  • 1
  • 22
  • 44
25
votes
1 answer

Using bootstrap under H0 to perform a test for the difference of two means: replacement within the groups or within the pooled sample

Suppose that I have a data with two independent groups: g1.lengths <- c (112.64, 97.10, 84.18, 106.96, 98.42, 101.66) g2.lengths <- c (84.44, 82.10, 83.26, 81.02, 81.86, 86.80, 85.84, 97.08, 79.64, 83.32, 91.04, 85.92, …
Newbie_R
  • 367
  • 1
  • 4
  • 8
24
votes
3 answers

t-test on highly skewed data

I have a data set with tens of thousands of observations of medical cost data. This data is highly skewed to the right and has a lot of zeros. It looks like this for two sets of people (in this case two age bands with > 3000 obs each): Min. 1st…
Chris
  • 575
  • 1
  • 5
  • 13
24
votes
4 answers

Is there an equivalent to Kruskal Wallis one-way test for a two-way model?

If the model does not satisfy ANOVA assumptions (normality in particular), if one-way, Kruskal-Wallis non-parametric test is recommended. But, what if you have multiple factors?
user4267
  • 301
  • 1
  • 2
  • 5
19
votes
1 answer

Difference between Randomization test and Permutation test

In the literature the terms Randomization and Permutation are used interchangeably. With many authors stating "Permutation (aka randomization) tests", or vice versa. At best I believe the difference is subtle, and it lies in their assumptions about…
user10961
18
votes
1 answer

What is the intuition behind exchangeable samples under the null hypothesis?

Permutation tests (also called a randomization test, re-randomization test, or an exact test) are very useful and come in handy when the assumption of normal distribution required by for instance, t-test is not met and when transformation of the…
doctorate
  • 1,045
  • 2
  • 14
  • 24
16
votes
2 answers

P-values equal to 0 in permutation test

I have two datasets and I would like to know if they are significantly different or not (this comes from "Two groups are significantly different? Test to use"). I decided to use a permutation test, doing the following in R: permutation.test <-…
user2886545
  • 261
  • 1
  • 2
  • 6
15
votes
5 answers

How to test an interaction effect with a non-parametric test (e.g. a permutation test)?

I have two categorical / nominal variables. Each of them can take only two distinct values (so, I have 4 combinations in total). Each combination of values comes with a set of numerical values. So, I have 4 sets of numbers. To make it more concrete,…
Roman
  • 1,013
  • 2
  • 23
  • 38
14
votes
1 answer

Is this method of resampling time-series known in the literature? Does it have a name?

I was recently looking for ways to resample time series, in ways that Approximately preserve the auto-correlation of long memory processes. Preserve the domain of the observations (for instance a resampled times series of integers is still a times…
gui11aume
  • 13,383
  • 2
  • 44
  • 89
1
2 3
…
29 30