Statistical Analysis Stack Exchange
Statistical Analysis Stack Exchange

Questions tagged [exact-test]

Statistical tests where the sampling distribution is necessarily exactly correct & not an approximation based on assumptions &/or large sample theory.

An exact test is a statistical tests where the sampling distribution is necessarily exactly correct instead of being an approximation based on assumptions and/or large sample theory.

43 questions
9
votes
3 answers

Test to distinguish periodic from almost periodic data

Suppose I have some unknown function $f$ with domain $ℝ$, which I know to fulfill some reasonable conditions like continuity. I know the exact values of $f$ (because the data comes from a simulation) at some equidistant sampling points $t_i=t_0 +…
Wrzlprmft
  • 2,225
  • 1
  • 18
  • 35
8
votes
2 answers

Paired vs independent t-test for A/B test with underlying trends

There is a metric which has a natural cyclic pattern. We want to measure the effect on this metric through a A/B test. Examples: Metric is daily revenue of ice-cream trucks which is low during weekdays and high on weekends; A/B test to check which…
BiGYaN
  • 183
  • 5
6
votes
2 answers

Drawing conclusions of several inferences with the same data in one study

In a sample size of 100, we identified the existence of two attributes A and B. Our goal is to assess whether there is any association between these two attributes. The data looks like following: A Present …
user81411
  • 731
  • 1
  • 7
  • 14
5
votes
1 answer

One-sided McNemar's test

In R, the function mcnemar.test has the following example: ## Agresti (1990), p. 350. ## Presidential Approval Ratings. ## Approval of the President's performance in office in two surveys, ## one month apart, for a random sample of 1600 voting-age…
thc
  • 388
  • 2
  • 16
5
votes
1 answer

z-test for one population proportion hypothesis with a small sample size

I have asked a "yes/no" question to a small group of people (n = 22). From the 22, 14 answered "yes" and 8 answered "no". I'd like to find out, with 90% confidence, if the majority of my population (> 50%) would answer "yes" to that question. I…
Charmander
  • 153
  • 5
5
votes
0 answers

Exact Null Distribution with Ties

I am interested in deriving exact null distributions for small-sample test statistics with non-trivial ties. Not fundamentally continuous variables that happen to have a few repeated values, but ordinal variables like grades (F to A) or Likert (very…
David Wright
  • 2,181
  • 12
  • 12
5
votes
1 answer

How do you know if your confidence interval is exact or approximate?

I would like to know the rules between the two, and know how you know the differences.
das
  • 51
  • 1
  • 2
4
votes
2 answers

Single sample t-test variance assumptions for non-normal data

So I understand that a key assumption behind the single sample t-test is that the sample mean is normally distributed. This is true when the data themselves are normally distributed, and it is approximately true when you have a large sample size of…
Zachary Miller
  • 41
  • 2
4
votes
1 answer

Exact distribution of number of transitions in a binary vector of length $n$

I am trying to derive an analytic solution for the exact distribution of the number of transitions in a binary vector of length $n$ conditional on the observed number of $1$'s and $0$'s and under the null model assumption that order of elements is…
sztal
  • 1,009
  • 1
  • 9
  • 14
4
votes
1 answer

Exact test for m x n contingency table conditional (i.e. fixed by design) on one margin

I have a $m \times n$ contingency table (with $m, n > 2)$ . The experiment yields ~15% cells with expected frequencies lower 5 and also zero counts in the empirical data. The prerequisite for the asymptotic $\chi^2$-Test do not hold. Thus, I want to…
Mark Heckmann
  • 261
  • 1
  • 11
4
votes
1 answer

Non-parametric, repeated measures two-sample test with small sample size

My data consists in two samples relative to the expression of a given gene, with 4 replications in both samples. I am interested in testing whether the two samples come from the same population, taking into account the fact that I have repeated…
Stefano Lombardi
  • 681
  • 9
  • 19
4
votes
1 answer

Proper analyses for 2x2 contingency tables

So I have multiple questions about the different types of analyses that can be used on a 2x2 contingency table and when different analyses should be used. To open, I'll note that this stems from my efforts to find the appropriate analyses to run…
Patrick Rich
  • 87
  • 6
4
votes
2 answers

Is the binomial sign test really the exact McNemar test?

I am calculating two different $p$ values for the McNemar test and what I believe is the corresponding exact test, even for combined counts larger than 25. Take, for example the contingency table Posttest Pretest Outcome A …
BMS
  • 155
  • 8
3
votes
1 answer

Confidence interval for quantiles: distribution-free, asymptotic and assuming a normal distribution

An acquaintance of mine has been using this wrong inference formula for years: given a i.i.d. sample $\mathbf{X}={X_1,\dots,X_N}$ for a continuous RV $X$, sample mean $\bar{X}=\frac{\sum X_i}{N}$ and sample standard deviation…
DeltaIV
  • 15,894
  • 4
  • 62
  • 104
3
votes
1 answer

Examples for exact tests which are not permutation tests

Wikipedia states (without citation): Sometimes permutation test is used as a synonym for exact test, but although all permutation tests are exact tests, not all exact tests are permutation tests. What are examples of exact tests which are not…
Arne Jonas Warnke
  • 3,085
  • 1
  • 22
  • 40
1
2 3