Statistical Analysis Stack Exchange
Statistical Analysis Stack Exchange

Questions tagged [sequential-analysis]

Hypothesis testing without a fixed sample size.

... data are evaluated as they are collected, and further sampling is stopped in accordance with a pre-defined stopping rule as soon as significant results are observed.

--http://en.wikipedia.org/wiki/Sequential_analysis

115 questions
19
votes
4 answers

How could one develop a stopping rule in a power analysis of two independent proportions?

I am a software developer working on A/B testing systems. I don't have a solid stats background but have been picking up knowledge over the past few months. A typical test scenario involves comparing two URLs on a website. A visitor visits…
jkndrkn
  • 653
  • 1
  • 4
  • 9
13
votes
3 answers

Is dynamically increasing the sample size okay, if stated a priori?

I'm about to do a study about the merits of one stimulus compared to another with a within-subject design. I have a permutation scheme that is designed to reduce order effects of some parts of the study (task type order, stimulus order, task set…
xmjx
  • 765
  • 5
  • 11
12
votes
0 answers

Understanding Sequential Probability Ratio Test (SPRT) Likelihood Ratio

I am a software developer looking to develop an alternative for the simple hypothesis testing scheme described here. In short, the test works as follows: Two URLs are compared for their ability to convert visitors. Discrete samples are captured.…
jkndrkn
  • 653
  • 1
  • 4
  • 9
12
votes
5 answers

Determining sample size before starting an experiment or run the experiment indefinitely?

I studied statistics years ago and have forgotten it all so these may seem like general conceptual questions than anything specific but here is my issue. I work for an e-commerce website as a UX Designer. We have an A/B testing framework that was…
Tech 75
  • 221
  • 1
  • 4
12
votes
2 answers

Adjusting the p-value for adaptive sequential analysis (for chi square test)?

I wish to know what statistical literature is relevant for the following problem, and maybe even an idea on how to solve it. Imagine the following problem: We have 4 possible treatments for some disease. In order to check which treatment is better,…
Tal Galili
  • 19,935
  • 32
  • 133
  • 195
12
votes
2 answers

Are bayesian methods inherently sequential?

Ie, to do sequential analysis (you don't know ahead of time exactly how much data you will collect) with frequentist methods requires special care; you can't just collect data until the p-value gets sufficiently small or a confidence interval…
alecbz
  • 515
  • 3
  • 12
10
votes
3 answers

High autocorrelation when taking the L-th order of difference of a sequence of independent random numbers

To explain this question in more detail, I'll first elaborate my approach: I simulated a sequence of independent random numbers $X = \{x_1,...,x_N\}$. I then take $L$ times the difference; i.e. I create the variables: $dX_{1} =…
JohnAndrews
  • 537
  • 1
  • 5
  • 15
10
votes
2 answers

Updating a Bayes factor

A Bayes factor is defined in Bayesian testing of hypothesis and Bayesian model selection by the ratio of two marginal likelihoods: given an iid sample $(x_1,\ldots,x_n)$ and respective sampling densities $f_1(x|\theta)$ and $f_2(x|\eta)$, with…
Xi'an
  • 90,397
  • 9
  • 157
  • 575
8
votes
1 answer

How to distinguish between two biased coins

I have two very biased coins: The first coin (C1) lands on heads 95% of the time. The second coin (C2) lands on heads only 1% of the time. One of the coins has been selected and I want to determine which coin that is. I can toss the coin as many…
WW.
  • 105
  • 6
7
votes
1 answer

Testing for Benford Law in real time

Suppose I have data of a certain quantity $X$, given by $x_1,...,x_n$. Now, I take the first digit $d_i$ of each quantity $x_i$, and I want to study the relationship between the empirical distribution of first digits…
Nate River
  • 171
  • 4
7
votes
1 answer

Explanation for the thresholds in the sequential probability ratio test

Recently I have learned about sequential analysis especially sequential probability ratio tests (after I have struggled a lot with cumulation of alpha - errrors). See also this question Sequential hypothesis testing in basic science. My question is:…
mlwida
  • 9,922
  • 2
  • 45
  • 74
7
votes
0 answers

bias of an estimator when using stopping rules

Consider the setting where $X_1,X_2,...$ are i.i.d. real-valued random variables with $\mathbb{E}[X_i] = \theta$ and let the random variable $\tau$ be an associated stopping time. I'm wondering what can be said about the bias of the estimator (of…
fairidox
  • 1,188
  • 5
  • 16
6
votes
1 answer

Speed up web a/b tests with sample size checkpoints

Before starting an a/b test with a control and one experimental route, I can calculate a required sample size based on conversion rate estimates for both routes. I can get a good estimate of the conversion rate of the control by looking at…
nates
  • 161
  • 4
6
votes
1 answer

Stopping rules in group sequential designs

I have read that in group sequential designs, having a maximum of $k$ stops, for the $j$-th stop $j < k$ there are 4 bounds chosen $a_j,b_j, c_j, d_j$ such that the three decisions are considered. A set of lecture notes supporting this concept can…
AdamO
  • 52,330
  • 5
  • 104
  • 209
5
votes
1 answer

Sequential testing of multiple probabilities (binomial variables)

Task: I have e.g. 1 control group and 7 experimental groups which are treated differently. All groups have a dependent binomial variable: 1 = success, 0 = no success. I want to conduct sequential tests, which version is "performing" better, …
Andreas
  • 51
  • 1
1
2 3 4 5 6 7 8