Statistical Analysis Stack Exchange
Statistical Analysis Stack Exchange

Questions tagged [optimal-stopping]

39 questions
19
votes
4 answers

How could one develop a stopping rule in a power analysis of two independent proportions?

I am a software developer working on A/B testing systems. I don't have a solid stats background but have been picking up knowledge over the past few months. A typical test scenario involves comparing two URLs on a website. A visitor visits…
jkndrkn
  • 653
  • 1
  • 4
  • 9
16
votes
3 answers

Optional stopping rules not in textbooks

Stopping rules affect the relationship between P-values and the error rates associated with decisions. A recent paper by Simmons et al. 2011 coins the term researcher degrees of freedom to describe a collection of behaviours that they consider to be…
Michael Lew
  • 10,995
  • 2
  • 29
  • 47
14
votes
2 answers

Why is it wrong to stop an A/B test before optimal sample size is reached?

I am in charge of presenting the results of A/B tests (run on website variations) at my company. We run the test for a month and then check the p-values at regular intervals until we reach significance (or abandon if significance is not reached…
sgk
  • 195
  • 1
  • 7
12
votes
0 answers

Understanding Sequential Probability Ratio Test (SPRT) Likelihood Ratio

I am a software developer looking to develop an alternative for the simple hypothesis testing scheme described here. In short, the test works as follows: Two URLs are compared for their ability to convert visitors. Discrete samples are captured.…
jkndrkn
  • 653
  • 1
  • 4
  • 9
11
votes
1 answer

The Fishing Problem

Suppose you want to go fishing at the nearby lake from 8AM-8PM. Due to overfishing, a law has been instated that says you may only catch one fish per day. When you catch a fish, you can choose to either keep it (and thus go home with that fish), or…
b2coutts
  • 219
  • 1
  • 2
7
votes
0 answers

bias of an estimator when using stopping rules

Consider the setting where $X_1,X_2,...$ are i.i.d. real-valued random variables with $\mathbb{E}[X_i] = \theta$ and let the random variable $\tau$ be an associated stopping time. I'm wondering what can be said about the bias of the estimator (of…
fairidox
  • 1,188
  • 5
  • 16
6
votes
1 answer

How are optional stopping rules based on e.g. sample confidence (width of confidence interval) biased?

Inspired by this: http://pss.sagepub.com/content/22/11/1359 In the context of open-ended data collection where the necessary sample size cannot be properly estimated, for the purpose of a frequentists test; I understand that a stopping condition…
jona
  • 1,694
  • 12
  • 21
5
votes
0 answers

Optimal Stopping for Bernoulli One-Armed Bandit with a Fixed, Known Payout

I'm very new to bandit problems (apologies if I've formatted my question incorrectly), but I have to solve the optimal stopping of what I think is a very simple case. Suppose I have two arms $k = {1, 2}$. At each step, arm one gives a known, fixed…
Matt
  • 51
  • 1
4
votes
2 answers

Optimal stopping from an unknown distribution

The Secretary problem has an algorithm for fixed N and immediate accept/reject (that is, reject reject ... accept one, stop). There are several variants; in mine, secretaries or samples come from a real-valued source Xj, payoff is from best-so-far…
denis
  • 3,187
  • 20
  • 34
4
votes
1 answer

Problem of accepting a prize versus trying to get a better one

I don't know how to better formulate the general problem I am thinking about, let me try formulate an example. Assume you are playing a game with N rounds, and at each round the following happens: the gamemaster randomly draws a prize in the…
Bogdan Alexandru
  • 275
  • 2
  • 8
4
votes
0 answers

"Approved" switch criterion for the Secretary problem

The secretary problem ( 1, 2, 3, 4, 5 ) optimal stopping, says "stop and keep the best" in a randomized sequence of known length "k" where you can't select previously elements of the sequence. One stopping criteria says "go through 37% of the total…
EngrStudent
  • 8,232
  • 2
  • 29
  • 82
4
votes
1 answer

Prove $Z_n = X_n1_{n \le T} + Y_n1_{n+1\ge T}$ is a martingale

Given a filtered probability space $(\Omega, \mathscr F, \{\mathscr F_n\}, \mathbb P)$, let $X = (X_n)_{n \in \mathbb N}$ and $Y = (Y_n)_{n \in \mathbb N}$ be $(\{\mathscr F_n\}, \mathbb P)-$martingales and $T$ be a $\{\mathscr F_n\}$-stopping time…
BCLC
  • 2,166
  • 2
  • 22
  • 47
4
votes
0 answers

Stopping rule for chi-squared discretization algorithm

I developed an algorithm that uses the chi-squared test to perform supervised discretization of a continuous variable. I described it in the paper "ChiD-A Chi-Squared Discretization Algorithm" published in the WUSS 2011 Proceedings available at …
Ross Bettinger
  • 41
  • 1
3
votes
1 answer

When to stop enumerating a fixed set of unknown cardinality via random sampling?

DNS resolution can sometimes return one of multiple IP addresses, for load balancing. I would like to enumerate a list of IPs for a service so I can whitelist traffic to a domain without performing an excessive amount of reverse lookups. How many…
Iiridayn
  • 141
  • 6
3
votes
1 answer

Thoughts on model self-penalization amidst difficult parameter estimation

It is well accepted that one should account for model complexity when performing model comparisons, and the general procedure is to penalize more complex models more strongly. While this makes sense when the parameters of a given model are easily…
Mike Lawrence
  • 12,691
  • 8
  • 40
  • 65
1
2 3