Statistical Analysis Stack Exchange
Statistical Analysis Stack Exchange

Questions tagged [iid]

iid is an acronym for independent and identically distributed. Many statistical methods assume that the data are iid; that is, that each observation comes from the same distribution and is independent of other observations.

214 questions
89
votes
5 answers

On the importance of the i.i.d. assumption in statistical learning

In statistical learning, implicitly or explicitly, one always assumes that the training set $\mathcal{D} = \{ \bf {X}, \bf{y} \}$ is composed of $N$ input/response tuples $({\bf{X}}_i,y_i)$ that are independently drawn from the same joint…
Quantuple
  • 1,296
  • 1
  • 8
  • 20
72
votes
6 answers

What are i.i.d. random variables?

How would you go about explaining i.i.d (independent and identically distributed) to non-technical people?
user333
  • 6,621
  • 17
  • 44
  • 54
39
votes
3 answers

Brain-teaser: What is the expected length of an iid sequence that is monotonically increasing when drawn from a uniform [0,1] distribution?

This is an interview question for a quantitative analyst position, reported here. Suppose we are drawing from a uniform $[0,1]$ distribution and the draws are iid, what is the expected length of a monotonically increasing distribution? I.e., we…
Amazonian
  • 1,394
  • 1
  • 10
  • 19
25
votes
1 answer

Properties of PCA for dependent observations

We usually use PCA as a dimensionality reduction technique for data where cases are assumed to be i.i.d. Question: What are the typical nuances in applying PCA for dependent, non-i.i.d. data? What nice/useful properties of PCA that hold for i.i.d.…
Richard Hardy
  • 54,375
  • 10
  • 95
  • 219
22
votes
2 answers

Test for IID sampling

How would you test or check that sampling is IID (Independent and Identically Distributed)? Note that I do not mean Gaussian and Identically Distributed, just IID. And idea that comes to my mind is to repeatedly split the sample in two sub-samples…
gui11aume
  • 13,383
  • 2
  • 44
  • 89
21
votes
2 answers

Is there i.i.d. assumption on logistic regression?

Is there i.i.d. assumption on the response variable of logistic regression? For example, suppose we have $1000$ data points. It seems the response $Y_i$ is coming from a Bernoulli distribution with $p_i=\text{logit}^{-1}(\beta_0+\beta_1 x_i)$.…
Haitao Du
  • 32,885
  • 17
  • 118
  • 213
20
votes
4 answers

Are "random sample" and "iid random variable" synonyms?

I have been facing hard time understanding meaning of "random sample" as well as "iid random variable". I tried to find out the meaning from several sources, but just got more and more confused. I am posting here what I tried and got to…
Silent
  • 459
  • 1
  • 4
  • 9
18
votes
3 answers

Realistically, does the i.i.d. assumption hold for the vast majority of supervised learning tasks?

The i.i.d. assumption states: We are given a data set, $\{(x_i,y_i)\}_{i = 1, \ldots, n}$, each data $(x_i,y_i)$ is generated in an independent and identically distributed fashion. To me, physically this means that we can imagine that the…
Olórin
  • 674
  • 5
  • 16
15
votes
1 answer

How does one show that there is no unbiased estimator of $\lambda^{-1}$ for a Poisson distribution with mean $\lambda$?

Suppose that $ X_{0},X_{1},\ldots,X_{n} $ are i.i.d. random variables that follow the Poisson distribution with mean $ \lambda $. How can I prove that there is no unbiased estimator of the quantity $ \dfrac{1}{\lambda} $?
billlee1231
  • 251
  • 2
  • 4
14
votes
3 answers

Why is it valid to detrend time series with regression?

It may be a weird question at all but as a novice to the subject I am wondering why do we use regression to detrend a time series if one of the regression's assumption is the data should i.i.d. while the data on which regression is being applied is…
FarrukhJ
  • 161
  • 1
  • 8
11
votes
2 answers

Uniform PDF of the difference of two r.v

Is it possible to have the PDF of the difference of two iid r.v.'s look like a rectangle (instead of, say, the triangle we get if the r.v.'s are taken from the uniform distribution). i.e. is it possible for the PDF f of j-k (for two iid r.v.'s taken…
Nathan
  • 113
  • 4
10
votes
3 answers

Exchangeability and IID random variables

Every IID sequence of random variables is considered to be exchangeable, i understand why its necessary for the random variables to be identically distributed to assume exchangeability, but why the need for independence, (or is there a need)? In the…
s.g
  • 311
  • 3
  • 11
10
votes
1 answer

Expected value of iid random variables

I came across this derivation which I don't understand: If $X_1, X_2, ..., X_n$ are random samples of size n taken from a population of mean $\mu$ and variance $\sigma^2$, then $\bar{X} = (X_1 + X_2 + ... + X_n)/n$ $E(\bar{X}) = E(X_1 + X_2 + ... +…
RenamedUser7008
  • 103
  • 1
  • 1
  • 6
10
votes
4 answers

Suppose $X_1, X_2, \dotsc, X_n$ are i.i.d. random variables. When is the sequence expected to decrease for the first time?

As suggested in the title. Suppose $X_1, X_2, \dotsc, X_n$ are continuous i.i.d. random variables with pdf $f$. Consider the event that $X_1 \leq X_2 \dotsc \leq X_{N-1} > X_N$, $N \geq 2$, thus $N$ is when the sequence decreases for the first…
Hao The Cabbage
  • 183
  • 6
8
votes
2 answers

Why do copulas need the i.i.d assumption for marginal distribution?

Does anyone know if are there some assumptions for Copula method? I heard from someone that the data should be i.i.d (independent and identically distributed). Let's say, if I want to capture the dependence structure between two variables. I have to…
argan
  • 81
  • 4
1
2 3
…
14 15