It's common to see statements that data must be i.i.d. But if data in a time series are independent, aren't they just noise? At one point I thought, they're only referring to the errors, i.e. the errors must be independent. Why should the deterministic part be serially independent? I can see the necessity for information calculations. But, all in all it's confusing.
Can anyone tell me exactly what i.i.d. means? Writers don't kill themselves making it clear. Thank you.
We often assume that random variables of our interest are independent and identically distributed (i.i.d.). You may also be interested in more broader term exchangability (see also here).
What this means is that they are independent, so loosely speaking, knowing one value tells us nothing new about other value, and more formally if events $A$ and $B$ are independent, then
$$ \Pr(A \cap B) = \Pr(A) \, \Pr(B) $$
So probabilities of observing each of the events alone, provide us with full information about probability of observing those events jointly. They not influence or interact with each other.
By saying that they are identically distributed, we mean that they can be characterized by the same probability distribution. So they are "of the same kind" in terms of their probabilistic behavior.
But if data in a time series are independent, aren't they just noise?
Yes they are. For time series we are interested in the temporal dependence between the values, so we do not assume independence.
Why should the deterministic part be serially independent?
If it is deterministic, then it can't be independent.
