I am having trouble fully grasping the concept of stationarity in time series.
Here is what I have gathered so far.
A stochastic process is a collection of random variables
with mean $\mu$ and variance $\sigma^2$.A (A weakly) stationary stochastic process is one whose statistical properties do not change over time. For weak stationarity it is required that the mean and the variance of the process are constant over time and that the autocovariance between lagged values of the process only depends on the lag and not on the time t.
A time series is a realization of a stochastic process. It is a sequence of fixed numbers obtained from sampling the underlying stochastic process at discrete time instants.
Questions:
If we say weak stationarity requires the mean and the variance to be constant over time, doesn't that mean that the autocovariance of any pair of random variables should just be the variance of the time series, since $autocov(X_t ,X_t) = var(X_t)$? If that is the case, how can there be different values for the autocovariance for different lags?
What do we get when we calculate the mean and variance of all the values from one time series (meaning, take all the measured data points, add them up and divide them by the number of data points). How do those statistics relate to the mean and variance of the underlying stochastic process?
Regarding my third point: I keep coming across sentences to the extent of: " A time series is a collection of random variables." Can I just assume that people are misusing terminology or am I missing something?