1

in this question Can stationary time series contain regulary cycles and periods with different fluctuations I was told that stationary time series do not have regular cycles and that having constant cycles implies that the expectation value is not constant. One of the properties of stationary time series is "Expected value is constant for every t".

Now I am wondering why this is the case. Let's have a look at this blue time series:enter image description here I personally would argue that it has a constant expected value which is just the mean in this case because on average this is the value of the time series. How can I show that this time series in fact does not have a constant expectation value. I do not have any further information about the time series but just the values themselves. How could I calculate the expectation value for every t and show that they are not constant for every t which would imply that this time series is not stationary.

I would like to know just from the time series if this time series is stationary and how I can calculate the expected value (because this is essential for saying whether the time series is stationary or not).

I'd appreciate every further comment and would be quite thankful for your help.

Reminder: As this question is really important for me and I am not entirely satisfied with the answers given so far, I will award a bounty for a more detailed answer. I'll be quite happy for your help.

PeterBe
  • 230
  • 3
  • 13
  • This probably boils down to whether you are after an unconditional or a conditional expectation. Conditional expectation would be specific to a given $t$. Unconditional expectation would average across $t$s. (*Long-term mean* might also be considered; it would be (related to) the unconditional expectation.) I am a bit unsure about the formalities of all this, so I am posting this as a comment rather than an answer. – Richard Hardy Nov 25 '20 at 10:42
  • Thanks Richard for your answer. According to my search a stationary time series has 3 properties: Expactation value is constant for every t; Variance is constant for every t; Autocovariance is constant for every t; I would like to show that they are violated by the blue line as in two posts I was told that a cyclic time series are not stationary – PeterBe Nov 26 '20 at 09:35
  • Are these things constant for every $t$ (e.g. $\mathbb{E}(y_t)=c_t \ \forall \ t$) or across $t$ (e.g. $\mathbb{E}(y_t)=c \ \forall \ t$)? If across $t$, it looks like this does not hold for your series. – Richard Hardy Nov 26 '20 at 09:47
  • Thanks Richard for your further answer and effort. I really appreciate it. I just have read the statement "Expactation value is constant for every t". I think it should be for every t. You mentioned "If across t, it looks like this does not hold for your series". How can you calculate that? – PeterBe Nov 26 '20 at 10:02
  • Unless the cycles observed in your series are not representative of the data generating process (DGP) (i.e. just a weird sample), you can model the DGP as containing cycles where $\mathbb{E}(y_t)$ varies with $t$. In the bottom of the cycle, $\mathbb{E}(y_t)$ is low, and at the top it is high. – Richard Hardy Nov 26 '20 at 10:57
  • Thanks Richard for your comment. Let's say the time series are the sales of a company and the x-axis ticks are January 2019, February, 2019, March 2019 etc.. How can I now calculate the expectation values for the different t? I only have one value for January 2019. What sense does it make to calculate an expectation value? For me it does not make sense to derive an expectation value just from one datapoint. – PeterBe Nov 26 '20 at 14:19
  • IMHO it will make more sense to calculate the expectation value of the whole time series with all datapoints. And here I would argue that the expected value is constant for a window of ts (the window should not be too small). Thus you could argue that the expected value is constant and thus the time series is stationary. But apparently the notion of stationarity (which I have honestely not fully comprehend so far) does not allow time series to have cycles (this was told to me by many others in Cross Validated). – PeterBe Nov 26 '20 at 14:21
  • Thanks Richard for your answer. Any comments to my last responses to your answer? I'd highly appreciate every further comment from you and would be thankful for further help. – PeterBe Nov 27 '20 at 08:28
  • Sorry, have been pretty busy the last 24+ hours... – Richard Hardy Nov 27 '20 at 11:44
  • Thanks Richard for the information. I can fully understand it if you are busy at the moment. Whenenver you have a little bit more time I would highly appreciate a further comment from you. – PeterBe Nov 27 '20 at 14:14
  • Hi Richard, do you have any further comments on this one? I'd be happy if you would not mind sharing your knowledge – PeterBe Dec 01 '20 at 07:55

2 Answers2

3

If you by a time series means a concrete series of numbers/observations attached to some time points, then it is neither stationary nor not stationary, neither does it have an expectation (or not!) Those concepts apply only to time series models, that is, some probability model for a time series.

So by just looking at your blue wavy curve, nobody can answer your question! Such a curve could be generated by a stationary process, but it could also be generated by some nonstationary process. In the comments some argue about strong/weak stationarity, but I think that is irrelevant here. Everything said here applies equally to both.

To advance from here, you could tell us a model, then we can answer ... but more realistically, you do not have a model yet, you have some observations from some phenomenon, an the question should really be if a stationary time series model for that process is reasonable. An answer to that depends on the process, the phenomenon, not just the numerical data.

  • a monthly time series of some economics type ... seasonal variation is often seen, sales of christmas trees, for instance, is cyclical, with maximum around the same time each year ... not stationary

  • some natural phenomena might have dynamically generated cycles (or quasi-cycles). An example could be predator-prey cycles

  • some stationary ar(2) models can generate cyclic behavior

To followup on the last bullet point, let us simulate some (stationary) ar(2) model with cyclic behavior. I am following the example used in this excellent discussion of difference between seasonality and a cyclic series (some of the comments to your post indicates that you really have seasonality, not cyclicity.) The simulation is done in R:

set.seed(7*11*13)# My public seed

n <- 250
ar.sim <- arima.sim(list(ar=c(1.147, -0.6)), n)

library(scorepeak)

peaks <- detect_localmaxima(ar.sim, 5)

plot(ar.sim, type="l" ) 
points(which(peaks) , ar.sim[peaks],  col="red", pch=18)

simulated series with local maxima

The red diamonds in the plot indicates the position of local maxima, and looking at the distance between them could be a way to study period/cycle length. But that is to subjective, so we look for some other approach.

The periodogram is the tool of choice here, trying to estimate which fraction of the total variance in the series comes from different cycle lengths:

ar.sim.spec <- spectrum(ar.sim, spans=5)

Usually this is plotted on the log scale, to show better the details also in the low frequency part of the plot. But here we are more interested in seeing where the main contributions to total variance is, and the a linear scale is more appropriate, since then area under the curve is directly proportional to proportion of variance:

plot(ar.sim.spec, log="no",
     main="Periodogram (linear scale) for ar.sim")

plot of periodogram on a linear scale

We can read the main contribution periodicities from the plot. Since period is inverse of frequency:

 1/c(0.16, 0.08)
[1]  6.25 12.50
kjetil b halvorsen
  • 63,378
  • 26
  • 142
  • 467
  • Thanks a lot kjetil for your answer. In 2 posts (https://stats.stackexchange.com/questions/491785/can-stationary-time-series-contain-regulary-cycles-and-periods-with-different-fl) and (https://stats.stackexchange.com/questions/496828/which-property-of-stationarity-is-violated-by-a-time-series-with-regular-cycles) I was told that time series with cycles can't be stationary and I was literally told that ""Contains cycles" is a special form of not having a constant expectation." – PeterBe Nov 26 '20 at 09:33
  • The drawn blue line clearly has cycles. So I would like to know how I can calculate the expectation value to show that this time series is not stationary as it has a constant expectation value. – PeterBe Nov 26 '20 at 09:33
  • 1
    Thanks a lot kjetil for your answer. Any comments to my last responses to your answer? I'd highly appreciate every further comment from you and would be thankful for further help. – PeterBe Nov 27 '20 at 08:28
  • Hi: you have to be careful with the definition of stationarity. weak ( wide sense ) stationarity is just unconditional mean and unconditional variance constant. these do hold for your series. but strong stationarity is that joint distribution is time invariant ( shift the data by some fixed time and joint distribution stays the same ). For a time series with cycles, this is not going to be true so strong stationarity doesn't hold. So the question of stationary, depends on which one is being referred to. – mlofton Nov 29 '20 at 05:16
  • @mlofton: Do you have are ference for your definition of weak stationarity? For me, that always meant time invariance, but only for the mean and variance. – kjetil b halvorsen Nov 29 '20 at 14:54
  • right. that's what I meant by unconditional mean and variance constant. the strong one requires the joint dist to be invariant which is stronger. apologies if I wasn't clear. – mlofton Nov 29 '20 at 15:32
  • Of course, in the case of the gaussian, the mean and variance define the joint so, in the gaussian case, weak implies strong. – mlofton Nov 29 '20 at 20:29
  • Hello mlofton and kjetil. Thanks for your answers. mlofton are you saying that the blue line is in fact (weakly) stationary? In the 2 posts mentioned above it is clearly said that such lines are not stationary. So can you give me a link to a website with a defintion of both? Further how can I 'proove' or calculate that the weak stationaryity properties hold "unconditional mean and unconditional variance constant" – PeterBe Nov 30 '20 at 17:06
  • Read carefully. You did not tellus what the blue line represents, and that matters! It could be cycles, and it could be seasonality. Those are different! – kjetil b halvorsen Nov 30 '20 at 17:08
  • Moreover I do not understand the definition of strong stationarity that mlofton gave with "joint distribution is time invariant ( shift the data by some fixed time and joint distribution stays the same )". Why does this not hold for my blue time series and can you show me a time series for which this holds? The distribution of a time series changes everytime so I think it is extremely unlikely to find a time series for any given distance d the joint distribution stays the same – PeterBe Nov 30 '20 at 17:09
  • 1
    thank kjetil for your comment and effort. I really appreciate it. Well basically for me it did not matter what the blue line represents. I thought that the definition of stationarity should be independent from the application. But let's say these are the monthly sales of a company or the price of a stock. I would like to know just from the time series if this time series is stationary and how I can calculate the expected value (because this is essential for saying whether the time series is stationary or not) – PeterBe Nov 30 '20 at 17:13
  • Any further comments on this? I would highly appreciate your help because for me this topic is extremely confusing and I am struggeling strongly on this one. – PeterBe Dec 01 '20 at 17:58
2

I would like to know just from the time series if this time series is stationary and how I can calculate the expected value

The "just from the time series" is not working. You need to have some idea about the underlying process that generates the time series.

For instance, the image below shows a time series that is seemingly trend stationary for $t<0$ but differs for $t>0$.

weird example

To know whether a time series is stationary requires not just the data, but also some assumptions about a theory for the process that generates the time series.

Then you can test whether

  • The data fulfills the properties for stationarity (e.g. use some test to find out whether the process has a unit roots or not)
  • Estimate other parameters of the model (for which there are many different approaches and it depends on your type of time series)
  • Predict future values (extrapolate the model along with estimates of the parameters).

Even when the data is not stationary you can still fit a model to it. But it has some problems for some of the fitting methods.

Sextus Empiricus
  • 43,080
  • 1
  • 72
  • 161
  • Thanks Sextus Empiricus for your answer and help (I upvoted your answer). I understand what you say. However, in 2 posts (https://stats.stackexchange.com/questions/491785/can-stationary-time-series-contain-regulary-cycles-and-periods-with-different-fl and (https://stats.stackexchange.com/questions/496828/which-property-of-stationarity-is-violated-by-a-time-series-with-regular-cycles) I was told that time series with cycles can't be stationary and I was literally told that "Contains cycles" is a special form of not having a constant expectation." What is your take on that? – PeterBe Dec 08 '20 at 09:14
  • So if we need further informationen about the time series to decide whether it is stationary or not, then I think you can't immediately rule out just from the data that a time series that contains cycles is not stationary. In fact I would even argue that the blue time series tends to be stationary as it seem so have a constant expectation value when you just look at the data. But of course we can't know this for sure because we do not have any model. – PeterBe Dec 08 '20 at 09:17
  • @PeterBe I generated the time series in this example as a sinus wave plus Gaussian noise. It is not *strictly* stationary because the expectation is not independent from the time $t$. $$X_t\vert t \sim N(1+\sin(a\cdot t), \sigma^2)$$ However if you subtract the sinus wave component from the time series then what remains is Gaussian noise that is independent of time. $$Y_t = X_t - (1+\sin(a\cdot t))\vert t \sim N(0, \sigma^2)$$ So it is *trend* stationary. – Sextus Empiricus Dec 08 '20 at 10:58
  • Thanks Sextus for your answer. I really appreciate your help. Okay, so it is wrong to say that time series with cycles are generally not stationary. Thanks for this conclusion. Regarding the expectation value I would argue that your plotted time series has a constant expectation value of 1. You can just take the mean and I think that on average the time series produces the value of 1 for all t. – PeterBe Dec 08 '20 at 11:18
  • @PeterBe whether or not time series with cycles are stationary depends on what sort of stationarity you consider. It is not generally stationary because it is not stationary when you use the strict sense of stationarity. Regarding the expectation value, this is *not* constant if you have cycles. Saying that it is constant would be like saying that the expected outdoor temperature is the same in summer as in winter. – Sextus Empiricus Dec 08 '20 at 11:45
  • Thanks a lot Sextus for your answer and help. Basically I was referring to my blue time series and partially also to your posted time series. Here I would argue that the expectation value is constant and is equal to the mean (only for the plotted times). A statistical process generating those time series should have a constant expectation value because on average it generates data with the mean of the plotted ones. In the posted blue time series the variance is also constant for every t but of course in your time series the variance is not constant. – PeterBe Dec 08 '20 at 13:21
  • So if you have regular cycles without any trend (as my blue time series) I think that the expectation value should be constant for the whole time series. Of course if you have a trend in combinaiton with cycles or seasonality the expectation value is not constant for every t because as you pointed out in summer and winter the expected outdoor temperature is not constant – PeterBe Dec 08 '20 at 13:24
  • @PeterBe maybe you need to explain what you mean by (constant/regular) cycles. Why are your cycles different from the yearly cycle of seasons? – Sextus Empiricus Dec 08 '20 at 18:54
  • Thanks Sextus for your answer. By cycles I just mean regular patters as you can clearly see in my blue time series. Here I would argue that the expectation value is constant for this time series (and also the variance is constant) for every t. Given this I would deduce that this time series is in fact stationary (opposed to the opinion of others from other threads). – PeterBe Dec 09 '20 at 08:45
  • @PeterBe How is this different from expected outdoor temperature which is *not* constant throughout the year and higher in summer than in winter? – Sextus Empiricus Dec 09 '20 at 08:58
  • Thanks Sextus for your answer. Okay, you are saying that constant expectation value for a time series means that it should have a constant value for every t? Would not only a flat line fulfill that property? Because whenever you have variations the expectation value can't be constant for every t. You only have one measurement for every t (in case of a univariate time series) so you can't really calculate the expectation value. This would mean that stationary time series have to have constant values for every t (=horizontal line) because otherwise the expectation won't be constant for every t – PeterBe Dec 09 '20 at 09:06
  • I think I have difficulties understanding what constant expectation and constant variance for every t means when it comes to time series. – PeterBe Dec 09 '20 at 09:08
  • @PeterBe, the expectation value does not apply to a single realization of a process. [Expectation values](https://en.m.wikipedia.org/wiki/Expected_value) relate to the mean of a *random* process. The observed time series is only one example of an outcome of the process and we can use it to estimate parameters in *some* model. This model might include an expected value that is not independent from time. – Sextus Empiricus Dec 09 '20 at 09:57
  • A time series with random fluctuations can be stationary because the expectation value of the underlying probability distribution that describes the fluctuations is constant (so constant expectation will still allows random fluctuations). But if those fluctuations are systematically following some function of time then the distribution that governs those fluctuations is *not* constant in time. – Sextus Empiricus Dec 09 '20 at 09:58
  • If you roll repeatedly a fair dice then on average the value will be $(1+2+3+4+5+6)/6 = 3.5$. You have fluctuations of the outcomes but the mean is a constant function of time. On the other hand, if instead of rolling a random dice you repeatedly write the numbers 1,2,3,4,5,6,6,5,4,3,2,1 then you have probability distribution for the outcomes that is not independent of time. – Sextus Empiricus Dec 09 '20 at 10:01
  • Thanks for your answers and effort Sextus. I really appreciate it. So in your example time series can you argue that for t<0 the fluctuations are regular and thus the time series does not have a constant expectation value for every t and is thus not stationary while for the t>0 the fluctuations are not regular but seem to have the same mean. This might mean that the expectation value is constant for every t and thus the time series is stationary? – PeterBe Dec 09 '20 at 10:32
  • So can we deduce that whenever we have regular cycles the expectation value is not constant for every t and the time seris can thus not be stationary? – PeterBe Dec 09 '20 at 10:36
  • But in your example time seris for t>0 the variance is surely not constant for every t. Thus also for t>0 the time series can't be stationary as it violates the properties of stationarity. Is that correct? – PeterBe Dec 09 '20 at 10:38
  • @PeterBe For $t<0$. I generated the time series according to a normal distribution with a varying mean so the expectation value is *not* constant in time. For $t>0$ I generated the time series according to a normal distribution with increasing variance so it is neither strictly stationary. Strict stationarity requires that the *entire* probability distribution (not just the expectation) is independent of time. – Sextus Empiricus Dec 09 '20 at 12:38
  • You have many less strict definitions of stationarity. In my example for $t<0$ you have trend stationarity. And for $t>0$ you have a constant mean but not a constant variance (this is sometimes called first-order stationarity, although note that [this term can be ambiguous](https://stats.stackexchange.com/a/65359) and is used for other types as well). (and in addition to my last comment 'strict' stationarity requires that the entire probability distribution is independent of time, that relates *also to the joint distribution* of subsequences $x_k,x_{k+1}, \dots x_{k+n}$ for any length $n$) – Sextus Empiricus Dec 09 '20 at 12:43
  • Thanks Sextus for your comment and help. I really appreciate it. Do you have a link to a webseite that (correctly) shows stationary and non-stationary time series and the difference. For me the notion of stationarity is still confusing. – PeterBe Dec 09 '20 at 15:50
  • I accepted your answer and awarded the bounty to you for your tremendous help – PeterBe Dec 09 '20 at 15:51
  • Do you have a link to a website that (correctly) shows stationary and non-stationary time series and the difference. For me the notion of stationarity is still confusing. – PeterBe Dec 11 '20 at 10:29
  • @PeterBe I believe that you are overthinking it. A strictly stationary process is a process for which the probability distribution remains constant in time (and you have various less strict definitions requiring not the entire probability distribution to be constant in time). You might be thinking of something like a stationary 'process', as in a process/mechanism governed by some equations that do not change in time. – Sextus Empiricus Dec 11 '20 at 11:09
  • Thanks Sextus for your further comment. I really appreciate it. Bascially the notion of stationarity is quite important for time series. I have to admit that I am still a little confused about when a time series is stationary and when not (and even at Cross Validated I have received some contrary answers). This is why I wanted to know whether you know a website that explains this in a comprehensive way. I read more than 10 websites (including Wikipedia) but there it is not explained in a good way and information are sometimes in contrast to each other. – PeterBe Dec 11 '20 at 13:22
  • This suprises me a lot, because stationarity seems to be quite imoportant but everyone seems to have different priciples for stationarity. – PeterBe Dec 11 '20 at 13:22