3

From what I'm reading, it seems that the best predictor, in the sense of minimizing $E(|X_{n+1}-l|^2)$ where $l$ is an $\mathcal{U}_0$-measurable function, is the projection of $X_{n+1}$ on $\mathcal{U}_0$. And this best predictor is always the conditional expectation $E(X_{n+1}|\mathcal{U}_0)$.

  1. How can we tell that the best predictor will be linear or not? Does it depend on the generated $\sigma$-algebra on which we condition?

  2. If $\mathcal{U}_0=\sigma(X_n,..., X_1)$, would it be necessarily linear? What about if $\mathcal{U}_0=\sigma(X_n^2,..., X_1^2)$, would it be necessarily non-linear?

Any help would be appreciated.

P.S.: I've also put it here. Not sure which one was best suited.

Community
  • 1
An old man in the sea.
  • 5,070
  • 3
  • 23
  • 57

1 Answers1

1

I understand the question as asking :

Are there universal or partial conditions/results that if true we know that the conditional expectation (which is the best predictor in mean-squared error sense) will be a linear function of the conditioning variables?

Then, apart from the well known Multivariate Normal Distribution case (where the condition is that all variables involved follow jointly a mutlivariate normal), we find some results in Johnson, N. L., Kotz, S., & Balakrishnan, N. (2002). Continuous multivariate distributions, volume 1, models and applications (2nd ed) . p.6-10.

Specifically in p. 9, there is a table(44.1) of bivariate distributions whose resulting conditional expectation (of the one variable on the other) is linear in the conditioning variable. All these bivariate distributions belong to the Pearson system (it requires some familiarity with the Pearson System to understand which "named" distributions are included in the table). The authors say that most of these are treated in detail later in their book.

Also, at the end of p.8 the authors mention the work of Steyn, H. S. (1960, January). On regression properties of multivariate probability functions of Pearson’s types. In Indagationes Mathematicae (Proceedings) (Vol. 63, pp. 302-311). who showed that for multivariate (not just bivariate) (Pearson) distributions, if the joint density vanishes at the extremes of the support of the conditioned variable, then the conditional expectation will also be linear (in all the other variables).

What I take as a lesson from the above, is that general results seem to exist only for mutlivariate distributions that give the same marginals (same family possibly different parameters) to all variables.

Alecos Papadopoulos
  • 52,923
  • 5
  • 131
  • 241
  • Alecos, thanks for your answer. However, I'm not sure I understand it... We know marginal distributions do not to completely determine the joint distribution except in few cases like the normal, etc. What you're saying is that when they do, we may have linearity in the conditional expectation? – An old man in the sea. Nov 29 '16 at 08:53
  • @Anoldmaninthesea. No, what I am saying is that the linearity results appear to be linked to multivariate distributions comprised of rv's that have the same marginals. A normal a gamma and a uniform will have some multivariate/joint distribution, but we don't seem to have linearity results for such beasts. A note on terminology: maybe the term "multivariate" is reserved strictly for joint distributions of rv's having the same marginals, while for any other case we just say "joint". If this is the case then the last sentence of my answer is superfluous and possibly misleading. – Alecos Papadopoulos Nov 29 '16 at 09:23
  • @Anoldmaninthesea. By the way I recently answered an old question of yours, http://stats.stackexchange.com/q/178719/28746 . Since you usually comment on the answers, I wonder, was it helpful? – Alecos Papadopoulos Nov 29 '16 at 09:24
  • Alecos, Let's see if I understood correctly. If the joint distribution gives the same marginal dist.(does it need to belong to the same family, but in the multivariate setting?) for every variable, then we may have a linear conditional expect. – An old man in the sea. Nov 29 '16 at 11:11
  • @Anoldmaninthesea. Yes the linearity results I have found are only for this subset of multivariate distributions. – Alecos Papadopoulos Nov 29 '16 at 11:24
  • @AlecosPapadopoulos Just a thought: Can it be related with certain type of copulas? – Cagdas Ozgenc Nov 29 '16 at 12:23
  • @CagdasOzgenc This merits a question on its own – Alecos Papadopoulos Nov 29 '16 at 12:33