3

I'm trying to derive the variance for a control variate estimator, but I seem to be missing a term that allows me to end up with the covariance in the final answer.

Let $f(x)$ be my function and let $h(x)$ be my control variate with $x \sim p(x)$. If I define the surrogate function:

$\tilde{f}(x) = f(x) - \beta (h(x) - \mathbb{E}_x[h(x)])$

I'd like to derive the variance of an estimator of the surrogate function:

enter image description here

As you can see, the middle term should be $Cov(f, h)$. What mistake am I making?

Also, how are there not tags for "control-variates" or "variance-reduction"?

Rylan Schaeffer
  • 689
  • 3
  • 10
  • With respect to your last question, this is a statistics site, not a simulation / programming site! – jbowman Jan 10 '20 at 00:39
  • $\mathbb{E}[f(h-\mathbb{E}[h])] = \text{Cov}(f,h)$, so I'm not sure where you are seeing a mistake. – jbowman Jan 10 '20 at 00:44
  • @jbowman , I thought $$Cov(f, h) = \mathbb{E}[(f-\mathbb{E}[f])(h-\mathbb{E}[h])]$$. What happened to subtracting the expected value of $f$? – Rylan Schaeffer Jan 10 '20 at 01:25
  • Also, how are control variates and variance reduction not related to statistics? Topics like these are clearly related to estimators and estimator properties. – Rylan Schaeffer Jan 10 '20 at 01:26
  • Control variates are pretty much a special-purpose simulation tool, and as such are going to, typically, be discussed within the context of programming and simulation rather than in the context of data analysis or math. Your question appears on-topic to me, but the reason there isn't a control variate tag is likely that few people with control variate questions come here, instead going to stackoverflow or someplace else. – jbowman Jan 10 '20 at 01:47
  • I can't help but disagree. Minimum variance estimators are very much a subject in Statistics. Here are lecture notes from Stanford's statistics department https://statweb.stanford.edu/~owen/mc/Ch-var-basic.pdf – Rylan Schaeffer Jan 10 '20 at 02:15
  • ... and as your tags show, "minimum-variance" is a tag. But we are talking about why "control-variates" *isn't* a tag. If more people ask questions specifically about control variates *on this site*, then it probably will become one. – jbowman Jan 10 '20 at 02:57

1 Answers1

4

Your derivation is correct; the two expressions for $\text{Cov}(f,h)$ referred to in the comments are the same:

$$\mathbb{E}[f(h-\mathbb{E}h)] = \mathbb{E}[fh]-\mathbb{E}[f\mathbb{E}h] \\ = \mathbb{E}[fh]-\mathbb{E}f\mathbb{E}h$$

where the last line follows as $\mathbb{E}h$ is a constant, and the expectation of an r.v. times a constant is just the constant times the expectation.

$$\mathbb{E}[(f-\mathbb{E}f)(h-\mathbb{E}h)] = \mathbb{E}[fh] -\mathbb{E}[f\mathbb{E}h]-\mathbb{E}[h\mathbb{E}f]+\mathbb{E}f\mathbb{E}h \\ =\mathbb{E}[fh] -\mathbb{E}f\mathbb{E}h-\mathbb{E}h\mathbb{E}f+\mathbb{E}f\mathbb{E}h \\ =\mathbb{E}[fh]-\mathbb{E}f\mathbb{E}h$$

Surprising, but true!

jbowman
  • 31,550
  • 8
  • 54
  • 107