0

Let $X$ be a random variable on the space $\mathcal{X}$, and let $f, g$ be two well-behaved functions $: \mathcal{X} \to \mathbf{R}$ such that

\begin{align} \bar{f} = \mathbf{E} [ f(X) ] < \infty \\ \bar{g} = \mathbf{E} [ g(X) ] < \infty. \end{align}

I am interested in unbiased estimation of the following covariance-like quantity:

\begin{align} K = \mathbf{E} \left[ \left\{ f(X) - \bar{f} \right\} \cdot \left\{ g(X) - \bar{g} \right\}^2 \right], \end{align}

given iid samples $X_1, \ldots, X_N$. I would also like for the estimator to be efficiently-computable (ideally in time $\mathcal{O}(N)$).

I have computed that

\begin{align} K = \text{Cov} \left( f(X), g(X)^2\right) + 2 \left[ \left( \mathbf{E} f \right) \left( \mathbf{E} g \right)^2 - \left( \mathbf{E} fg \right) \left( \mathbf{E} g \right) \right] \end{align}

and this naturally suggests some unbiased estimators, using standard approaches for unbiased estimation of covariances, and tricks like

$$\mathbf{E} \left[ \sum_{1 \leqslant i < j \leqslant N}u(X_i) v(X_j) \right] = {N \choose 2} \cdot \mathbf{E} \left[ u(X) \right] \cdot \mathbf{E} \left[ v(X) \right]$$

to handle the products of expectations.

Using all of this, I can write down expressions for an estimator which I think will be sensible and efficiently computable, but there are a couple of shortcomings:

i) it's very messy, so I don't completely trust my calculations

ii) from the way I write down the estimator, it's not completely clear that I will get the same answer if I replace $(f, g)$ by $(f + c_1, g + c_2)$ for constants $(c_1, c_2)$, which is aesthetically a little bit frustrating. I would trust an estimator which does have this property slightly more (and possibly it could improve the complexity / numerical stability of the estimator).

My main question is mostly whether there exists a reference for such an estimator, so that I can check my solution against it. Of course, I'd also accept an answer which derives an estimator directly, but I'm hoping that there's a nice reference which can save us all the effort!

πr8
  • 1,286
  • 5
  • 22
  • by 3rd order moment do you mean skewness? if so, then "mixed skewness"? – develarist Oct 21 '20 at 13:48
  • 1
    @develarist No: by 3rd-order I mean that the dependence on the functions of interest $(f, g)$ is a 3rd-degree polynomial, and that the expansion involves products of up to 3 expectations. There are certainly similarities to skewness estimation here, but I believe that this is a distinct problem. If the title is inappropriate / uninformative / misleading, then I am happy to change it to something more suitable. – πr8 Oct 21 '20 at 13:51
  • 1
    The central moments of a bivariate random variable $(X_1,X_2)$ are written $$\mu_{jk}(X_1,X_2)=E[(X_1-\bar X_1)^j(X_2-\bar X_2)^k],$$ which extends the standard notation for univariate variables. You ask for an unbiased estimator of $\mu_{12}(X_1,X_2).$ From the [polarization identity](https://stats.stackexchange.com/a/267021/919) $$24\mu_{12}(X_1,X_2)=24\mu_{111}(X_1,X_2,X_2)=\mu_3(X_1+2X_2)-2\mu_3(X_1)+\mu_3(X_1-2X_2)$$ it follows that when you have an unbiased estimator $\hat\mu_3(Y)$ for any *univariate* variable $Y,$ you have a corresponding unbiased estimator for $\mu_{12}(X_1,X_2).$ – whuber Oct 21 '20 at 14:07
  • 1
    @whuber This seems like what I'm looking for - I guess I'll just look up a standard estimate for $\mu_3$ and then be on my way! – πr8 Oct 21 '20 at 14:23

0 Answers0