A technical question about the reparametrisation trick

Question

I was reading this post which enlightened me about the technicalities of the reparametrisation trick, but I only get the intuition of this equivalent transform and I'm not sure why it is true: $$_[x^2]=_[(+)^2]$$

The intuition is clear - we have introduced another random variable which produces our previous random variable but our $ \epsilon $ is sampled from a new random variable, which has different distribution.

The definitions are: $$ q_{\theta}(x) = \mathcal{N}(\theta,1) $$ $$ x = \theta + \epsilon $$ $$ \epsilon \sim \mathcal{N}(0,1) $$

My attempt: I think p in this context means $ p(\epsilon) = \mathcal{N}(0,1) $

So then the exercise is to prove:

$$ E_q [ x^2] = \int_{X} x^2 q_{\theta}(x) dx = \int_{\eta} (\theta + \epsilon)^2 p(\epsilon) d\epsilon $$

I assume it is possible then to argue that for a given $ x_0 = \theta + \epsilon_0 $ the equation will only agree if we make a shift, but for me this is still too intuitive. Could we express this more rigorously? (feel free to throw Measure Theory at me if needed)

Answer 1

This is merely a change of variable. The expectation operator uses the density of the random variable(s) inside the expression. However, if you just look from a calculus perspective: $$x=\theta+\epsilon\rightarrow dx=d\epsilon, \ \ \epsilon=x-\theta$$ $$q_\theta(x)=\frac{1}{\sqrt{2\pi}}\exp\left(\frac{-(x-\theta)^2}{2}\right)=\frac{1}{\sqrt{2\pi}}\exp\left(-\frac{\epsilon^2}{2}\right)=p(\epsilon)$$ And, the limits of $x$, i.e. $(-\infty,\infty)$ are shifted by $\theta$, but, since infinity, they're the same. Or, you could leave them as in your notation, $X\rightarrow\eta$. Then, it follows that $E_q[x^2]=E_p[(x+\theta)^2]$