Change of metric for probability density vs for probability

Question

When one changes the variable in a probability density function, one must account for the jacobian to ensure the elementary probability is constant (eg Derivation of change of variables of a probability density function?): $$ p_Y(y) |dy| = p_X(x) |dx| \Rightarrow p_Y(y) = p_X(x) |\frac{dx}{dy}| $$ So for instance if $X \sim U(0,1)$ and $Y=X^2$, then $Y$ too will be between 0 and 1 with pdf: $$p_Y(y) = \frac{1}{2\sqrt{y}}$$

In the case of a discrete distribution, I don't think the jacobian appears since probabilities, and not probability densities, are in play. I would therefore write for discrete probabilities: $$ P_Y(y) = P_X(x)$$ So for instance if $K$ follows a Poisson distribution, $$P_K(k;\lambda)=\frac{e^{-\lambda}\lambda^k}{k!} \text{ for } k = 0, 1, 2, 3...$$ and if $N=K^2$, then: $$P_N(n;\lambda) = \frac{e^{-\lambda}\lambda^{\sqrt{n}}}{\sqrt{n}!} \text{ for } n = 0, 1, 4, 9...$$ Is my reasoning correct?

Answer 1

You are correct: for a transformation $T$ we have $$ P(T(X) = a) = P(X \in T^{-1}(\{a\})) = \sum_{x \in T^{-1}(a)} f_X(x). $$ If $T$ is invertible then $$ P(T(X)=a) = f_X(T^{-1}(a)) = (f_X\circ T^{-1})(a) $$ so we are indeed just plugging $T^{-1}(a)$ into the density of $X$.

---

As to why the continuous and discrete cases are different, in one sentence I'd say it's because of differences in how the Lebesgue and counting measures behave under transformations.

With discrete distributions we fundamentally measure the size of a set by just counting how many elements are involved, and then probabilities come from giving a weight to each element of this set. The size of a set is unchanged by a bijection so if $T$ is a bijection then the size of $A$ is the same as that of $T(A)$. More formally, if $c$ is the counting measure then $c = c \circ T^{-1}$, i.e. $c$ is unchanged by pushing forward with $T$.

In the continuous case this is no longer true because now we're measuring the size of a set with its length, not the cardinality, so when we push forward with a transformation we need to account for how it stretches space. That's where the determinant of the Jacobian comes in.