For two stochastic variables x and y, the rule for independence is given as $$ p(x,y) = p(x) \, p(y) $$ On the left hand side of this equation is a 2D quantity, and on the right hand side are two 1D quantities. In order for this equality to work, the product must therefore have the form of an outer product. Say that $p(x)$ is a Gaussian probability density representing the voltage drop measured across a (defective) diode, so that $p(x)$ has units of inverse volts. Next say that $p(y)$ is an exponential probability density representing the amount of light (optical power) hitting the diode, and thus has units of inverse watts.
If we were to plot $p(x,y)$ using a computer, then we could sample the distribution at various values of $y$, and would find the Gaussian distribution $p(x)$ scaled in amplitude, reaching a maximum at $y=0$, and tailing off to zero amplitude as $y \to \infty$, but keeping the same overall position and width for all $y$.
So far all of this makes sense, though I have yet to find a textbook that explicitly treats the dimensionality issue. But what if we invert the equation, as many texts do. Then we have $$ p(x) = p(x,y) / p(y) $$ and the question is: how do we perform this division correctly? We are dividing a 2D quantity by a 1D quantity, and the most logical interpretation would seem to be to do exactly the inverse of an outer product: at each $x$-value, we divide the distribution by $p(y)$, which leaves us with a 2D distribution like $p(x) \cdot \mathbb{1}(y)$. Assuming that $p(y)$ is nowhere zero, we are left with a uniform value in $y$, but we still have a 2D quantity. To get rid of the extra dimension, and to make the equality work, we appear to need an integral over $y$. But $\int \mathbb{1} (y) dy$ is an improper integral, so that we would appear to need to add a $\delta$-function inside the integral.
But this seems absurdly over-reasoned given that this is such a basic point and that the dimensionality issues are treated in no textbook on probability theory that I have been able to find. But how else can one compute the above division, considering that $x$ and $y$ are not in the same space as one another?
