4

In my recent question an answer was given, and I am able to compute it myself. Still, I'd like to understand where does that answer come from. Hence, what's the approach to handle dependent variables in order to get a PDF? Specifically, $X,Y$ are iid uniform[0,1] RVs, and $W$'s PDF is $f_W(w)=-\frac{1}{4}\ln\frac{w}{4}$, $w\in(0,1]$. I want (preferably, using pen and paper) to calculate the PDF of $Z=X+Y+\sqrt{(X-Y)^2+W}$. The PDFs of $A=X+Y$ and $B=\sqrt{(X-Y)^2+W}$ (separately) are known. How do I find the PDF of $Z$?

As the answer obtained via Mathematica/mathStatica is (piecewise) given by elementary functions, I'd expect $Z$'s PDF to be possible to calculate analytically (the software does it somehow, so a human should be able to do it as well). I browsed the web in search for theory of dependent RVs or anything related, but couldn't find anything useful (i.e., anything I'd be able to comprehend). Any help in providing hints, sketching the solution or specifically pointing to any relevant source (publication, textbook, etc.) will be appreciated.

EDIT Is it possible to find conditional probability $Pr(A|B)$ (or $Pr(B|A)$), given the (marginal) distributions of $A$ and $B$? Then the joint probability would be $Pr(A,B)=Pr(A|B)\cdot Pr(B)$. Or not?

This can be a general question if there exists a general answer, i.e., given $f_X(x)$, $f_Y(y)$ and $f_W(w)$, how to find $f_Z(z)$ when $Z=G(X,Y,g(X,Y,W))$.

If there's an approach that exploits particular forms of the functions $G$ and $g$ (from the first paragraphs of this question), I'd be happy to know it.

Community
  • 1
corey979
  • 1,135
  • 3
  • 17
  • 28
  • Are you asking a general question -- *how to compute the density of a transformation of dependent variables* (where the joint distribution is presumably completely specified)? You seem to waver between asking a general question and a specific one. – Glen_b Apr 06 '15 at 02:10
  • 1
    You do not _add_ pdfs, nor can you find the pdf of $A+B$ knowing only the (marginal) pdfs of $A$ and $B$: you need the _joint_ pdf or the credulity to believe that $A$ and $B$ are _independent_. For example, I would be very skeptical of _unsupported_ claims that $X+Y$ and $\sqrt{(X-Y)^2+W}$ are independent random variables but your imagination might be running wild... That being said, the pdf of $A+B$ is $$f_{A+B}(z)=\int_{-\infty}^\infty f_{A,B}(x,z-x)\,\mathrm dx=\int_{-\infty}^\infty f_{A,B}(z-y,y)\,\mathrm dy$$ which gives the _convolution_ formula when $A$ and $B$ are independent. – Dilip Sarwate Apr 06 '15 at 03:03
  • Gleb_b: in fact, both; I have no idea what to start with because my lack of knowledge in this field. To focus attention and further explain what I really want, I ask a specific question. Dilip Sarwate: they are not independent, which imho is visible straightforward, but I also made a simulation and the PDF obtained assuming independence is quite different from the correct one. The formula you give is general and known, but how do I get the joint PDF of $A$ and $B$? – corey979 Apr 06 '15 at 07:54
  • 3
    One cannot _get_ the joint pdf from the marginal pdfs alone; there is a standard method for computing the joint pdf of $W=g(X,Y)$ and $Z=h(X,Y)$ which involves partial derivatives and Jacobians etc., and it is, in many interesting cases, fairly tedious to apply. Symbolic computation packages such as Mathematica, Maple, and perhaps R and MathStatica often are used in such cases (see e.g. wolfie's answer to your other question. – Dilip Sarwate Apr 07 '15 at 23:11
  • Dilip Sarwate, I know that and know how to deal with this kind of simple situations. And the convolution formula from your previous comment is a tool I used extensively with previous calculations in this topic. I'm really curious how to get the answer in this particular situation with overt approach, not symbolic computations; as Mathematica can give an analytic answer, I strongly believe it's possible to obtain it without a computer. I don't only want to *know* the answer, I want to know also *where does it come from*. – corey979 Apr 07 '15 at 23:35
  • -1 You have been told what method is used, that the computations are often tedious, and so you _know_ _"where the answer comes from,_" and you _still_ want someone to spend their time writing out an answer to your specific problem for your edification, so much so that you are offering bounties to induce other people to write out a detailed answer? Why don't you put your money where your mouth is when you say "I strongly believe it's possible to obtain it without a computer" and write an answer to your own question? – Dilip Sarwate Apr 08 '15 at 23:12
  • This question, after edits, and the offering of a bounty is **unanswerable as asked.** The OP wants to know how to compute the density of a _function_ of _dependent_ random variables from knowledge _only_ of the _marginal_ densities of the variables, and without requiring _any_ knowledge of the _joint_ distribution. The last sentence of "I'd like to know a method that allows to find analytically a PDF of non-independent RVs, without knowledge on the joint distribution, but knowing PDFs of the components. Software give analytic answers, so it has to be possible to calculate manually." is false. – Dilip Sarwate Apr 08 '15 at 23:17
  • Dilip, I also I have a strong feeling I'm not understood. I really don't have the knowledge to grasp your ideas after a word or two. You said *there is a standard method for computing the joint pdf of $W=g(X,Y)$ and $Z=h(X,Y)$ which involves partial derivatives and Jacobians etc.*, i.e. $f_2(w,z)=f_1(g(x,y),w(x,y))|J|$. But I don't know $f_1$. All the information on the RVs ($X,Y,W$) is overt. After all your uptight comments I have a feeling I'm missing something obvious to you or can't put all the chunks together. Please clarify if I'm wrong (it seems like I'm missing sth like $2+2=4$). – corey979 Apr 09 '15 at 07:08
  • @corey979 There are _many_ joint distributions that can give rise to the same marginal distributions. See, for example, [this answer of cardinal's](http://stats.stackexchange.com/a/30205/6633) for ways that marginally normal random variables can fail to be _jointly_ normal. Computation of the _joint_ density of $W=g(X,Y)$ and $Z=h(X,Y)$ **requires** knowledge of the _joint_ density of $X$ and $Y$; knowledge of just the marginals, and nothing else, is _insufficient_ **even for Mathematica, et al.** Any "analytic" answers provided by software packages .... – Dilip Sarwate Apr 09 '15 at 13:09
  • .... necessarily make some **assumptions** about what the joint density of $X$ and $Y$ is, and proceed from there. Most likely, the packages assume that $X$ and $Y$ are _independent_ which is definitely not the situation in your specific case. You think that since software "gives analytic answers" it should be possible to replicate this on pencil and paper. Sure it can be done, but tell us what assumptions the software has made about the joint density. Note that even _Monte Carlo simulation_ to estimate the joint density of $W$ and $Z$ makes _assumptions_ about the joint density of $X$ and $Y$ – Dilip Sarwate Apr 09 '15 at 13:15
  • Ok, this may be a dumb question: why are we referring to marginals? I mean, let's step back and consider a simpler example of $(X-Y)^2$, where X, Y are iid & uniform on [0,1]. Transformation: $U_1=(X-Y)^2$ & $U_2=Y$ gives $X=\sqrt{U_1}+U_2,\,Y=U_2$, and now the joint $f(X,Y)$ is separable and one can proceed in a standard way. So, if I have A, D, U as in my question, the key is finding a transformation $(A,D,U)\rightarrow(V,W,Z)$ and then using the fact that A, D, U are independent, so their joint is separable. This is general; am I substantially wrong somewhere? – corey979 Apr 09 '15 at 14:16
  • I have a feeling I'm very near to pointing out exactly what I mean and want to do, but it keeps slipping out of my hands... Or I may be totally wrong in everything. – corey979 Apr 09 '15 at 14:22
  • Is $W$ independent of $(X,Y)$? – Daniel Weissman Apr 10 '15 at 23:11
  • @Daniel: yes, it is. – corey979 Apr 11 '15 at 12:59

1 Answers1

4

I specify my approach:

Let $X=A+D+\sqrt{(A-D)^2+U}$ with $Y=A+D$ and $Z=(A-D)^2$; hence $A=\frac{Y+\sqrt{Z}}{2}$, $D=\frac{Y-\sqrt{Z}}{2}$ and $U=(X-Y)^2-Z$, with a Jacobian $|J|=\left|\frac{y-x}{2\sqrt{z}}\right|$. Then, the PDF of $X$ is $$f_X(x)=-\frac{1}{4}\int\limits_{-\infty}^{\infty}\int\limits_{-\infty}^{\infty}f_A\left(\frac{y+\sqrt{z}}{2}\right)\cdot f_D\left(\frac{y-\sqrt{z}}{2}\right)\cdot\ln\frac{(x-y)^2-z}{4}\cdot\left|\frac{y-x}{2\sqrt{z}}\right|dydz,$$ where $f_A$ and $f_D$ are equal to 1 if their argument is in $[0,1]$ and equals 0 otherwise, and $-\frac{1}{4}\ln\frac{*}{4}$ is the PDF of $U$, with $*\in(0,4]$. The integration region is given by the inequalities $$0\leq y+\sqrt{z}\leq 2$$ $$0\leq y-\sqrt{z}\leq 2$$ $$0<(x-y)^2-z\leq 4$$ with $x\in[0,4]$, $y\in[0,2]$ and $z\in[0,1]$. The integration region is additionaly split by $x=y$, as at this surface the Jacobian changes its sign and in the formula for $f_X(x)$ we have its module.

Can someone verify whether my reasoning and the formula for $f_X(x)$ are correct? And the last sentence about splitting the integral by the Jacobian.

(Yes, calculating this integral seems to be a pain; but is it at least correct?)

corey979
  • 1,135
  • 3
  • 17
  • 28