1

For iid random variables $Y_1,\dots,Y_n$ from a random sample that are mutually independent with pdf $f(y|\theta)$, my book defines the likelihood as $L(y_1,\dots,y_n|\theta)= f(y_1,\dots,y_n|\theta) = f(y_1|\theta)\dots f(y_n|\theta)$. For a random sample of the form $(X_1, Y_1) \dots (X_n,Y_n)$ with conditional densities $f(y|x,\theta)$, I've seen other sources define the conditional likelihoods as $\Pi_{i=1}^{n} f(y_i|x_i,\theta)$.

Based on how the likelihood is defined in my book, I interpret the conditional likelihood as a joint probability distribution $f(y_1,\dots,y_n|\theta,x_1,\dots,x_n)$ (written as $L(\theta; y|x)$ in some sources). If this is correct, is requiring that $Y_1,\dots,Y_n$ be mutually independent sufficient to define the likelihood as $\Pi_{i=1}^{n} f(y_i|x_i, \theta)$? I thought for the product be valid, $f(y_i|x_i, \theta)$ must equal $f(y_i|x_1, \dots, x_n, \theta)$ which implies that $Y_i$ and $X_1,\dots,X_{i-1},X_{i+1},\dots,X_n$ are conditionally independent given $X_i$? i.e. that $f(y_i,x_1,\dots,x_{i-1},x_{i+1},x_n|x_i)=f(y_i|x_i)f(x_1,\dots,x_{i-1},x_{i+1},x_n|x_i)$.

I haven't been able to find a source that explicitly state that $Y_i$ and $X_1,\dots,X_{i-1},X_{i+1},\dots,X_n$ be conditionally independent given $X_i$; is this implied or am I misunderstanding something?

Yandle
  • 743
  • 2
  • 12

1 Answers1

1

Normally, iid assumption is for the pairs, not just outcome or feature RVs, i.e. $(x_1,y_1)...(x_n, y_n)$ are iid pairs in a dataset. This may not be explicitly stated in some sources. This assumption directly means $Y_i$ and $X_{-i}$, i.e. variables other than $X_i$, are conditionally independent given $X_i$.

Here is another topic which might be quite useful.

gunes
  • 49,700
  • 3
  • 39
  • 75
  • Based on the link, iid for pairs assumes $p(x_1,y_1,\dots,x_n,y_n)=f(x_1,y_1)\dots f(x_n,y_n)$ ($p$ and $f$ being joint densities)? If so, does this assumption mean that $Y_i$ and $X_{-i}$ are mutually independent and also that all the $X$'s are mutually independent from each other, which is stronger than just assuming $Y_i$ and $X_{-i}$ are conditionally independent given $X_i$ [per this answer](https://stats.stackexchange.com/a/399098/243601)? – Yandle Apr 05 '20 at 16:51