5

I want to fit a model to a number of observations, each of them being a k-dimensional binary vector $(x_1, x_2, ..., x_k)$ where $x_i \in \{0,1\}$.

Naturally I would like to fit a multivariate bernoulli distribution $\mathbf X = (X_1, X_2, ... , X_k)$ where each $X_i$ is a bernoulli variable. I realize that there are $2^k - 1$ parameters for such a multivariate bernoulli distribution but for simplicity's sake I would like to simplify it by parameterizing the distribution by mean and second order interaction (higher order interaction are ignored):

  1. Marginal expectation $E[\mathbf X] = (p_1, p_2, ..., p_k)$
  2. Pairwise correlation $\rho_{ij}, 1 \le i,j \le k $

I have read a number of papers on this distribution and found that sometimes it is called Ising Model. But I still need some major pointers/directions due my limited statistical background:

  1. How do I find an MLE of the parameters given my observations?
  2. If I have a partial observation $(x_1, x_2, ..., x_j), j < k$, how do I estimate the remaining values? In other words, is there a simple formula for the conditional expectation? $$ E[X_{j+1}, X_{j+2},..., X_k | X_1=x_1, X_2=x_2, ..., X_j=x_j], 1 < j < k $$
mtrbean
  • 161
  • 4
  • Just wanted to add that this is related to this question: [link](http://stats.stackexchange.com/questions/7089/probability-formula-for-a-multivariate-bernoulli-distribution) but it seems that there is no satisfactory answer to that question. – mtrbean Oct 20 '14 at 02:33
  • When I think about it, the mean $(p_1, p_2, ..., p_k)$ of course can be estimated by sample mean $(\bar{x_1}, \bar{x_2}, ..., \bar{x_k})$. And same for sample covariance $\hat{Cov}(\mathbf{X})$. – mtrbean Nov 06 '14 at 08:08
  • Also, using the facts that (1) the conditional distribution of the second partition given the first is also multivariate bernoulli distribution, (2) such a multivariate bernoulli distribution is defined by only first and second moments, I suspect that the conditional expectation can be derived in exactly [the same manner as multivariate normal distribution](http://stats.stackexchange.com/questions/30588/deriving-the-conditional-distributions-of-a-multivariate-normal-distribution). Thoughts here? – mtrbean Nov 06 '14 at 08:16

0 Answers0