1

I am reading about point estimation from Theory of Point Estimation by Lehmann and Casella (1999). I couldn't understand the following point mentioned in p.24, under the exponential family of distributions :

An exponential family of distributions of dimension $s$ (in canonical form) is the set of distributions $p_\theta$ such that. $p_\theta = h(x) . \exp[\sum_{i=1}^s \{\eta_iT_i(x) - A(\eta)\}]$.

It is mentioned that if the $\{T_i(x)\} $'s are linearly dependent (i.e one of $T_i(x)$ can be written as a linear combination of others) then the number of parameters of $\eta_i$ can be reduced (from $s$ to $s-1$). I understand why this is so.

What I don't understand is the following:

"if the $\{T_i(x)\}$ are linearly dependent, then the corresponding parameters $\{\eta_i\}$ cannot be estimated individually."

Xi'an
  • 90,397
  • 9
  • 157
  • 575

1 Answers1

2

When the $T_i$'s are linearly dependent, the model is not identifiable, that is, there exist several parameters $\eta=(\eta_1,\ldots,\eta_s)$ producing the same density. Indeed, if there exists a vector $\alpha=(\alpha_1,\ldots,\alpha_s)\ne(0,\ldots,0)$ such that $$\sum_{i=1} \alpha_i T_i(\cdot) \equiv 0$$ then $$\sum_{i=1} \eta_i T_i(\cdot) \equiv \sum_{i=1} \{\alpha_i+\eta_i\} T_i(\cdot)$$ which means that $\eta$ and $\eta+\alpha$ lead to the same density function.

Xi'an
  • 90,397
  • 9
  • 157
  • 575
  • Yeah. I get that the model is unidentifiable but I don't understand why if distributions are the same for different parameter values, the parameters cannot be estimated. Can you provide some intuition for that? –  Dec 12 '20 at 16:33
  • I am afraid my intuition stops at the fact that if several values of the parameter $\eta$ returns the same distribution/density, I cannot hope to estimate this parameter consistently. – Xi'an Dec 12 '20 at 16:44