4

This question focuses on a specific aspect of this one: How to derive the likelihood function for binomial distribution for parameter estimation?

In my own derivation, I start with: $$f(x\mid p) = mC_x~p^x(1-p)^{m-x}$$

Ignoring $mC_x$, the likelihood function is then given by:

$$L(p) = \prod_{i=1}^np^{x_i}(1-p)^{m-x_i} = p^{\sum_1^n x_i}(1-p)^{\sum_1^n m-x_i} = p^{x}(1-p)^{nm-x}$$

However, in the question I referenced, they have this instead: $$\prod_{i=1}^np^{x_i}(1-p)^{1-x_i} = p^{\sum_1^n x_i}(1-p)^{\sum_1^n1-x_i} = p^{x}(1-p)^{n-x}$$

My question is, are both approaches correct? If so, why does the referenced question use $1$ in place of $m$ on the exponents?

Michael Hardy
  • 7,094
  • 1
  • 20
  • 38
Data
  • 474
  • 3
  • 11

3 Answers3

6

It looks as if you intended $X_1,\ldots,X_n \sim \operatorname{i{.}i{.}d{.}} \operatorname{Binomial}(m,p).$ Then you have $$ L(p) \propto \prod_{i=1}^n p^{x_i} (1-p)^{m-x_i} = p^{\sum_{i=1}^n x_i} (1-p)^{nm - \sum_{i=1}^n x_i} = p^x (1-p)^{nm-x}. $$ It appears that in the question that you "referenced" (I don't know what "referenced" means in this context, but that doesn't appear to matter.) one has $X_1,\ldots,X_n \sim \operatorname{i{.}i{.}d{.}} \operatorname{Bernoulli}(p)$, so that $X_1+\cdots+X_n \sim \operatorname{Binomial} (n,p).$ That leads to $$ L(p) \propto p^{\sum_{i=1}^n x_i} (1-p)^{n - \sum_{i=1}^n x_i} = p^x(1-p)^{n-x}. $$ Therefore both are right, but they're answers to different questions.

Michael Hardy
  • 7,094
  • 1
  • 20
  • 38
2

The question you are referencing is starting with a Bernoulli distribution. To be sure, $x$ only takes on 0 or 1 in that question. In your work, you are starting with a Binomial distribution. To be sure, your values of $x=0, 1, 2, ... m$.

Remember that the sum of $n$ independent $Bernoulli(p)$ variables is a $Binomial(n, p)$ distribution. This should account for the differences you are seeing.

Your derivation for the likelihood of a binomial is just fine, ignoring the $mCx$ term, but you shouldn't ignore it. You can treat it as ignorable for the purposes of calculating the likelihood function since the likelihood function is a function only of the parameter $p$ and $p$ does not show up in $mCx$.

StatsStudent
  • 10,205
  • 4
  • 37
  • 68
  • Are you using "To be sure, [...]" to mean "You can be sure of this because [...]"? (I ask because "To be sure, [...]" means something quite different in English, which makes your usage very confusing. If you didn't intend the idiomatic meaning, I recommend finding a different phrase to express what you meant.) – ruakh Dec 24 '18 at 04:07
  • @ruakh, indeed the intention was idiomatic. It is correctly used to mean "certainly, undoubtedly, admittedly." I will keep it. – StatsStudent Dec 26 '18 at 17:27
  • But "admittedly" makes no sense in this context. You must be misunderstanding the idiom. :-/ – ruakh Dec 26 '18 at 18:02
  • No, I have a perfect understanding. I'm using the idiomatic meaning of "certainly" or "undoubtedly" - not "admittedly." – StatsStudent Dec 26 '18 at 18:03
  • That's not a separate meaning. You can use "certainly" to mean "to be sure" (for example, "To be sure, not everyone does it; but most people do" can be rephrased as "Certainly, not everyone does it; but most people do"); but you can't use "to be sure" in all cases where you can use "certainly" (for example, "It's certainly possible; would you like us to do it now?" can't be rephrased as "To be sure, it's possible; would you like us to do it now?"). If the word "admittedly" doesn't work, then the phrase "to be sure" doesn't work, either. – ruakh Dec 26 '18 at 18:38
  • (Specifically: the issue is that "to be sure" indicates a *concession*; you use it when you're conceding a point that might seem to contradict yours, in order to give yourself the opportunity to address that point and show how your point survives.) See e.g. https://www.merriam-webster.com/dictionary/sure, which defines it as "it must be acknowledged" and "admittedly". – ruakh Dec 26 '18 at 18:40
  • There's no doubt that the phase has multiple meanings. To be sure, see Collins English Dictionary (https://www.collinsdictionary.com/us/dictionary/english/to-be-sure), which lists one meaning as "surely; certainly" and another as "without doubt; certainly." Yet another, shows your usage preferred usage: "it has to be acknowledged; admittedly." The usage you've selected is only one of many is use in standard English and British vernacular. Thanks for the suggestion though, but I'll keep it. I think any confusion of the phrase is now aptly documented here in these comments now. – StatsStudent Dec 26 '18 at 19:10
1

The PMF in the question, $f(x_i)=p^{x_i}(1-p)^{1-x_i}$ belongs to Bernoulli distribution, where $x_i$ is a binary variable. Yours is the PMF of Binomial distribution, and $x_i$'s are Binomial RVs ($n$ of them actually) with parameters $(m,p)$.

gunes
  • 49,700
  • 3
  • 39
  • 75