In various places (see quotes below) it says that the likelihood is "proportional to a probablility". Which probability is it proportional to?
In the context of Bayes theorem, it is not proportional to the posterior probability, unless the prior is constant.
If we denote the likelihood as $P(B|A)$ viewed as a function of $A$ with $B$ fixed, this is also not proportional to $P(B|A)$ viewed as a function of $B$ with $A$ fixed. I give an example below.
If the likelihood is proportional to a normalized form of itself, then ok, but that seems like a very trivial statement!
That this question is related to another recent question, Likelihood is not "proportional to" a single probability density? but it is different. That other question disputes the quotes, asking how can likelihood be related to the probability $P(B|A)$ using a single constant of proportionality. This question accepts the quotes but asks for clarification to identify which probability is the one that is mentioned in the quotes.
This is self study, and of course I understand that one or more of my assertions in these questions must be wrong. It is just my way of explaining my incorrect understanding that leads to the question.
QUOTES https://alexanderetz.com/2015/04/15/understanding-bayes-a-look-at-the-likelihood/ "likelihood is proportional to a probability"
A similar statement is in the book Think Bayes: "likelihood doesn't need to compute a probability, it only has to compute something proportional to probability"
EXAMPLE: $P(B|A)$ as a likelihood is not proportional to $P(B|A)$ as a probability density
A simple case, a PMF that is parameterized only by a single parameter $M$. It assigns probabilities to integer values,
and the parameter translates the PMF on the integer axis.
Here is the PMF for the fixed parameter value M=5:
$$
\begin{align*}
& P(X=5|M=5) = .5 \\
& P(X=6|M=5) = .3 \\
& P(X=7|M=5) = .2 \\
\end{align*}
$$
(and other values zero).
And the PMF for parameter value M=6: $$ \begin{align*} & P(X=5|M=6) = 0 \\ & P(X=6|M=6) = .5 \\ & P(X=7|M=6) = .3 \\ & P(X=8|M=6) = .2 \\ \end{align*} $$ More generically, the PMF is like P(X=M)=0.5, P(X=M+1)=.3, P(X=M+2)=.2, zero otherwise.
Now consider the likelihood form of this, where the data is given as the fixed value X=5 and the parameter $M$ varies: $$ \begin{align*} & P(X=5|M=3) = .2 \\ & P(X=5|M=4) = .3 \\ & P(X=5|M=5) = .5 \\ \end{align*}] $$
So in the PMF case the shape is (.5,.3,.2), whereas in the likelihood case the shape is (.2,.3,.5). There is no single constant of proportionality that relates these.
