likelihood of machine learning model with continuous data must be zero?

Question

Unlike the discrete case, the probability of any particular point in a continuous probability is zero. We must integrate over a small range of the pdf to bring a non-zero value.

In a machine learning model that assumes continuous data (as is oftend done for sound or images), the probability of any particular training data is zero.

Some models (such as variational inference models) are evaluated by their log likelihood.

My question: if the data is continuous, how can the likelihood be non-zero? The likelihood is assumed to factor over the data points, and the probability of each data point is zero, ...

With continuous variables, likelihood is defined in terms of probability density functions. — Tim, Jun 13 '18 at 08:10
IN machine learning you will certainly have data that are granular to the level of the last significant figure. You do not need to deal with continuous models if you don't want to. — Michael Lew, Jun 13 '18 at 08:18

score 2 · Accepted Answer · answered Jun 13 '18 at 09:26

2

The probability p(X|theta) is indeed zero, but the likelihood function is the probability density. That is in general non-zero.

answered Jun 13 '18 at 09:26

Helene Hoegsbro Thygesen

361
1
7

Can you explain a little more? So you just evaluate the pdf and ignore the fact that it is an infinitely thin sample that integrates to zero? – matchingmoments Jun 14 '18 at 10:02
1

You can think of the observation as not being Y=y but rather something like (y-d) – Helene Hoegsbro Thygesen Jun 15 '18 at 04:31

likelihood of machine learning model with continuous data must be zero?

1 Answers1