0

I need to optimize parameters of a Gaussian shaped function to best fit my data points using maximum likelihood.

I first make an initial estimate for the parameters (mu, sigma and peak) and apply a gradient descent (fmincon in Matlab) algorithm to optimize these parameters.

The cost function is the sum of the negative log values of probabilities of observing each data point given the model parameters and the model noise parameter. 

sum(-log( normpdf( Y - model( X, params) ,0,error_sigma) ));

where, Y: observations

normpdf : normal probability densitiy function.

model: a function that takes the x values and parameters to be optimized

params : parameters of the model

error_sigma : noise of the data

I optimize the model parameters and the noise parameter simultaneously to obtain highest -loglikelihood values.

I am puzzled by obtaining negative likelihood values. I would have estimated that -log of probabilities should always evaluate in positive values. However, the matlab function normpdf, especially for small spread values, returns probability values which are bigger than 1.

This is because the integral of the normpdf function does nearly never sums up to 1. Therefore I am thinking that I should be using the normal cumulative function for optimization, instead of normpdf in order to get valid loglikelihood values. However, the parameters never converges as good as the normpdf case, when I use normcdf.

I hope I am clear.

NoNameNo
  • 9
  • 2
  • 6
    You are confusing probability and probability density. Probability density can be greater than 1. The assumption that you should replace probability density by cumulative probability for maximum likelihood is quite fallacious. – Nick Cox Sep 05 '13 at 18:42
  • sure it is not the same thing. I could create a cost function using min(1-normcdf(x,parameters),normcdf(x,parameters)). this is monotonically related to the cost function normpdf(x,parameters). – NoNameNo Sep 05 '13 at 19:27
  • 2
    You should write in your question the objective function you try to maximize -now its exact form is not clear and so it is unlikely that we will be able to give a clear answer. – Alecos Papadopoulos Sep 05 '13 at 22:21
  • Thanks for adding details, but the reply is just as before. Probability densities are _not_ probabilities. A density has units the inverse or reciprocal of whatever measurement units you are using. Imagine a distribution that is uniform on [0, 0.5]: the density must be 2. This is confusing you as you are not seeing its implication for log-likelihood; nothing else appears to be wrong. – Nick Cox Sep 06 '13 at 09:23

0 Answers0