8

My notes define a loss function as the 'cost' incurred when the true value of $\theta$ is estimated by $\hat\theta$. What kind of cost is it talking about? monetary cost? or is it something related to errors?

kjetil b halvorsen
  • 63,378
  • 26
  • 142
  • 467
ankc
  • 799
  • 2
  • 8
  • 21
  • Loss *could* be monetary, but is far more general. You could think of it as something akin to utility, but it doesn't have to be actual utility, or even very much like it. It's just some specifies measure of the 'badness' of the outcome in some sense. – Glen_b Oct 19 '13 at 14:03

1 Answers1

12

A loss function is a mathematical representation of anything bad or at least undesirable: the point is that it is therefore something you want to minimise.

Calling a loss function a cost is in general just terminology designed to be simple and evocative. The intention is to appeal to your sense that cost is something you want to avoid and (specifically and crucially) that (other things being equal) you prefer a smaller cost to a larger cost.

It doesn't necessarily imply a cost in any monetary, financial, economic or business sense.

Simple examples of a loss function arise when we consider the difference between some true or correct value $\theta$ and an estimate $\hat\theta$, which you would like to be as small as possible. Possible ways of taking that further are to work with $(\theta - \hat\theta)^2$ or $|\theta - \hat\theta|$, which are both loss functions. In either case there is a minimum loss of 0 when $\hat\theta = \theta$.

Nick Cox
  • 48,377
  • 8
  • 110
  • 156
  • is it possible to have loss less than 0? – ankc Oct 19 '13 at 12:35
  • This is more a matter of convention than of logic. In practice, at least in my experience, loss functions are functions with zero or positive value. I don't think anything stops anyone calling something a loss function when that something might be negative. It's just like thinking that you want to minimise expenditure, but if you get some net income that is better yet than zero expenditure. – Nick Cox Oct 19 '13 at 12:45
  • 1
    @Tim not non-negative? The loss of loss functions that could be zero would be a real loss. – Nick Cox Mar 17 '15 at 11:49
  • @ankc Some definitions specify that loss function *has to be non-negative* to be the loss function (e.g. Robert, 2007, *The Bayesian Choice*) – Tim Mar 17 '15 at 11:51