5

When we have a statistical model $\{ P_\theta, \theta\in\Theta\}$ on some common probability space, usually we define "the" likelihood function $L(\theta)$ via a Radon-Nikodym derivative with respect to some common dominating measure, say $\mu$. For that to be possible it is necessary that all the $P_\theta$ are equivalent in the sense that they agree on which events have probability zero or one. Without that assumption there is no common dominating measure.

What can we do to get a likelihood function when this is not the case? For instance, in the nonparametric setting of Is Maximum Likelihood Estimation (MLE) a parametric approach? there is no common dominating measure. Ideas or references?

kjetil b halvorsen
  • 63,378
  • 26
  • 142
  • 467
  • 2
    Interesting question. Even in the non-parametric setting---take the classical fixed design non-parametric regression problem, in the large sample limit there are Radon Nikodym densities w.r.t. to a dominating measure (the Wiener measure), given by Girsanov's theorem. In this sense, "nonparametric" just means the parameter space for densities is infinite dimensional. – Michael Jul 20 '20 at 20:33
  • @Michael: I have to read up on Girsanov T then ... – kjetil b halvorsen Jul 20 '20 at 20:40

1 Answers1

3

(Not really an answer, simply an elaboration on the comment. Apologies in advance. It is a very interesting question. Hopefully the comment will be complementary to the answer(s) when they come along.)

It seems that, in order to make meaningful statistical statements, one needs densities/likelihood functions. Therefore a dominating measure necessarily shows up somewhere in the formulation, even in the non-parametric setting.

For example, take the classical fixed design non-parametric regression problem $$ y_t = f(t) + \epsilon_t, \;\;t = \frac{k}{n}, \; k = 0, \cdots, 1, $$ where $\epsilon_t \stackrel{i.i.d.}{\sim} (0, \sigma^2)$, and $f$ lies in, say, $C[0,1]$, the continuous functions on $[0,1]$.

The problem of estimating $f$ from $(y_t)$ is asymptotically equivalent to estimating drift $f$ from a sample path $Y_t$ of the stochastic process (Ito diffusion) $$ dY_t = f dt + \sigma dW_t $$ where $W_t$ is standard Brownian motion. In this formulation, the problem becomes estimating a element $f$ of an infinite dimensional "parameter space" $C[0,1]$.

Statistically speaking, $Y_t$ is a probability measure $\mathbb{Q}^f$ on the Skorohod space $D[0,1]$, with Radon-Nikodym density $$ \frac{d \mathbb{Q}^f}{ d \mathbb{P}} =e^{\int_0^1 \frac{f}{\sigma} dW_t - \frac{1}{2} \int_0^1 \frac{f^2}{\sigma^2} dt} $$ with respect to the Wiener measure $\mathbb{P}$, which defines the law of $W$ (i.e. $f = 0$). This is exactly like the parametric setting, except the model $\{ \mathbb{Q}^f \}_{ f \in C[0,1] }$ has an infinite dimensional "parameter space".

I believe the notion of contiguity, introduced by Le Cam, is in similar spirit---to introduce a framework where one can speak about densities and likelihood functions, when the parameter space is not necessarily finite-dimensional.

Michael
  • 2,853
  • 10
  • 15