What is the quantity $\delta_x$ at point mass $1$ for any point $x$ in the Influence Function formula?

Question

I'm reading an article on the use of influence curves in robust estimation (Hampel, 1974) which includes the following definition of an influence curve for an estimator $T$:

Let $R$ be the real line, let $T$ be a real-valued functional defined on some subset of the set of all probability measures on $R$, and let $F$ denote a probability measure on $R$ for which $T$ is defined. Denote by $\delta_x$ the probability measure determined by the point mass $1$ in any given point $x \in R$. Mixtures of $F$ and some $\delta_x$ are written as $(1 - \epsilon)F + \epsilon \delta_x$, for $0 < \epsilon < 1$. Then the influence curve $IC_{T,F} (.)$ of (the "estimator") $T$ at (the "underlying probability distribution") $F$ is defined pointwise by $IC_{T,F}(x) = \lim_{\epsilon \to 0} \{ T[(1 - \epsilon)F + \epsilon \delta_x] -T(F) \}/\epsilon$ if this limit is defined for every point $x \in R$.

What is the quantity $\delta_x$ measuring?

Is $\delta_x$ the same as the infinitesimal probability $p_X(x)d x$ for a density $p_X(x)$ (say from cumulative distribution $P$) over the interval $[x,x+dx]$? $\delta_x$ is also called an "atomic probability measure" later in the article.

If so, then $IC_{T,F}(x)$ measures the "rate of change" in a function $T(F)$ as you mix in a little bit ($\epsilon$) of an alternate distribution $P$, is that correct?

I'm trying to wrap my mind around how one might have a weighted mixture of two probability distributions. It's an important concept to understand for new causal inference techniques such as Targeted Maximum Likelihood Estimation.

Answer 1

$\delta_x$ is the probability measure defined by $$ \delta_x(A) = \begin{cases} 1 & x \in A \\ 0 & \text{o.w.}\end{cases} $$ so it is just a point mass with all of the probability on a single value. If we integrate some function with respect to it we get $$ \int_{\mathbb R} f \,\text d\delta_x = \int_{\mathbb R\backslash\{x\}} f\,\text d\delta_x + \int_{\{x\}}f\,\text d\delta_x = 0 + f(x) $$ so it effectively evaluates $f$ at $x$ (and the other properties of a measure can be verified). So you can also think of $\delta_x$ as an "evaluation functional" that does the mapping $f\mapsto f(x)$. You can get more on this and other uses in the wikipedia article on the Dirac delta.

Given some other probability measure $\nu$ on $(\mathbb R,\mathbb B)$ it's totally fine to consider a new measure given by a convex combination like $$ P := \alpha \nu + (1-\alpha)\delta_x $$ for $0 \leq \alpha \leq 1$. For some Borel $A$ this is $$ P(A) = \alpha \nu(A) + (1-\alpha)\delta_x(A) = \begin{cases} \alpha \nu(A) + 1-\alpha & x \in A \\ \alpha \nu(A) & \text{o.w.}\end{cases}. $$ Note $P(\mathbb R) = 1$ so this still is a probability measure.

As a side comment, any discrete distribution can be viewed as a convex combination of $\delta_x$ for various $x$. E.g. the Poisson distribution can be written as $$ P(A) = \sum_{n\in\mathbb N} \frac{\lambda^ne^{-\lambda}}{n!}\delta_{n}(A) $$ so we have a countable infinity of weights and the weight for $\delta_n$ is $\frac{\lambda^ne^{-\lambda}}{n!}$.

And it turns out there's nothing wrong with doing these combinations between discrete and continuous measures. For example, suppose $X\sim\mathcal N(0,1)$ and define $Y = \max\{0,X\}$. $Y$ is continuous on $(0,\infty)$ but has a positive probability of being exactly $0$, so it is neither discrete nor continuous. The correct dominating measure here is $$ \frac 12 \delta_0 + \frac 12 \lambda $$ where $\lambda$ is the Lebesgue measure.

---

Regarding $IC$, we have$\newcommand{\e}{\varepsilon}$ $$ \lim_{\e\to 0} \frac{T[(1-\e)F + \e\delta_x] - T[F]}{\e} $$ so I think we can interpret this like a directional derivative where we have our probability measure $F$ and we take a "step" by shifting some mass onto just $x$.