5

Related to Exponential-like distribution with support [0,1] I wondered just how close to memorylessness a continuous distribution with bounded support can get. For a continuous variable to be memoryless, it has to be exponential, just as a discrete memoryless distributions must be geometric, so this is a defining feature of the exponential distribution. If the support is bounded, the distribution cannot be exponential so cannot be memoryless, but we may still be able to define a sense in which it comes "close" to being memoryless.

We say a continuous distribution is memoryless if for all $s, t \geq 0$ we have $$\Pr(X>t+s \mid X>t)=\Pr(X>s)$$.

Let's say that we have got "close" to being memoryless if, for example, the absolute value of $$\Pr(X>t+s \mid X>t) - \Pr(X>s)$$ is very small for any choice of $s, t$ and we might want to restrict it so that $X, s, t, s+t$ all lie between 0 and 1. One metric for "closeness to memorylessness" might be the least upper bound for that absolute value of the difference, but if another metric has been proposed before that's fine too.

So whichever sensible way we measure it, just how close to memorylessness can we get?

I suspect the answer is we can get arbitrarily close by using a truncated exponential distribution with mean increasingly nearer to zero. But for a fixed mean of $X$, e.g. $\mathbb{E}(X) = 0.1$, it's no longer intuitive (at least to me) that a truncated exponential would be optimal... does anyone have any suggestions? Is it something that has been researched?

Silverfish
  • 20,678
  • 23
  • 92
  • 180
  • The [geometric distribution is memoryless](https://www.google.com/search?client=firefox-b-1-d&q=geometric+distribution+memoryless). – whuber Sep 18 '20 at 15:44
  • @whuber Indeed, I meant to state I wanted a continuous distribution - which I thought I had, but reading through I see that I haven't... will make a quick edit. – Silverfish Sep 18 '20 at 15:49
  • "Finite support" implies *discrete.* Maybe you mean *bounded* support? If so, if you don't supply a limit to the bound, you can approach memorylessness simply by truncating the exponential at a sufficiently high limit; equivalently, if you wish to bound the support by the interval $[0,1],$ just consider exponentials with arbitrarily small scale parameters. – whuber Sep 18 '20 at 15:50
  • 1
    @whuber Yes this is what I was referring to at the end of the question, I suspect the question is more interesting if we set the support to be $[0,1]$ then impose a restriction that prevents taking the limit of truncated exponential distribution like that, e.g.. requiring $\mathbb{E}(X)=0.1$, – Silverfish Sep 18 '20 at 15:59
  • 1
    Agreed, because that constraint eliminates the trivial constructions I was describing. – whuber Sep 18 '20 at 16:03
  • 1
    for the discrete case, a true memoryless bounded distribution is a dirichlet in 1. you get a process where the event always happens, no memory required. – carlo Sep 19 '20 at 08:15

1 Answers1

1

In terms of the CDF $F(t)$ or the survival function $S(t) = 1-F(t)$ you have

$$p(X>t+s|X>t) = \frac{S(t+s)}{S(t)}$$

You get this fraction to be constant for different $t$ and $s$ when $S(t)$ is an exponential function.

(And obviously the relation breaks when $t>1$ or $t+s>1$, because that exponential relation ends above 1. So you only have memoryless in some narrow sense)

Truncated exponential with point masses

We can have an exponential function for the survival function as follows

$$S(t) = \begin{cases} 1 &\quad & \text{for $t\leq0$}\\ a \exp(-bt) &\quad &\text{for $0<t\leq1$}\\ 0 &\quad& \text{for $t>1$} \end{cases}$$

This is a truncated exponential distribution with extra point masses at $t=0$ and $t=1$ (a mixture of a continuous and discrete distribution).

The most extreme case is when you have a single point mass at $t=1$, by setting $a=1$ and $b=0$, which is $S(t)=1$ for $t<1$ and $S(t)=0$ for $t\geq1$. Or when you have a single point mass at $t=0$, by setting $a=0$, in which case the definition of the conditional probability (which equals zero) becomes a vacuous truth.

Truncated exponential

At first I thought that the truncated exponential would satisfy as well. But in this case the survival function will be

$$S(t) = \begin{cases} 1 &\quad & \text{for $t\leq0$}\\ \frac{\exp(-bt) - \exp(-b)}{1-\exp(-b)} &\quad &\text{for $0<t\leq1$}\\ 0 &\quad& \text{for $t>1$} \end{cases}$$

It is translated/shifted by a constant $\exp(-b)$ to ensure that $S(1)=0$ and continuous.

If the distribution needs to be continuous

In this case you can use the distribution with point masses and substitute the point masses with a continuous function and make them arbitrarily small.

You can also use the truncated exponential and make the constant $\exp(-b)$ arbitrarily small. In the extreme cases $b\to \infty$ you approach the situation with a point mass in $t=0$.

Sextus Empiricus
  • 43,080
  • 1
  • 72
  • 161
  • (+1) Hadn't thought about adding a point mass in, nice idea! Still leaves me wondering just how good you can get with a continuous distribution... – Silverfish Sep 18 '20 at 19:38
  • One idea is to follow https://stats.stackexchange.com/questions/188903/intuition-on-the-kullback-leibler-kl-divergence/189758#189758 and minimize a KL divergence, the solution to the minimization problem is obvious (but that will not give point masses) – kjetil b halvorsen Sep 18 '20 at 19:39
  • @Silverfish, I believe I made a mistake with the truncated exponential. It's survival function is not a plain exponential. – Sextus Empiricus Sep 19 '20 at 07:23