Modelling when the dependent variable has a "cut-off"

Question

Apologies in advance if any of the terminology I use is incorrect. I'd welcome any correction. If what I describe as a "cut-off" goes by a different name, let me know and I can update the question.

The situation I'm interested in is this: you have independent variables $\bf{x}$ and a single dependent variable $y$. I'll leave it vague, but assume that it would be relatively straightforward getting a good regression model for these variables.

However, the model you're aiming to create is for independent variables $\bf{x}$ and dependent variable $w = \min(y,a)$, where $a$ is some fixed value within the range of $y$. Equally, the data you have access to does not include $y$, only $w$.

A (somewhat unrealistic) example of this would be if you were trying to model how many years people will collect their pension for. In this case, $\bf{x}$ could be relevant information such as gender, weight, hours of exercise per week, etc. The 'underlying' variable $y$ would be life expectancy. However the variable you'd have access to and be trying to predict in your model would be $w = \min(0, y-r)$ where r is the retirement age (assuming for simplicity it's fixed).

Is there a good approach for dealing with this in regression modelling?

Answer 1

This kind of model goes by several names, depending on discipline and topic area. Common names for it are Censored Dependent Variables, Truncated Dependent Variables, Limited Dependent Variables, Survival Analysis, Tobit, and Censored Regression. I am probably leaving out several other names.

The setup you suggest where $\min\{y_i,a\}$ is observed is called "right censoring," because values of $y_i$ too far to the right on the real line are censored---and instead we just see the censoring point, $a$.

One way of dealing with data like this is through the use of latent variables (and this is basically what you propose). Here is one way to proceed:

\begin{align} y_i &= x_i'\beta+\varepsilon_i\\ w_i &= \min\{y_i, a\}\\ \varepsilon_i &\sim N(0,\sigma^2)\; \ {\rm iid} \end{align}

Then, you can analyze this by maximum likelihood. The observations where the censoring occurs contribute $P\{y_i>a\}=\Phi(\frac{1}{\sigma}x_i'\beta-a)$ to the likelihood function, and the observations where censoring does not occur contribute $\frac{1}{\sigma}\phi((y_i-x_i'\beta)/\sigma)$ to the likelihood function. The CDF of standard normal is $\Phi$ and the density of standard normal is $\phi$. So, the likelihood function looks like:

\begin{align} L(\beta,\sigma) &= \prod_{i\ \in\ \text{censored}} \Phi\left(\frac{1}{\sigma}x_i'\beta-a\right) \prod_{i\ \not\in\ \text{censored}} \frac{1}{\sigma}\phi\big((y_i-x_i'\beta)/\sigma\big) \end{align}

You estimate the $\beta$ and $\sigma$ by maximizing this. You get standard errors as the usual maximum likelihood standard errors.

As you might imagine, this is just one approach among many.