2

In my job, I often use variables to build models that traditional business sense would suggest are monotonic in their relationship with the target variable of interest. For example, in business lending, we would always suppose that the older the business, the lower the probability of default is.

In reality, we often see that there are areas where the assumption of monotonicity is broken. My question is, is there a test we can perform which tests whether or not our assumption is valid?

  • In linear models, this is sort of the default assumption. Are you specifically asking about cases where the effect of the covariate is non-linear, such as in GAMs? – Demetri Pananos Mar 30 '20 at 14:25

1 Answers1

0

Let's call $X$ and $Y$ two random variables, and say you want to test if $X$ and $Y$ are monotically dependent.

One basic way of doing so can be to use Kendall's tau correlation coefficient. It has the advantage of being non parametric : it does not suppose any particular distribution on $X$ or $Y$, nor suppose any specific link between the two variables. If $X$ increases with $Y$, you should get $\tau > 0$ and if $X$ decreases with $Y$, you'd get $\tau < 0$.

If your dataset is big enough, you can make a (one or two sided) test for $\mathcal{H}_0 : \tau = 0 $ by using the asymptotic distribution of $\tau$ which is $\mathcal{N}(0, {\frac {2(2n+5)}{9n(n-1)}})$ under null hypothesis $\mathcal{H}_0$.

One could also use other non parametric correlation coefficient like Spearman's $\rho$, but this discussion seems to show that Kendall's $\tau$ is better for what you're interested in.

Pohoua
  • 2,003
  • 2
  • 15