Variable Importance Plot & Hypothesis Testing

Question

I am trying to predict whether client would take up a promotion or not using the random forest model. From the variable importance plot output, it is shown that the number of days pass the day of contact(which I bin in days of 5) is the most important variable in the predictive modelling.

However, when I put this into a hypothesis testing, chi square test, the p-value is greater than 0.05. How do I reconcile these two facts or if there is a better way of testing that the number of days in relation to whether client would take up a promotion?

@rep_ho that would be chi^2 test on the categorical variables: ContactDay & Yes/No — AGZH12, Apr 16 '21 at 12:57
Might be the case that the effect is confounded by other variables. Generally though, the p value does not tell you how important a variable is or if its effect is large or not. — Demetri Pananos, Apr 16 '21 at 13:20

score 2 · Answer 1 · answered Apr 16 '21 at 13:20

2

You lost information by binning, see Why should binning be avoided at all costs?. For an alternative, use logistic regression and spline the day of contact variable. For details see Logistic Regression with regression splines in R or Using splines to address non-linearity in logistic regression

answered Apr 16 '21 at 13:20

kjetil b halvorsen

63,378
26
142
467

thanks for the reply. I tried logistic regression before but the accuracy rate is not as good as the random forest model. I would just like to check if using any other hypothesis method would be good for testing – AGZH12 Apr 16 '21 at 14:23
Then please edit the post to give some more info, like other covariables, sample size, show us a plot, ... – kjetil b halvorsen Apr 16 '21 at 15:54

Variable Importance Plot & Hypothesis Testing

1 Answers1