I'd like to understand the differences between these two concepts: Information Value vs Variable Importance.
I understand that they can be used for feature engineering purposes but what is the main difference between them?
Asked
Active
Viewed 117 times
0
kjetil b halvorsen
- 63,378
- 26
- 142
- 467
-
If you were walking in a cave in the dark, it would be important to know that your next step might be into a deep sink hole. That information would be of value. An important variable might be how flat the cave floor is. – Carl Jun 21 '21 at 06:29
-
Thanks for the analogy. In machine learning, there are two formulas to calculate both terms so we can calculate values for each variable. I was wondering which one should I use in which case? Let's say I will be making a churn prediction with logistic regression. How these two terms will help me in this case? – user326138 Jun 21 '21 at 06:44
-
All information is context dependent, so what is important depends on the specifics of what you are asking. More information is needed about the context to help beyond providing Socratic style cave analogies. – Carl Jun 21 '21 at 06:50
-
Does it makes sense to come to a conclusion that variable importance is calculated relatively for each variable but in information value, the variables do not interact with each other. For example, both variables might have high IV but if they are correlated one would have higher variable importance? – user326138 Jun 21 '21 at 08:16
-
Does this help? https://stats.stackexchange.com/a/427641/99274 – Carl Jun 21 '21 at 15:14