0

I have one sample and several features. I calculate a z-score for various features, and for various combinations of features. Is there a way to quantify the significance of the difference between two z-scores?

For example, suppose I calculate a z-score for the combination of features (male, homeowner) and also for the combination (male, homeowner, married). For one scenario, suppose the first z-score is 3.0, and the second z-score is 6.0. For a different scenario, suppose the first z-score is 25.000, and the second z-score is 25.001. I'm trying to determine whether or not (male, homeowner, married) is a better predictor than (male, homeowner). Is there a generally applicable formula to quantify the significance of the difference, perhaps taking into account the sample size?

I have no training in statistics, and am trying to implement a software program. A cookbook formula accompanied by some beginner-appropriate explanation would be ideal and greatly appreciated.

Thanks

Tyro
  • 141
  • 1
  • 9
  • It would be helpful to have more detail on your outcome/dependent variable, especially whether it is a continuous or categorical variable. My other question is why are you calculating z-scores? Do you mean that you are getting a z-statistic as a hypothesis test about overall model fit? There will be answers to your question, but more detail will help people answer as there are many, many potential pitfalls for what you want to do. – James Stanley Jan 07 '13 at 22:24
  • Currently, both dependent and independent variables are dichotomous, although I would like to also handle the case when one or the other is continuous. And yes, I am using the z-test for hypothesis testing for the selected feature as a predictor. – Tyro Jan 07 '13 at 23:01
  • Anything? Your comment sounds promising. – Tyro Jan 13 '13 at 23:24
  • It's probably a question of reading up on model selection methods -- I don't have time to write an answer (which would need to be quite long), but I could recommend in the first instance Frank Harrell's "Regression Modelling Strategies." – James Stanley Jan 14 '13 at 21:52
  • From a statistical perspective, when comparing two model fits in logistic regression, you might look into likelihood ratio tests. The main validity concerns I have are around using automated selection of predictors -- see http://stats.stackexchange.com/questions/18638/model-selection-logistic-regression and linked questions for more points on this. – James Stanley Jan 14 '13 at 21:53
  • Thank you for providing these references. A quick look at Frank Harrell's book on Amazon shows that it is way over my head. A quick look at likelihood ratio tests on Wikipedia seems to say it provides a p-value for one test versus the null test. Is your suggestion to compare the p-values from several such tests? This sounds similar to my original question regarding z-values, to my uneducated mind. I will follow your suggestion and Google model selection methods to try to educate myself, but I fear it is a vast and deep subject. – Tyro Jan 17 '13 at 02:38
  • Model selection is difficult -- and the main issue is not to proceed with what one **can** do without considering whether it is **reasonable/valid** to do so, which is why the preparatory work is important. Sorry to not have a nice quick solution. – James Stanley Jan 17 '13 at 02:58

0 Answers0