2

I have a data set with 10 predictors and a categorical response variable of 39 levels. The number of observations is nearly 8 million. I'm thinking of binning the response variable into 5 classes, and then perform neural networks.

I know it's a good idea, because it will reduce the no of weights, but I need to justify this. Any research article supporting this theory would be nice.

kjetil b halvorsen
  • 63,378
  • 26
  • 142
  • 467
  • This question is off topic, too broad, and not about programming. I've flagged to close it. – Alex W Dec 06 '15 at 22:43
  • 2
    If you actually *know* it's a good idea (rather than just believe it because you've been told), surely you must know why. I'd love to hear the reasons. – Glen_b Dec 07 '15 at 03:02
  • Check http://stats.stackexchange.com/questions/68834/what-is-the-benefit-of-breaking-up-a-continuous-predictor-variable – Tim Mar 03 '17 at 12:05

1 Answers1

1

There are two parts to this answer:

1) As a direct answer to your question, what you are suggesting is OK if you have classes that you don't mind taking into account jointly. For instance, if your classes are 'dog', 'cat', 'car' and 'red car', depending on the context it might be OK to treat 'red car' as simply 'car'. I very much doubt you will find literature on this as it is very problem specific. So the really what you need to answer is "do I care if I can't tell these categories apart?"

2) You can reduce the number of weights in you NN by relabeling your response variable. For instance, suppose you had 32 categories. In that case, instead of 32 output neurons, you could recode them as 4-digit binary numbers and have your NN try to predict that. For example, if you would code class "9" as "1001". For 39 classes you'd need 5 output neurons.

kjetil b halvorsen
  • 63,378
  • 26
  • 142
  • 467
Felipe Gerard
  • 622
  • 3
  • 7