9

I am not sure if this is the right place to ask this but here goes:

Sometimes times two or more inputs of a neural networks can often be related to a single "real world" entity.

E.g : Height and weight of a person to predict the probability of disease in population or price and volume of a stock to predict the market.

When a single training set contains data about a number of these entities, how can we make a neural network understand that two inputs (or more) are related to the same entity?

Amongst all the people I have asked, the general consensus seems to be:

  • Neural Networks do not work this way
  • It is not possible
  • Such a grouping of data is not required
  • Neural Networks are supposed to find the relationship amongst inputs, you are not supposed to feed it the relationships
  • The training data set need to be reworked / reconfigured
  • I have never heard of such a thing

So, obviously this is not in the mainstream. Has anyone heard of any research in this direction?

P.S. If you agree with the above opinion (it can't / shouldn't be done) please provide a reason why.

gung - Reinstate Monica
  • 132,789
  • 81
  • 357
  • 650
Shayan RC
  • 191
  • 1
  • 5
  • 2
    The fact that the values appear in the same input vector already conveys the information that they correspond to the same instance (entity). – alto Jan 13 '14 at 16:07
  • 1
    But what happens when we have the data of a population of entities to predict something about the entire population? See my (edited) eg. – Shayan RC Jan 13 '14 at 16:12
  • You'll need to add more details as there are several different ways one could approach such a problem. The most important issue would probably be if the size of the population you want to make a prediction about is fixed or not. – alto Jan 13 '14 at 16:21
  • 1
    I'm trying to predict stock market crashes using a neural network but I was hoping for general answer. In this case the population size is fixed. – Shayan RC Jan 13 '14 at 16:27
  • 1
    You can use a network that isn't fully connected until the end result is calculated. Certain inputs get connected together down one path, while other inputs are connected down a different path, and then they fully connect to get a result. – Frobot Mar 30 '16 at 03:48
  • @Frobot That's exactly what I ended up doing :) But in most cases, more than one fully connected layer at the end improves performance vastly. – Shayan RC Mar 30 '16 at 04:49
  • I'm surprised no one mentioned [PCA](https://en.wikipedia.org/wiki/Principal_component_analysis). Basically, reduce the inputs to non correlated items. Here's a [paper](https://www.researchgate.net/publication/228968356_Improved_neural_network_performance_using_principal_component_analysis_on_Matlab) on its effectiveness. – Phil P Jun 27 '18 at 00:50

1 Answers1

4

Sometimes the correlation level between any of the two input variables are calculated, and then the input is partitioned into several independent sub-groups before the training starts, like what was implemented in this paper.

But generally, like @alto says, when you provide those inputs, the neurons will treat them as they correspond to the same entity. Each neuron at hidden layer will response different variables to different extent, reflected by its connection strength to the variables (i.e, the weights). And those responses are combined to generate a final response at the output layer (linear combination, or plus some activation functions). During training process the weights are adjusted to better learn the output they are given. And finally when the training is done, with the obtained strengths between each neuron and each input variable, the network can respond to any other inputs to different levels, and that is the prediction part.

Note that the neurons will reduce the connection strengths to some input variables if they learn that those variables do not contribute the final consequence very much.

lennon310
  • 2,582
  • 2
  • 21
  • 30