1

I am training a 4-class neural network classifier. The details of my data are:

featurelength = 280

              training      testing
---------------------------------------------
no. samples      438           250
samples class1   203           135
samples class2   19            10
samples class3   16             5
samples class4   200           100

After I train the neural network.

The confusion matrices are as follows:

203 0 0 0 
19  0 0 0
16  0 0 0
200 0 0 0

Similarly for test data too. All the data is predicted to be of class 1.

How can I debug this issue? What is the reason behind such performance? In addition to this, no. of samples for class 2 and class 3 are very less. How can I make sure that they are not dominated?

Swagatika
  • 111
  • 4
  • my first bet is that the labelling is wrong. are you sure all samples are correctly labelled when you feed them into the classifier? – jpmuc Apr 23 '13 at 07:44
  • I think so. But let me check again. – Swagatika Apr 23 '13 at 07:48
  • @juampa, data seems alright. funny thing is I added some more data of class 4 in training, and the no. of samples in class 4 is now 300. With this, I get good results in training. Test result is not so good though. But why so? AccTrain: 0.814126 AccTest: 0.566581 Mean Diag train: 0.894766 Mean Diag test: 0.416521 Confusion matrix training: [ 148 0 0 55; 0 19 0 0; 0 0 16 0; 44 0 1 255] – Swagatika Apr 23 '13 at 10:39
  • Right now I cannot figure out what your problem could be. I do not know the matlab package for neural networks, but I guess you are using sigmoidal activation function for the output units and you are trying to minimize the cross-entropy function. Do you normalize data previously?. You have little # of samples in contrast to #features. Maybe preprocessing with PCA helps. Last, that tests are worse than training could be an indication of overfitting (too many neurons). Try cross validation. – jpmuc Apr 23 '13 at 12:25
  • Yes, I normalize the data. I will try the rest two approaches i.e., PCA and cross validation. Thanks :) – Swagatika Apr 24 '13 at 09:13
  • I tried PCA. Here is the steps I followed: %%%%%%%% %coeffs : nFeatures x nfeatures, scores = nSamples x nFeatures [coeffs, scores, variances, t2] = princomp(trainData); testDataMapped = coeffs * testData'; testDataMapped = testDataMapped'; testDataPCA = testDataMapped(:, 1 : THRESH); %%%% Is this the right way to perform PCA? When I combined all the data and performed PCA, I got infinity values for variance. – Swagatika Apr 26 '13 at 15:56
  • PCA gave slightly lower performance. I tried reducing number of updates while training and got a better accuracy (74 % for training and 64% for testing). – Swagatika Apr 29 '13 at 05:48

0 Answers0