-1

I have the following dataset that constitutes of four attributes that I can use for classification of objects. I know that there are two classes of objects from this dataset but I do not have target output. i want to classify these into two classes. Which type of ANN would you recommend for this problem?

Below is the data set:

enter image description here

CN1002
  • 123
  • 7
  • 4
    You have 4 features but no target value or class? This sounds more like a unsupervised learning task where you want to find structure within the data. If you insist on using neural networks have a look at [autoencoders](https://en.wikipedia.org/wiki/Autoencoder). – Harald Thomson Feb 19 '16 at 12:16
  • Yes, I have no target values but I am certain there are two classes from this data set. Let me check your link, thanks. – CN1002 Feb 19 '16 at 12:23
  • 1
    Why you want to use ANN? Your dataset is too small for ANN, and there is no point if your input data is not highly non-linear or high dimentional. Give a try to K-means. – yasin.yazici Feb 19 '16 at 12:33
  • Well, from reading I am told ANN work better in classification problems. i am new to ANN and would like to see them working – CN1002 Feb 19 '16 at 12:35
  • My data set has 1000 entries (-rows). That image i meant for quick view and understanding – CN1002 Feb 19 '16 at 12:37
  • 1
    ANN is not necessarly better than other classification techniques if learning is sufficient with local generalization. If you want to work with ANN find more appropriate dataset like MNIST, CIFAR-10 etc. – yasin.yazici Feb 19 '16 at 12:38
  • @yasin.yazici i could start with this data set and see how ANN work, I want to see how they classify objects given parameters. – CN1002 Feb 19 '16 at 12:41
  • 2
    The main problem is that this is an unsupervised learning task, hence PCA/ICA would probably be the methods to start with. – Harald Thomson Feb 19 '16 at 13:02
  • @HaraldThomson Well, i am checking it out now – CN1002 Feb 19 '16 at 13:09
  • 1
    Read this post also http://stats.stackexchange.com/questions/140148/how-can-an-artificial-neural-network-ann-be-used-for-unsupervised-clustering . If your aim is to learn ANN, I recommend you to start with simple classification task. – yasin.yazici Feb 19 '16 at 13:12
  • @yasin.yazici offered a good suggestion. This problem seems ideally suited for $k$ means. – dsaxton Feb 19 '16 at 13:53
  • @yasin.yazici I checked k-mean clustering and run it in in Weka though I could not better interpret the results. but that is not my worry, my worry is - Is it possible to use a trained model of a k-means algorithm in some java application? Like, suppose I would like to test and see what this `0.1; 0.011;0.023;0.45;` entry belongs to which of the two classes? – CN1002 Feb 19 '16 at 13:55
  • 1
    You don't classify them but cluster them. So if you don't know which class an input belongs, you can't know if the clustered point belongs to the true classes. Lets say you run k-means code and collect cluster points for each instance then, mapped your input features to 2D or 3D for visualization. Even if you see clear distinction between the two cluster it doesn't mean it catched the true classes. Maybe half of the instances of a feature collected to some part of the input space and the other half to the other part of it. Hence, two different clusters corresponds to the same class. – yasin.yazici Feb 19 '16 at 14:05
  • @yasin.yazici Well, I got it...I wanted something that could learn (heard ANN do learn) from the data set and then I would used a trained/learned version to perform classification in my application. – CN1002 Feb 19 '16 at 14:13

2 Answers2

1

You have 4 features but no target value or class?

This sounds more like a unsupervised learning task where you want to find structure within the data. If you insist on using neural networks have a look at autoencoders.

They are neural networks which aim to predict their own input. The trick is to force the system to learn a compressed representation by ensuring that the number of hidden units is smaller than the number of input units.

$|l_{input}| = |l_{output}| > |l_{hidden}|$

Once the autoencoder is trained you can use the representation of the hidden layer. If the training was successful, this representation is a lot simpler than the one of your initial data.

There are various different variants. Some give you a sparse representation, some just limit the number of active neurons. In any case, regularization is key.

In case you have two classes I would use a single hidden layer, with a sigmoid activation. The value within the hidden layer then should represent the probability of a possible two class distinction.

However, this representation is not necessary the one you intended to find. It's just the one which was found by the autoencoder. If you want to make sure that the representation is right, you might plot the data or use ICA or a similar approach.

Harald Thomson
  • 384
  • 1
  • 9
  • I been to the linked page, and seen this `The differences between autoencoders and MLPs, though, are that in an autoencoder, the output layer has the same number of nodes as the input layer` -Is not that supposed to mean that I would have 4 output neurons? If so how would I then interpret the results? I was thinking of having two output neurons - one for each class. – CN1002 Feb 19 '16 at 12:31
  • A added more info. – Harald Thomson Feb 19 '16 at 12:40
  • Well, suppose then I would like to implement this in MATLAB or Neuroph Studio - I am not seeing the option for autoencoder. How do i go about it? – CN1002 Feb 19 '16 at 12:46
  • Probably the easiest way is to construct your own the autoencoder from layers just as you would do with standard neural networks. I'm not familar with MATLAB, hence I can't help you there. – Harald Thomson Feb 19 '16 at 12:53
  • @HaraldThomson The network you constructed produces single feature that is useful to recontruct the original data. So, It doesn't catch the clusters but rather single most variable direction (non-linear) in the input space. – yasin.yazici Feb 19 '16 at 13:02
  • @yasin.yazici Yes. In fact, it can be proven that autoencoders learn a PCA representation by training them with a MSE Loss function. Actually PCA is a good start for his analysis. However, you are able to optimize to different objectives by adding regularization and other nonlinear activation functions. – Harald Thomson Feb 19 '16 at 13:22
1

The data looks like nonlinear structure. In that case, nonlinear dimensionality reduction method like Sammon Mapping could be applied. I have seen Sammon Mapping with neural network in the Internet. This method could be alternative to Autoencoder mentioned by Harald Thomson in this page.

Cloud Cho
  • 140
  • 6