I have
I would like some advice/pointers as to the best machine learning classifier for my task. To be implemented in Java to train. I have three main choice decision tress, navie bayes or a perception but am not sure which would best fit my problem. Any java implementations i could be pointed in the direction of would also be great. Thanks
There's really no way to tell apriori which algorithm will be best for a given problem. The best approach is usually to try several different algorithms, validate them out-of-sample on a test set or through cross-validation, and then choose the algorithm with the lowest out-of-sample error.
Weka is a great Java machine learning library. I'd also add logistic regression to your list, as it's one of the simplest and most common approaches to binary classification.
Before selecting learning algorithm you have to answer several questions (actual program and library names follow the long introduction) :
The first question is very important in the cases that you need to provide a mechanism or causality explanation of the results (i.e. the more educated a person is, the bigger is its income). In these questions you may stick to simple models such as logistic regression or pruned decision trees. If all you need is accurate predictions, then you may try more "black box-ish" methods such as neural networks, random forests etc. Do note though that in these cases you gain flexibility but loose the ability to debug and troubleshoot your models.
Which brings us to a second question: how different do you expect the testing cases to be from the training set? If the answer is "pretty much" or "I don't know", then you have to limit yourself to less flexible models, as your chances to overfit your model raise exponentially with the flexibility (number of parameters and descriptors/predictors) in your model.
You also need to formulate how you compare classifier performance. This hangout provides a nice background on this issue.
Another pivotal issue is the nature of the data. Bayesian classifiers, for example, are most suitable for discrete data, so proper discretization IS an issue, neural networks that user perceptrons handle continuous and discrete values, but you have to be careful how you encode nominal values.
Weka is a java-based program full with loads ready to use machine learning algorithm, evaluation methods etc. There is also excellent data mining book that uses Weka for all the examnples: Data Mining: Practical Machine Learning Tools and Techniques
Knime is somewhat similar to Weka (uses Eclipse), it also has a companion book: Guide to Intelligent Data Analysis
Jubaus is a machine learning framework (no GUI) with Java (among others) bindings.
If you want to implement an algorithm by yourself or study from a source-code, I highly recommend Programming Collective intelligence. Although it uses Python and not Java, and also it has been criticized for coding style and too shallow theoretical background, it shows how popular learning algorithms are implemented in easy to comprehend step-by-step manner.
If you are trying to learn a mapping between people's features to their incomes, then you don't have a binary classification problem, you have a regression problem.
You might start experimenting with linear regression, polynomial regression, and if they fail, move to multi-layer perceptron neural networks (with an unbounded output layer activation function) or support vector machine regression.
Keep in mind that if your data set comes from a random sample of the population, income will be distributed according to a heavy-tailed distribution (e.g. Pareto), that is, there will be a small, but non-negligible, number of people with an income much higher (orders of magnitude) than the average.
This is problematic for typical regression algorithms because it makes the underlying optimization problems "stiff", resulting poor speed or even numerical instability.
Furthermore, evaluating the accuracy of your regression algorithms with standard error measures such as mean squared error might be misleading, because the algorithm will typically mispredict these high-income instances (since they will be probably intrinsically difficult to predict given the input features) and these will have a disproportionally high effect on the overall error measure.
I suggest you might try to preprocess your data applying a log or a log log transformation to incomes, and then normalize them, and all your numeric input features, to fall approximately within some small interval (e.g. -1..1 or 0..1).
