In this study:
The researchers attempt to classify Parkinson's disease (PD) with a control group of 20 participants and a PD patients group of 20 participants, while the participants are writing on a digitized pen and paper.
The researchers found that a MANOVA analysis predicts which is which with 95% accuracy.
I'm pretty new to MANOVA, but it seems to me that there is a need to separate the groups to training set and validation sets. Am I correct? In other words, is MANOVA prone to overfit?
Unfortunately I don't have access to this paper and could not find it available online. So I will comment only based on their abstract.
First, they did not really use MANOVA to do the classification, they used linear discriminant analysis (LDA). LDA and MANOVA are very related (see this rather mathematical explanation by @ttnphns and my own account here), but MANOVA is usually understood as a procedure for statistical testing and LDA as a procedure for classification (and also for dimensionality reduction). Notice that the abstract reads:
Results of the MANOVAs [...] showed significant group effects [...]. A discriminant analysis was performed for the two tasks. One discriminant function was found for the group classification of all participants [...]. Based on this function, 97.5 % of participants were correctly classified [...].
So they use MANOVA to provide statistical support to their claim that two groups are different, and then use LDA to perform classification.
Second, I see nothing in the abstract that suggests that they did any cross-validation, and if they indeed did not, as you say in your question, I cannot imagine any excuse. LDA can certainly overfit and reporting classification accuracy without doing cross-validation is, well, a very questionable practice, to put it mildly.
If you manage to put this paper online somewhere, I could take a closer look.
