2

I am trying to contrast results of various MDS approaches applied on the same dataset and understand their comparative interpretation.

I calculate the goodness of fit for the various models with the functions:

cmdscale(Dataset,2)$GOF                                             #Torgerson (Classical) 
smacofSym(Dataset,2)$Stress                                         #Metric
EurocitNM <- smacofSym(Eurocit,2, type="ordinal")$Stress            #Non Metric

As a rule of thumb the first value should be >0.8 to be acceptable and the second and third should follow the classification given by Kruskal, J. B. (1964) Multidimensional scaling by optimizing goodness of fit to a nonmetric hypothesis. Psychometrika [p.3].

Now, I would like to make a comparison between these interpretations. I have seen in Patrick Mair, Ingwer Borg, and Thomas Rusch (2016) Goodness-of-Fit Assessment in Multidimensional Scaling and Unfolding Multivariate Behavioral Research [p.774] that :

By default, most MDS programs use a classical scaling solution as starting configuration. Classical scaling (Torgerson, 1952) is an old MDS technique that tackles the problem analytically but it is much less flexible than numerical MDS approaches. Classical scaling often provides a good initial guess.

So, should I compare the Metric and Non Metric configuration matrices with the Torgerson (Classical) configuration, maybe calculating their correlations element-wise? Are Shepard diagrams useful for my goal, should I study them? Are there anymore methods to make this kind of comparison I have in mind?

q0mlm
  • 21
  • 2
  • Term "Classical MDS" is ambiguous and ought to be avoided. "Torgerson's MDS" is a better term. See "A note on terminology for a reader" [here](https://stats.stackexchange.com/a/14017/3277). – ttnphns Nov 22 '20 at 10:04
  • I would concur completely with the passage you cite. In fact, iterative MDS algorithms are better then the PCA-based Torgerson's method (read the thread I link to above) but they need initial solution of which Torgerson's result is often the best choice. – ttnphns Nov 22 '20 at 11:57
  • 1
    If you need to compare the fitting qualities of different MDS algorithms, run a simulation study with different input distance matrices and different goodness of fit (stress) criteria. Apply each algorithm to to each matrix. Compare values of each criterion. Some algoritms may win on some stress criteria, other may win on other. – ttnphns Nov 22 '20 at 12:01
  • 1
    @ttnphns: This looks like a good answer. One can add that the key issue here is that different algorithms optimise different criteria, and therefore it depends on the precise criterion what's best. – Christian Hennig Nov 22 '20 at 12:54
  • That is very true. It is instructive experience to compare algorithms on respective "native" as well as "extraneous" criteria. – ttnphns Nov 22 '20 at 13:04
  • @ttnphns thanks, I have seen there is some ambiguity in the sources, I will use the term Torgerson's MDS. So, there is no way to compare these algorithms on the same input matrix? What are the criteria that I should change, for example the number of iteration on a SMACOF algorithm? – q0mlm Nov 22 '20 at 19:31
  • It's not that there is "no way" to compare the algorithms. Rather there are several ways, and unfortunately which one comes out best will normally depend on which way you take... Sometimes what the MDS is used for in practice is helpful to define a good "impartial" comparison criterion. – Christian Hennig Nov 22 '20 at 22:26
  • @Lewian should I establish a "first guess" benchmark beforehand based on my knowledge of the field of application, take it as a good approximation of how the results should be and then compare the results of the various algorithms with it? Is this what you mean with define a good "impartial" criterion? – q0mlm Nov 22 '20 at 22:54
  • As far as I understand what you're saying, without knowing any of the details, that looks good to me. Your "first guess" benchmark should be good and reliable of course. – Christian Hennig Nov 22 '20 at 23:42
  • @Lewian I'll try to explain myself: let's say I have a distance matrix of cities, take the typical dataset (eurodist) used in many MDS examples. I would like to compare the resulting 2D maps in a Torgerson, in a metric and in a non-metric model. The only way I have to understand which is more accurate is having a map of Europe and then compare the three results with it? Are there some non knowledge specific tests that could tell me which model is better for the case, i.e. if I don't have a Europe map could I still make some comparison? Is there a way compare cmdscale_GOF with smacofSym_stress? – q0mlm Nov 23 '20 at 11:03
  • I can only repeat myself. The problem is not that there is no way, but rather that there are many ways, and which way makes sense depends on the specific situation. – Christian Hennig Nov 23 '20 at 16:24
  • @Lewian, perfect, I understand. Is there any source where I can find some of these methods and see if they can be applied on my case study? – q0mlm Nov 23 '20 at 17:10
  • 1
    Many MDS methods are defined by optimising certain criteria (stress) that measure fit quality. One can always ask, how good is one method regarding the criterion used by another one. Furthermore, if there is known structure in the data (such as groupings) one can measure how well the solution fits the groupings in various way... sorry for not being more precise, there is a reason why I didn't write a proper answer. – Christian Hennig Nov 23 '20 at 21:14
  • @Lewian thank you very much for your answers! – q0mlm Nov 23 '20 at 22:35

0 Answers0