7

I recently received a review back for a paper in which I referred to some previous studies as 'correlational' where they used multiple regression to analyze some population data and make biological conclusions (specifically a linear mixed effects regression). One reviewer made a very big deal about this, suggesting that I "completely mischaracterized" this work (I suspect he/she was an author) which was "far from correlational", "much more sophisticated", and provided "much stronger evidence than mere correlational methods". I suspect the reviewer was referring to the fact that multiple regression models control for other included variables. From my (current) point of view, they are all correlational, just of differing complexity and assumptions between independent and dependent relations. Our study was trying to give a more mechanistic account of the data, therefore I wanted to make a distinction between statistical and mechanistic relationships. So I used the term correlational in a very broad sense.

So my question: Is it inappropriate to describe linear regression models as 'correlational', and would you yourself do so? If not, why not?

I am familiar with the mathematical relationships between regression coefficients and the 'partial correlation coefficients' e.g. discussed here, and here. My question, rather, is about terminology and whether you folks find it too loose (in some sense) to refer to regression methodology as correlational, or if a more broad term like 'statistical' would be more appropriate in this case.

Much thanks.

Community
  • 1
gregory_britten
  • 1,253
  • 9
  • 15
  • 8
    Both you and the reviewer have a defensible case. The reviewer is treating the term narrowly and you are treating it very broadly. I have been on both sides of the relationship here, and a journal editor too, and advise treating this as needing clarification, not confrontation. But as you ask for specific advice: (1) I wouldn't use the term "correlational" for multiple regression; (2) I would make my point in other terms when one choice of terminology is clearly an irritant. – Nick Cox Mar 18 '14 at 19:02
  • Thank you very much @NickCox, I appreciate the input. All is well taken :) – gregory_britten Mar 18 '14 at 19:12

2 Answers2

8

Correlation can be two things. Correlation is a mathematical construct on one hand, which is de facto Pearson correlation. Correlation is also a counterpart to causal on the other, meaning conditional dependence between an "exposure" and "outcome" that may be mediated by 100s of unmeasured factors. Calling work (i.e. the analyses/results of a study) "correlational" doesn't immediately suggest to me whether you mean they summarized several bivariate associations using partial correlations or whether the study was conducted from observational data.

I am strongly inclined to believe that you and the reviewer hold opposing ideas of what "correlational" means in this context. This is giving generous credit to the idea that other aspects of this communication did not denigrate anyone's research/findings.

As far as regression analyses are concerned, you can use regression models to analyze "quasiexperimental" data (or observational data) in which adjustment for confounding variables is used to infer what a hypothetically controlled (blocked/randomized) experiment would yield as a result. This leads to the distinction between correlation and causation. Only randomized controlled trials are worthy of discussing results in a causal context. Other results are not "correlational" but you may refer to findings as "associations".

The word correlation is confusing. In literature presented to a statistical audience, I am careful to avoid correlation altogether except in the context of Pearson's correlation. I would favor "empirical" or "epidemiological" or something of that ilk to refer to findings from observational studies.

AdamO
  • 52,330
  • 5
  • 104
  • 209
  • very good points, but I raise the question: would you call the results of a designed experiment (for which there appears to be causal evidence) 'empirical'? – gregory_britten Mar 18 '14 at 19:45
  • 1
    I suppose it is the difference between 'mechanistic' and 'statistical' I am grappling with. What I meant in the paper was to say that previous studies found a statistical relationship, but the demonstration suggested no biological mechanism. I called this correlational in the sense that they observed a purely statistical dependency. We fitted a process-based model to the empirical data to suggest a biological mechanism, and therefore claimed that we provided a mechanism for previously correlational results. Regardless, the comments are clarifying my usage. So thanks. – gregory_britten Mar 18 '14 at 19:50
  • 1
    @gregory_britten I guess I fell into my own trap there! It's all about avoiding words with ambiguous double entendres in mathematical statistics. "Empirical", alas, is victim of that. From the perspective of describing results, empirical is a good descriptor of *in vivo* (observational) studies whereas results from controlled experiments have *in vitro* applications. In summary, the word "correlational" should be dropped altogether. If findings are consistent with other literature and your own, then good. If the findings are inconsistent, then don't dismiss that research. Discuss it. – AdamO Mar 18 '14 at 20:11
  • 1
    I'd differentiate between *empirical* & *mechanistic* models, the form of the latter being more heavily based on theory, though not necessarily implying a causal relationship from predictor to response; & between *observational* & *experimental* studies. – Scortchi - Reinstate Monica Mar 18 '14 at 21:38
  • 1
    @Scortchi agreed, although there is a great deal of correspondence between the two distinctions: observational data, error prone and subject the infinite unmeasured mediation pathways, is carefully adjusted by measured confounders, but the results are important to interpret as "population averaged" which is why large public health data almost *necessitates* GEE (the empirical approach). On the other hand, experimental *in vitro* data are so carefully controlled, little wonder analysts favor measuring and fitting sophisticated probabilistic models for their data (the mechanistic approach). – AdamO Mar 19 '14 at 18:36
  • You're right, though it depends on the area too. When I worked for an engineering company I fitted empirical models to experimental data (response surface methodology) perhaps rather more than mechanistic models to experimental data; but I don't recall ever fitting a mechanistic model to observational data, only empirical models. – Scortchi - Reinstate Monica Mar 19 '14 at 19:01
3

Knowing no more than what you've said, I agree with @NickCox, @AdamO, and with you for the most part. If you have discussed this "completely mischaracterized" work in further depth than you've said here, it may not be safe to assume the objection is mostly to your characterization of it as "correlational", unless the reviewer has made that really clear. His/her objections seem very emphatic, so you're right to suspect s/he is an author. Might you be able to talk to this reviewer more directly to seek consensus (or at least compromise) on how to describe the work? I suppose some review processes might rule out conferral outside the formal framework, but it seems counterproductive in this case if you can't work together with this person.

Of course, this isn't to say there's any certainty of whom you're dealing with beyond what you actually know; I only agree it seems likely, and echo the other Nick's suggestion to seek clarification from this person. It would seem to be in your mutual best interest to describe everything optimally from both perspectives. Hopefully all the initial bluster will die down as you communicate further. Sometimes people begin by making a bigger fuss than is really necessary just to ensure they get your attention and make an impression, despite the evident risk of that being a worse impression (e.g., biased, unreasonable, alarmist). It's easier for some to proceed with restraint and mutual respect once they've been reassured that the lines of communication are open and the other party is paying attention. It seems you've made the opposite first impression with what you've written, so an overblown reaction is understandable, if still unreasonable. Clarification should help greatly if s/he's willing to work with you.

Regression is indeed correlational in the broad sense, as has already been said here, but the strictest, most simplistic sense of correlational may appeal to those who have a less nuanced understanding of general linear models. The stricter usage may also appeal more to people who understand correlational analysis as a loaded term in causal research contexts, such as your problem reviewer, it seems. If your intention was to imply a weakness in causal evidence, then your usage was loaded intentionally, and some defensiveness is to be expected. You'd be right to say that multiple regression doesn't really provide more causal evidence than a bivariate correlation (the stricter sense of "correlational" that AdamO described) – it mostly makes relational evidence clearer regardless of whether these relationships are causal. Hence I doubt your reviewer is simply objecting to your characterization because the method involved multiple regression, not just bivariate correlations. Maybe the reviewer felt the original design had other, "more sophisticated" elements that provide "much stronger evidence". Maybe I give this person too much credit though; I've been led to think one should never underestimate the capacity of reviewers to overreact to issues that amount to nitpicking.

Generally, I wouldn't object to describing regression as correlational, and might have done so myself in your case initially, but given the apparent offense this has caused, I don't see any harm in backing off and rephrasing. If your intention was to imply critique of causal evidence, it would be better to state the critique clearly and delicately, not to just imply it. If your alternate phrasing of the "distinction between statistical and mechanistic relationships" captures your meaning just as well, maybe you can avoid the issue by replacing the "correlational" phrasing entirely, but again, if you can confer with your reviewer about this alternative, you'll stand a better chance of having the change received well, of course. AdamO has provided some other good alternatives, and your comment on his answer seems quite a bit clearer about the distinction you intended to make between your work and your reference. As for "empirical", I think you're encountering the same basic issue by using a single word with a variety of possible interpretations where several sentences that clarify your intention with context would be preferable.

Nick Stauner
  • 11,558
  • 5
  • 47
  • 105