2

Can a Spearman correlation coefficient of 0.38 for a specific parameter be considered to demonstrate reasonable agreement between two biological cell types, more specifically a cell line and a primary cell type?

The parameter in question is the difference between percent of mRNA with and without last exon in two subcellular compartments, so for every gene (of which there are ~40000):

$\text{param} = x_{i} - x_{j}$,

where

$x_{i}$ is the proportion of RNA with last exon in compartment $i$

$x_{j}$ is the proportion of RNA with last exon in compartment $j$.

For each cell type, this results in a table of

$\text{gene} - \text{param}$

And the Spearman correlation is found between the ranked order of this parameter in the two cell types.

Sample size is number of isoforms considered, in the order of 40000. Number of biological replicates per cell line - three each (standard in the field).

Nick Cox
  • 48,377
  • 8
  • 110
  • 156
dvanic
  • 43
  • 5
  • 4
    The phrases "for a specific parameter" and "two cell types" (biological? Cells in a table? Electrical cells?) are rather unexplained in your question. With the question written as it stands, you can only hope for a very generic answer, though Nick Cox's is excellent). – Silverfish Mar 03 '16 at 08:15
  • 1
    It would also be helpful to know the sample size, since the result may or may not even be statistically significant depending on the n. – Silverfish Mar 03 '16 at 08:20
  • @Silverfish Edited for clarity, and with more details - thanks! – dvanic Mar 06 '16 at 06:39
  • Thanks for the detail. Note that where you say _parameter_, statistical jargon is always _variable_. This translation is immediate, and not problematic. – Nick Cox Mar 06 '16 at 08:45

2 Answers2

9

Correlations, such as Pearson's product moment correlation or Spearman's rank correlation, are not measures of agreement, no matter what their values are (i.e., even if $r = 1.0$).

Consider a simple case using Pearson's correlation:

enter image description here
A guy takes a woman on a date to what he thinks is a nice restaurant. Afterwards, they talk about the restaurant and give it a rating on a scale of $0 - 10$ for the overall experience (ambiance, service, food). The guy rates it a $7$, his date a $1$. So the next day, he takes her to a nicer place. He rates it an $8$; she rates it a $2$. The next day he takes her to an even nicer place. He rates it a $9$; she rates it a $3$.

Here are the ratings: 

guy woman
7   1
8   2
9   3

The correlation is $r = 1.0$. You can decide for yourself if you think they agree on the quality of the restaurants. I suspect this relationship isn't going to last.

In essence, Pearson's correlation measures agreement with respect to the ordering of the ratings and the relative spacing between those ratings. Spearman's correlation measures the agreement on the ordering only. But people typically think of similar ratings as being at least as important as the ordering for there to be true agreement. For continuous ratings, Lin's concordance coefficient can be used as a measure of agreement that isn't subject to these flaws. For categorical ratings Cohen's kappa or Bangdawala's $B$ can be used to assess agreement instead of the chi-squared test for similar reasons.

gung - Reinstate Monica
  • 132,789
  • 81
  • 357
  • 650
  • The interpretation of the data here is tendentious. If they both agree to dates on three successive nights, that indicates strong interest on both sides. The advice is simply not to have restaurant meals every night but to try something else, perhaps to go for a walk around town instead. Otherwise +1 for flagging that correlation doesn't measure agreement; indeed you could have pressed the point that Spearman correlation is even further from a measure of agreement than Pearson correlation. – Nick Cox Mar 03 '16 at 11:51
  • Outdated sexism on the man taking a woman out: please, in 2016: they agree to go out together. – Nick Cox Mar 03 '16 at 11:59
  • @NickCox, I added explicit information on what correlations measure & why it isn't necessarily agreement. Also, in Zhao men still take women out on dates occasionally without it being considered sexist. Although sometimes they go 'dutch', & sometimes the woman pays--that's OK too. – gung - Reinstate Monica Mar 03 '16 at 14:24
  • If I could give 5 votes to @gung I would. In addition, it's hard to beat mean absolute error in many cases. – Frank Harrell Mar 03 '16 at 14:29
  • I provided a discussion of assessing agreement in Cox, N.J. 2006. Assessing agreement of measurements and predictions in geomorphology. _Geomorphology_ 76: 332-346, ISSN 0169-555X, http://dx.doi.org/10.1016/j.geomorph.2005.12.001 which some readers will be able to access via institutional libraries .http://www.sciencedirect.com/science/article/pii/S0169555X05003740 The specifics of 'in geomorphology' should not bite hard as the examples are not esoteric. – Nick Cox Mar 03 '16 at 16:21
3

This is a question for your field.

Here I comment generally on interpreting correlations. Specifically with your data, Spearman correlation might or might not be sensitive or insensitive to underlying patterns in scientifically helpful or unhelpful ways as compared with other possible correlations.

If an advisor, supervisor, mentor, experienced colleague or reviewer in your field considers "reasonable" to be, hmm, reasonable wording in a report that explains the scientific context, your data, methods and results, then well and good for you. There is nothing in the statistics to allow direct quantifications of surprise, success or otherwise by just looking at correlations.

In some fields correlations of 0.9 signal incompetent experimental technique and unpublishable results; in other fields correlations of 0.1 may flag the possibility of a real effect of interest or even of importance. For example, in many medical problems, all kinds of social or personal characteristics may have genuine but small effects on morbidity, mortality, life expectancy or other responses.

All sorts of situations are conceivable. In a paper I reviewed, and recommended for rejection, authors proposed a new measure but results in their own paper could be used to show a very high correlation with existing measures, so it was nothing but a standard idea in a slightly different guise. Here by implication authors were suggesting something new but correlation showed that it was anything but. Discretion obliges that I cast a misty net over all the details, but this was a case where strong correlation was scientific failure and a weak correlation could have been more interesting.

The vagueness of "a specific parameter" inhibits further comment; "two cell types" may allow some kinds of biologists to say more.

Nick Cox
  • 48,377
  • 8
  • 110
  • 156
  • Thanks for that! So I've added some extra details, although the crux of the matter is that in biology - including in the high-impact journal where the study I'm looking at was published /not my work/ - there is very limited understanding of statistics and mathematics - which is why we still use Spearman correlations as evidence of concordance (which they, realistically, aren't). – dvanic Mar 06 '16 at 06:29
  • The reason I asked is that if I had gotten such a result I would have made the opposite conclusion than the authors: that the two cell states share *some limited* similarity, but are mostly distinct in how they determine subcellular localisation, and that one cannot extrapolate from a cell line to the primary cells. And while deferring this to a more experienced colleague would be nice, I don't have anyone nearby who is an expert in this field - and I felt that tracking someone down for something that's not my result, but read in a paper is a bit much... – dvanic Mar 06 '16 at 06:38