3

I am developing an application to compare temperatures from two locations, in the same span of time.

For example, temperature is measured every hour for three months straight in two different cities.

What would be a good way to compare these two data sets, and to come up with a agreement measure? What I am asking for is some measure of how identical or different the temperatures are. For example, if at 11 o'clock in a certain day in Oslo it is 3 degrees and in Athens it's 26 degrees, they would be markedly different, while if the temperatures were 23 and 21 degrees, respectively, we would consider them to be closer to the same measurement for these two cities. What I desire is to compare the lists of temperatures to make a combined report of how these temperatures agree/disagree overall.

amoeba
  • 93,463
  • 28
  • 275
  • 317
romin21
  • 133
  • 4
  • 2
    Please explain what it means for two temperatures to be "similar," if it is anything other than *equal*. – whuber Apr 05 '18 at 16:55
  • 1
    All outside temperatures going to have daily cyclicality, and will look similar in that regard – Aksakal Apr 05 '18 at 16:57
  • @Aksakal Point taken. I added an edit to the question. – romin21 Apr 05 '18 at 19:01
  • 1
    @whuber Thanks for pointing that out. I edited the question. – romin21 Apr 05 '18 at 19:01
  • 1
    did you try root mean squared difference between temperatures at the same time of the day? – Aksakal Apr 05 '18 at 19:04
  • 5
    As you're aware, similarity or more precisely dissimilarity between temperatures is measured as a difference. (That may seem obvious, but there are many variables where it's arguable that a ratio or something else is a better scale for comparison.) So far so good, but what else are you asking? The more places differ in location, the more cycles of daily and seasonal variation will differ, all the way down to phase differences as well as amplitude differences. More positively, additive shift is often a very good first approximation for nearby places. – Nick Cox Apr 06 '18 at 11:52
  • Weather is naturally more than a matter of temperature. It's not clear what that is crucial to your project. – Nick Cox Apr 06 '18 at 11:54
  • 5
    "What would be a good way"...clearly there are multiple ways emerging, so can you specify what your definition of "good" is? Otherwise I'm afraid the thread is going to be closed as too open-ended. Though maybe it seems people are spending more time evaluating your question than answering it :-( – rolando2 Apr 06 '18 at 15:05
  • Similarity would usually be measured by agreement, and, that would not be the best choice here. Better would be ARIMA or other regression. Better yet would be to have multiple years of data to make a lookup table of temperature conversion. – Carl Apr 08 '18 at 03:48
  • You can view the edits by left clicking on the red text "edited...ago" above my logo. If you don't like the edit, and wish to keep the prior version, look at the second (vertical) gray bar for the "rollback" button. – Carl Apr 09 '18 at 02:16
  • 1
    I want to add to earlier comments and existing suggestions by suggesting that quoting mean temperatures and the difference between them is the most direct way of quantifying (dis)similarity, especially for non-technical audiences. – Nick Cox Apr 09 '18 at 10:09
  • @NickCox I do not see what is currently wrong with this question, do you? – Carl Apr 11 '18 at 15:26
  • 1
    I am not trying to find fault (least of all with @Carl who is only trying to help) but I didn't vote to reopen. I am now focusing on how this may or may not be useful to future readers. I won't argue backwards from there being an accepted answer to the question being clear. The question has to stand on its own two feet and seem clear. Future readers wanting to know how to measure similarity would also be asking a vague general question if they asked the same question: it needs to be made precise. I work with environmental data and have even published a little on comparing temperatures! – Nick Cox Apr 11 '18 at 15:35
  • 1
    (ctd) None of that stops the thread being indirectly helpful -- but is the question clear? Sorry, still no. – Nick Cox Apr 11 '18 at 15:39
  • @NickCox I cannot imagine how to pose the question so that it is more clear. Let us invert this and ask you how to pose a question whose answer is Lin's concordance correlation coefficient. For my own sake the question seems natural and realistic. Consider the truly stupid question, e.g., https://stats.stackexchange.com/q/4768/99274. We suspend [disbelief](https://stats.stackexchange.com/a/339941/99274) to answer those, even though such questions are worthless. Can you edit this question to clarify it? – Carl Apr 11 '18 at 17:39
  • 1
    @Carl. No, no, no, as far as I am concerned. First, whether other questions have been handled badly is immaterial to this one being handled well. Second, it's not any member's job to decide what a question should have or might have been. Being a moderator or even having a moderate reputation doesn't change that. At most, the OP proposes, the community may dispose, but the community doesn't change the question. At most, one can make edits to trivia and trivia are what don't discernibly change the question. – Nick Cox Apr 11 '18 at 18:01
  • 1
    https://stats.meta.stackexchange.com/questions/2810/editing-questions-dos-and-donts skates around this, but I think it is clear even if tacit: no member but the OP has privileged insight into what an OP's question really is, even if it turns out that they understand it well. That doesn't rule out suggesting a better or different question. – Nick Cox Apr 11 '18 at 18:07
  • 1
    We're stuck here and it's far from ideal, but only the OP's rewording to make the question match the answer can salvage that, and I've got to say that sounds like an unfair and contrived request. I've answered threads here with "use concordance correlation" as the focus of an answer, so I don't see need to do that again myself. – Nick Cox Apr 11 '18 at 18:22
  • @NickCox This question was closed, then reopened then closed and is 2/5 reopened. OK, so there are two point of view here. I see "In what ways can we compare the temperature time series in two different geographic locations?" Sure the OP had an inclination toward a measure of agreement, but I do not see that as central to the question because the OP lacks experience and implied discarding that preference if needed. What is unclear for you in this question? This goes beyond curiosity for me; I just don't get it. – Carl Apr 11 '18 at 18:34
  • 1
    @Carl I am just one of several people who have commented and/or voted one way or the other. I am unhappy as you are unhappy (although not as much) about the status of the thread, but pretending to change my mind to make you happier is (I trust) not what you want me to do. Further, I am queasy about discussing how much an OP knows as any kind of basis for what we do with a question. This is all Meta stuff again.... – Nick Cox Apr 11 '18 at 18:51
  • Let us [continue this discussion in chat](https://chat.stackexchange.com/rooms/75856/discussion-between-carl-and-nick-cox). – Carl Apr 11 '18 at 19:12
  • Sorry, no: I really have nothing else to add. – Nick Cox Apr 11 '18 at 22:42

1 Answers1

-2
Edit after OP edit and user comments, downvoters should reconsider their opinions, please. This is currently an accepted answer, one user has removed his downvote. Downvoting without leaving a comment is not constructive at this point.

I did not at first see the OP edit while I was answering. There are no really good answers to the question. What there is is a simplest answer to the question; to look up or calculate directly the mean temperatures in both locations, and then subtract them to find an expected difference in temperature at any particular time on any particular day. Mean temperatures require approximately 10 years of data to establish that mean, and or a time series like ARIMA. Three months of data from a single year would not be as useful. Next most useful would be regression, next most useful would be correlation. Agreement would not be a very useful measure because it is not information rich enough to explain how the temperatures covary. Details follow.

A squared correlation coefficient is called the coefficient of determination. The coefficient of determination, symbolically $r^2$, would give the fraction of variance between temperatures that is mutually explained, and occurs at the same time.

I found daily average temperatures compiled during 2010 for Augusta, Maine and Miami, Florida. Here is a chart:

enter image description here

The correlation is $r=0.990128135$ and the coefficient of determination is $r^2=0.980353723$. The 2261 km distance between those cities is similar to the 2607 km Athens to Oslo distance. Like Athens and Oslo, Miami and Augusta are in the same time zones, meaning that the daily temperature highs and lows will be correlated positively so that if hourly data were readily available, that would correlate. Now if unique measurements rather than averages were included, that may lower the coefficient of determination, nonetheless the correlations are higher than even I suspected. Note on the chart that there is a doubled hysteresis loop presumably largely due to the difference in timing of temperature modification by the proximity of the Labrador current in Maine, and the Gulf stream current for Miami.

One can convert between Miami and Augusta temperatures using bivariate regression, i.e., using Passing-Bablok invertible (unlike OLS) regression $F(Miami)= 0.3203 F(Augusta) + 62.38$. Moreover, the chart shows a quite smooth progression of temperatures along a curvilinear closed path in a one year loop. Now on any particular day of the year, or any particular time of day, there is a difference in temperature between the two sites that is on average fairly stable. If you want to know what it is, calculate or look up that difference. For example, for @gung, here are those average annual temperature differences between the two sites in degrees F compiled on days 0-365 of 2010.

enter image description here

If you want to measure agreement then Lin's concordance correlation coefficient would be one alternative, but not a very useful one (see below). This is similar to an intraclass correlation and measures how distant an agreement is from the identity line $y=x$. Agreement is not the right question to ask of a time series model, like an

 ARIMAProcess[1.8589*10^-17, {0.05033}, 2, {-1.58602, 0.747954}, 0.00377009].

See ARIMAProcess.

Here are average monthly temperatures from Oslo, Sweden and Melbourne, Australia in degrees centigrade with correlation and coefficient of determination. For the OP, coefficient of determination in Excel is =correl(cell:cell,cell:cell)^2.

     Melbourne  Oslo
Jan      21      -3
Feb      21      -3
Mar      19       2
Apr      17       5
May      14      12
Jun      11      16
Jul      10      18
Aug      11      16
Sep      13      12
Oct      15       7
Nov      17       2
Dec      19      -3

$r=-0.981726823$ and $r^2=0.963787555$. Note how high the negative correlation is between two points on Earth that could not be further apart.

OLS regression shows this

 Term       |Coefficient |95% CI            |SE     |t statistic |DF |p
 Intercept  |37.77       |33.42 to 42.13    |1.955  | 19.32      |10 |<0.0001
 Slope      |-1.98       |-2.25 to -1.71    |0.121  |-16.31      |10 |<0.0001

 Oslo = 37.77 - 1.98 Melbourne                      

Let us compare those results with Lin's concordance:

$$r_c=\frac{2 \rho \sigma _x \sigma _y}{\sigma _x^2+\left(\mu _x-\mu _y\right){}^2+\sigma _y^2}\,,$$

which is $r_c=0.148498 $ for the Augusta to Miami value, and $r_c=-0.38624$ for the Oslo to Melbourne concordance. So in the first case, the concordance is weak, and in the second case there is a slightly stronger disagreement. This doesn't go very far in explaining the differences in temperature.

Carl
  • 11,532
  • 7
  • 45
  • 102
  • 4
    (-1) Surely however closely correlated the hourly temperatures may be in Athens & Oslo, they're not *similar*. – Scortchi - Reinstate Monica Apr 07 '18 at 11:47
  • @Scortchi Extensive edits performed, downvotes remain, please reconsider your downvote. I am frankly appalled and am reconsidering my participation on this site, too negative for my tastes. – Carl Apr 08 '18 at 00:19
  • 2
    I don't think this merits 3 downvotes, although I don't think it's right, either. It is a valiant effort. Carl, you suggest $r^2$; the OP thought about the difference b/t temps. Consider the example you illustrate: $r^2=.98$, which sounds pretty similar, but consider that when A=19, M=68, a 49 degree difference, & when A=71, M=84, a 13 degree difference. Given the OP seems to want temps to have a small constant difference, do those seem as similar as $r=.98$ intuitively implies? They wouldn't to me. – gung - Reinstate Monica Apr 08 '18 at 01:24
  • 1
    For a fuller understanding of my concern about this, it may help to read my answer here: [Does Spearman's $r=0.38$ indicate agreement?](https://stats.stackexchange.com/a/199714/7290) – gung - Reinstate Monica Apr 08 '18 at 01:25
  • @gung Thanks, I know, but agreement would be not very useful either as the differences are predictable, but a time series, not a constant. – Carl Apr 08 '18 at 02:04
  • 4
    I didn't downvote but sorry, I won't upvote this either. I am still unclear what the OP wants and won't answer an unclear question. . Trying to guess what the question is and then answering it is fine, but does not justify remarks like "peanut gallery" or "vindictive" directed at people who take a different view. Downvoting answers you think is wrong is allowed. If I was obliged to guess what the OP wants, I think it's not much more than a mean or median difference in temperature. I don't think $R^2$ or correlation or concordance correlation is likely to help at all, but that is just a guess. – Nick Cox Apr 08 '18 at 08:19
  • 3
    Much better now IMO. Lin's concordance is perhaps the type of similarity index the OP's looking for, but I think you show how the reduction of two data series to a single number isn't very informative about the nature of the differences between them. – Scortchi - Reinstate Monica Apr 08 '18 at 11:09
  • 1
    @Carl sometimes I think I need a degree in Law to post a question in this site. Thank you, your answer answered my question, although many people found it unclear. – romin21 Apr 08 '18 at 12:16
  • @hms Your question was both clear and unpretentious. If you want, I can attempt a bit of a rewrite to try to get it off of a hold, but with no guarantees. Some of them, however, do have hearts of gold, speaking of which, thanks for the question, it took work to think it through and although the work is is own reward, it's nice to have it appreciated, at least occasionally, rather than just reacted to reflexly. – Carl Apr 08 '18 at 21:08
  • @NickCox Thanks, the OP just wanted something to quantify concordance or intraclass correlation, but apparently lacked the vocabulary to express that request in standard form. Understanding that required thought, and multiple attempts on my part. Whilst I was doing that the negativity to both the question and its answer was off putting, insulting really to both the OP and your's truly. People can take any view they want on this site, and trust me, it is too often ill-considered. However, there are people who help as well, and I work for them. – Carl Apr 08 '18 at 21:27
  • @Scortchi Thanks, you had constructive criticism, and that helped. The downvotes didn't and having the question, which after due consideration is clear to me, on hold, I do not think meritorious. – Carl Apr 08 '18 at 21:30
  • @NickCox The "peanut gallery" comment is removed. Downvoting is not as useful to a democracy as either qualified expert opinion or upvoting, read [this](https://stats.meta.stackexchange.com/a/4462/99274) please. – Carl Apr 08 '18 at 22:02
  • @gung Thanks. Your opinion helped me to appreciate that a measure quantifying agreement was central to the OP question, which allowed me to extensively rewrite my answer. – Carl Apr 08 '18 at 22:07