How to check whether or not two regression curves from different datasets are different in R

Question

Here is an example code:

set.seed(3)

data1 = iris[sample(c(1:dim(iris)[1]), 30), ]
data2 = iris[sample(c(1:dim(iris)[1]), 50), ]

model1 = lm(Petal.Length ~ log(Petal.Width),
            data = data1)
model2 = lm(data2$Petal.Length ~ log(data2$Petal.Width))

par(mfrow = c(1, 4))
plot(data1$Petal.Width, 
     data1$Petal.Length)
points(sort(data1$Petal.Width),
       predict(model1, newdata = data1[order(data1$Petal.Width), ]),
       col = "red", 
       type = "l")

plot(data2$Petal.Width, 
     data2$Petal.Length)
points(sort(data2$Petal.Width),
       predict(model1, newdata = data2[order(data2$Petal.Width), ]),
       col = "red", 
       type = "l")


summary(model1)
summary(model2)

As in the code, I fitted two different curves from two different datasets using the simple regression method in which the explanatory variable is in logarithm form. Like in the code, the datasets have the same variables with different size.
I want to check that the two curves (model1 and model2) are statistically different or not.
How can I do it in R?

---- edited ---- I know that there are many questions about comparing regression lines, but my case is a different one from previous ones because I want to compare different models built from different data sets while other cases are to compare different models built from the same data set.

Let me state more about the context of my purpose to elaborate on the term "compare."
I made a model (the curve) for an experimental site and want to apply this model to many other sites (over the country). I acknowledge that it would not proper to apply the model to other sites where the model is not based on (the resulted curve of another site may be different from the previous one). So, I made one more curve for another study site and trying to compare the two models built from different sites.
This is why the two data sets have the same variable list. And the common points of the two data sets in the example can be ignored in term of my purpose.

Thank you!

You still have lines -- in log-petal-width. The previous posts you refer to that are about comparing lines apply directly. — Glen_b, Aug 19 '19 at 08:32
@Glen_b Thank you for your reply. Can you provide example posts that are suitable for my case? What I found were all about comparing method applicable for the model fitting phase or for models from the same size of data sets. — htlee, Aug 19 '19 at 08:46
Assuming comparison of curves means comparing the models (i.e. the Null would be all coefficients are statistically same), use [chow.test](https://www.rdocumentation.org/packages/gap/versions/1.2.1/topics/chow.test) in R. Only thing is that since you are taking sample from a dataset, there will be common data points. These data points will be repeated in the combined regression. This may give undue weight to these points. To handle this you can find out common set of data points in data1 and data2 and remove them from one of the dataset (say data1) and then use chow.test. — Dayne, Aug 19 '19 at 10:14
What about building a confidence interval for each estimation and check if the other estimation fits in it? — David, Aug 19 '19 at 11:24
This actually isn't a duplicate--it's a totally bizarre situation in which the same model is being fit to different data, rather than different models to the same data (where you can just do anova). I've been thinking about how to answer it, but I think it was closed prematurely. — Sheridan Grant, Aug 20 '19 at 01:16
@SheridanGrant: You are right in pointing out that the question isn't exactly the duplicate. However, the situation here is not very different from anova. Chow test can be used here, which is essentially a form of anova (basically an F test) only. — Dayne, Aug 20 '19 at 04:51
Only difference here is that the common points need to be emitted from one of the datasets. Otherwise there would be an artificial equivalence between the two models due to common points and it's possible that Null may not be rejected when it should ideally be. — Dayne, Aug 20 '19 at 04:57
Right--which changes the problem, since you have to refit the model once you ensure the data sets don't overlap. Even then, the "already answered" question makes no mention of the Chow test. I would love to know what scientific problem, if any, led @HoonTaek to this question... I dunno how to get the mods to reopen this though. — Sheridan Grant, Aug 20 '19 at 20:50
I appreciate every reply. I have edited my question, which may lead to reopen the discussion. — htlee, Aug 21 '19 at 06:46
@HoonTaek I think no overlapping in data makes it simpler. Just use Chow Test. It is specifically designed for this purpose only. — Dayne, Aug 22 '19 at 02:21
@Dayne Thank you for your answer. It is easy and seems suitable for my case. — htlee, Aug 22 '19 at 05:29

score 2 · Accepted Answer · edited Jun 11 '20 at 14:32

Your case is that of comparing two models with same variables but different set of data points. You can use chow.test from library gap. However, as mentioned in my comment above, you need to remove common data points from one of the data frames. Following is the code for this (and also the result I obtained):

set.seed(3) 
data1 = iris[sample(c(1:dim(iris)[1]), 30), ]
data2 = iris[sample(c(1:dim(iris)[1]), 50), ]

data1_sno=as.numeric(rownames(data1))
data2_sno=as.numeric(rownames(data2))

# next line tells you which of the data points in data2 are also in data1.
new_index=data2_sno %in% data1_sno 
data2_new=data2[!new_index,]
gap::chow.test(y1=data1$Petal.Length,x1=log(data1$Petal.Width),
           y2=data2_new$Petal.Length,x2=log(data2_new$Petal.Width))


   F value      d.f.1      d.f.2    P value 
 0.8257495  2.0000000 64.0000000  0.4425175

So the null could not be rejected, i.e. it cannot be said that the curves are indeed different.

score 2 · Answer 2 · answered Aug 19 '19 at 11:45

Why do a statistical test of this? Any test will have the usual problems of p values - that is, the result will depend partly on sample size.

When you have two data sets, you know that the same line doesn't fit both equally well (except perhaps for some constructed data sets and some very small data sets). So, graph both and look at them.

Are they different?

Then look at the predictions that each makes from the same IVs. Are they different?

And, by "different" I mean "different enough to matter to anyone".

Thank you for your reply. Surely, they will be likely to differ from each other. I think that you replied like this because I provided little information on the term "different." — htlee, Aug 22 '19 at 05:26

How to check whether or not two regression curves from different datasets are different in R

2 Answers2