3

I am trying to simulate two data set for multiple linear regression. I want one data which is independent and identically distributed and the other is not. So far, I have done the following:

x1 <- rnorm(10000,11,.5)
x2<-rnorm(10000,5,95)
x3 <- rnorm(10000,5,.5)


b1 <- 0.1
b2 <- 0.9
b3 <- 0.6
sigma <- 0.4

eps <- rnorm(x1,10000,sigma)
y <- b1*x1  + b2*x2  + b3*x3  +eps
Y<-as.matrix(y)
X<-cbind(as.matrix(x1),as.matrix(x2),as.matrix(x3))

Does this satisfy the iid case? How to generate data that is not iid? For non iid case I am thinking of having autocorrelation.

Waqas
  • 386
  • 3
  • 15

2 Answers2

1

Does this satisfy the iid case?

Yes.

How to generate data that is not iid?

For example, data is generated from a mixture of Gaussian. When generate samples, we first sample the "membership", the according to the membership, we sample different Gaussian distribution.

In addition, Note that, linear regression does not have the assumption on the distribution on $X$, only on the residual to be normal distribution ed with constant variance, because of Gauss–Markov theorem.

In other words, your simulation satisfy linear regression assumptions and the simulation has stronger assumptions on distribution of $X$.

In your comment, you asked how to generate not IID on residuals, I think you have a confusion on the assumptions. IID is on data, and normal distribution is on residuals.

Related topics

Regression when the OLS residuals are not normally distributed

Why is the normality of residuals "barely important at all" for the purpose of estimating the regression line?

Assumptions of multiple regression: how is normality assumption different from constant variance assumption?

What is a complete list of the usual assumptions for linear regression?

How does linear regression use the normal distribution?

Haitao Du
  • 32,885
  • 17
  • 118
  • 213
  • How can I generate the data which violates this assumption on residuals? – Waqas Mar 26 '18 at 13:21
  • I was referring to your comment "In other words, your simulation satisfy linear regression assumptions and the simulation has stronger assumptions on distribution of X". I just wanted residual not normal, I did not use the word iid and residuals. Apologies for creating confusion – Waqas Mar 26 '18 at 13:32
1

To generate correlated independent variables, you could do something like this: 

x1 <- rnorm(10^5,0,1)
x2 <- rnorm(10^5,0,1)
x2 <- 0.4*x1 + sqrt(1-0.4^2)*x2
x1 <- 11 + 0.5*x1
x2 <- 5 + 95*x2
There are ways to do something comparable with 3 or more variables, as well...though I will stop here for now.

For autocorrelation, a similar strategy can be used: 

sigma <- 0.4
eps <- rnorm(10^5,0,1)
for(i in 2:10^5) eps[i] <- 0.15 * eps[i-1] + sqrt(1-0.15^2)*eps[i]
eps <- sigma * eps

Hope this helps with your simulation.

Community
  • 1
Gregg H
  • 3,571
  • 6
  • 25